Is Eddie Lampert's vast empire starting to crack?

Investors are starting to lose faith in the chief executive of Sears (SHLD) and prominent hedge fund manager. They're fleeing his fund in droves, and as a result he's lost his majority stake in the struggling retailer.

For a while, Lampert was one of the best in the business. His fund, ESL Investments, boasted annualized returns of more than 20 percent for 20 years, The Wall Street Journal reports. But then the recession hit, and the fund suffered a 33 percent loss in 2008, rebounded with gains of 55 percent and 16 percent in 2009 and 2010, and sagged back to a 12 percent loss in 2011. Although such volatile returns have been common on Wall Street in an economic recovery characterized by fits and spurts, those losses were enough to make some of Lampert's investors want to bail.

Now, a group of clients that came over through Goldman Sachs (GS) have asked Lampert to return the roughly $3.5 million they invested with him in 2007, The Journal reports.

But instead of giving those investors cash back, Lampert is paying them partly with Sears stock. He's returned some 7.4 million shares to them, in fact. As a result, ESL now only owns 48.4 percent of Sears' shares, down from 55 percent from as early as October, Bloomberg reports.

So were investors happy to get paid in Sears stock? The answer can be seen in the company's plummeting share price last week. The stock started a week ago in the $64 range -- the day before Lampert disclosed the fund redemptions -- and closed Friday at $48. And lurking behind that 25 percent drop is a harsh reality: Many of the investors who get paid in Sears shares will turn around and sell.

And why shouldn't they? Sears is set to mark three straight years of losses. The retail environment is "a vicious minefield," TheStreet's Jim Cramer wrote last week. "I don't know anyone who has been able to navigate it. Perhaps the best thing to do is to just stay away from the darned thing until we get some clarity."

And that's exactly what many investors will do with Sears, leaving Lampert on the outs. To be sure, he still has some $2.5 billion in outside investor money left to play with, and his legendary investing prowess may help him work wonders with that money in a booming market.

But for now, Lampert's empire looks like it's just starting to crack. November sales at stores open for longer than a year appear to be down by a percentage in the mid-to-high single digits, an analyst at Cleveland Research told Bloomberg, while Sears continues to lose market share in the important home-appliance market.

Can Lampert turn things around? Nothing spooks investors like a mass exit, and it will take an extraordinary amount of work on his part to calm the waters.