U.S. brokerages suffered through a difficult third quarter marked by lackluster customer trading, lofty recruiting expenses and anxious clients clinging to conservative investments that are less lucrative for Wall Street.

Bank of America kicks off the wealth management earnings season on Tuesday when it posts Merrill Lynch results, followed by Morgan Stanley Smith Barney the next day. Based on reported trading activity and wild swings in the financial market, analysts are not getting their hopes up.

"It's been a tough quarter all around," said Steve Stelmach, a brokerage analyst at FBR Capital Markets. "Investors recognize that and their expectations are low."

Analysts are keen to see what progress has been made at Morgan Stanley Smith Barney, as it melds two of Wall Street's largest brokerages, and Merrill Lynch, where the once-independent Thundering Herd is integrated with Bank of America's consumer and corporate banking businesses.

Retail investors stung by years of losses stayed on the sidelines in the third quarter, choosing to keep their money largely in cash or short-term bonds. These investments provide little revenue for brokerage firms.

Money moved out of equity mutual funds every week during the quarter, the Investment Company Institute said.

Charles Schwab, one of the largest U.S. brokerages, reported a 24 percent drop in third-quarter trading revenue as clients steered clear of the market.

"Perhaps more important than the third quarter was the weak client activity in September," Bernstein Research analyst Brad Hintz said of Schwab's results in a client note.

Investors usually jump back into the market after Labor Day, which marks the end of the summer vacation season. This year, Hintz said, "both (asset) flows and trading was flat in September versus August, a sign that retail clients remain paralyzed."

Further undermining confidence was the May 6 "flash crash," when the Dow Jones industrial average plunged nearly 1,000 points in a matter of minutes as computer-driven trading sent shares reeling for no apparent reason.

Morgan Stanley in July scaled back the asset and profit targets laid out for Morgan Stanley Smith Barney in February.

The unit, a joint venture between Morgan Stanley and Citigroup, reported $5.5 billion of second-quarter net withdrawals.

"Implicit in the targets was a better market environment," said FBR's Stelmach.

Across Wall Street, brokerages also found it more difficult and more expensive to expand through recruiting, a necessary evil to quickly attract more assets and increase revenue.

During the summer, Morgan Stanley offered adviser recruits an extra 20 percent cash bonus if they could transfer half of their assets to Morgan Stanley by the end of the third quarter.

The firm recently extended the offer until the end of the year.

Merrill Lynch has also extended its top recruitment package to advisers who work at some regional firms. Previously, the deal was reserved for advisers from the Big 4 brokerages.

Despite such sweeteners, adviser movement has remained subdued, say recruiters. Most of the industry's top producers are tied to their firms by generous retention packages doled out in the wake of the 2008 financial crisis.

"I don't think we'll see a big jump," said Scott Smith, a research analyst at Cerulli Associates. "I think the (brokerages) are making an aggressive effort just to tread water."

Indeed, if a firm boosts headcount significantly while recruiting deals are so rich, investors may be concerned the firm is paying too much, said Stelmach.

In the second quarter, UBS said it was still feeling the after-effects of massive recruitment packages the firm offered in late 2008 and early 2009. In the third quarter, UBS Wealth Management Americas paid out $1.04 for every dollar of revenue it took in, prompting a series of cost-cutting measures.