The stock market won't suffer if the Federal Reserve tapers its quantitative easing (QE), which many economists expect soon, says stock market guru Jeremy Siegel, finance professor at the University of Pennsylvania's Wharton School.

"Historically bull markets don’t end when the Fed starts tightening," he told CNBC. "It's when the Fed ends tightening, when there's inflationary pressure--that's when you really have to worry."

Siegel also takes issue with those who think the Fed is responsible for the stock market's surge to record highs this year.

Editor’s Note: Retired Americans Slammed by Obama’s Redistribution Plans

"I think one of the biggest myths on Wall Street is that it's quantitative easing that's driving the market," Siegel said.

"In 2012, operating earnings for the [Standard & Poor's 500 Index] were $96.82 [a share]. The current estimate for 2013 is $105.72. There was a good earnings jump this year."

And Siegel thinks the market has room to rise further, as economic growth accelerates above 3 percent in 2014.

"My fair value on the Dow is 18,000." That would represent a 13.8 percent rise from the Dow Jones Industrial Average's close of 15,822 Thursday.

Stocks dropped for the fifth straight day Thursday, as strong economic data led to speculation the Fed will taper as early as December.

"There’s angst in the short run, but I think it’s only positive in the long run that the Fed begin to taper and extricate itself from being the ultimate market maker," Matthew Kaufler, a portfolio manager at Federated Investors, told Bloomberg.

Editor’s Note: Retired Americans Slammed by Obama’s Redistribution Plans

Related Stories:

• Yale's Shiller: No Bubble in Housing, But Keep an Eye on Stocks
• David Stockman: Fed Is Creating the 'Mother of All Bubbles'