The Internal Revenue Service is losing billions of dollars a year to tax return fraud committed through identity theft, reveals a new report from a watchdog agency for the IRS.

"Identity theft continues to be a serious problem with devastating consequences for taxpayers and an enormous impact on tax administration," said J. Russell George, Treasury Inspector General for Tax Administration. "Undetected tax refund fraud results in significant unintended Federal outlays and erodes taxpayer confidence in the Federal tax system."

The watchdog's analysis of 2012 tax returns found that the IRS missed about 1.1 million tax returns that were filed using a Social Security number with the same characteristics of IRS-confirmed identity theft tax returns.

Editor’s Note: Retirees Slammed with 85% Pay Cut (New Video)

Potentially fraudulent tax refunds totaled about $3.6 billion in 2011. That's down by $1.6 billion from the $5.2 billion the watchdog reported the previous year. But the IRS still has plenty of room for improvement, it says.

Many of the same problems the inspector general previously cited — delayed access to third-party income and withholding information and multiple tax refunds deposited to the same bank account — continue to give identity thieves opportunities.

"Many individuals who are victims of this type of identity theft may be unaware that their identity has been used to file fraudulent tax returns," its report states. "The individuals victimized are typically those who are not required to file a tax return."

The inspector general recommends deactivating ITINs assigned to people before 2013 who are no longer required to file tax returns. It also recommends continuing to analyze characteristics of identity theft on returns. The inspector general says the IRS has agreed with those recommendations.

The inspector general also says the IRS takes too long to help identity theft victims. It took an average of 312 days to resolve tax-related identity theft cases.

Identity theft affected 1.2 million taxpayers in 2012 and another 1.6 million in just the first six months of 2013.

"Identity theft is a growing epidemic, and I continue to be troubled by the lengthy case processing delays and tax account errors experienced by victims of tax-related identity theft," George said.

The IRS says it now has more than 3,000 employees working on identity theft cases, more than twice as many as last year. It has also trained more than 35,000 employees to help taxpayers victimized by identify theft.

Editor’s Note: Retirees Slammed with 85% Pay Cut (New Video)

Related Articles:

• Report: IRS Refunded $4 Billion to Identity Thieves
• ID Theft Expert Sileo: Easiest Way to Protect Yourself Is to ‘Freeze Your Credit’