U.S. mortgage finance giants Fannie Mae and Freddie Mac may have suffered more than $3 billion in losses due to manipulation of the benchmark interest rate known as Libor, according to an internal memo by a federal watchdog.

The estimate was provided in a memo that was sent to Freddie and Fannie's regulator, the Federal Housing Finance Agency, by its inspector general. Reuters obtained a copy of the memo. The watchdog urged the regulator to consider whether the losses warranted a lawsuit against the banks that set Libor.

"We conducted a preliminary analysis of potential Libor-related losses at Fannie and Freddie and shared that with FHFA, recommending that they conduct a thorough review of the issue," a spokeswoman for the inspector general's office said when asked about the memo. "FHFA agreed to study the matter further."

A FHFA spokeswoman said the regulator "has not substantiated any particular Libor-related losses for Fannie Mae and Freddie Mac," and the regulator "has not made any determination regarding legal action."

"We continue to evaluate issues associated with Libor and monitor Libor-related developments, recognizing that other Federal agencies are also involved in related matters," she added.

Dozens of U.S. and European banks are under scrutiny for allegedly rigging the London interbank offered rate, known as Libor, which has an impact on borrowing costs throughout the global economy.

Fannie Mae and Freddie Mac use Libor for their investments on mortgage bonds and swaps, both popular floating-rate financial instruments. The two companies, which have received almost $190 billion in taxpayer aid since they were seized by the government in 2008, might have lost money if Libor was artificially depressed on mortgage debt and assets they hold in their portfolios.

Swiss bank UBS on Wednesday agreed to a $1.5 billion fine, the second-largest ever levied on a bank. after admitting to fraud and bribery as the Libor scandal deepened.

Libor is intended to measure the rate at which banks lend to one another and is used as a benchmark to set borrowing costs on financial instruments, including derivatives and mortgages.

Fannie Mae and Freddie Mac are the two biggest players in the U.S. mortgage market. They provide a steady stream of funds by purchasing mortgages from lenders and either holding them or repackaging them as securities, which they sell to investors with a guarantee.

The 14-page internal memo said the loss estimate compiled by the federal watchdog was based on Fannie Mae and Freddie Mac's public financial statements and historical interest rate data.