The government agents "undertook illegal tactics" to prosecute Stanford and "engaged in unfair, abusive law enforcement methods and tactics" that left him broke and unable to properly defend himself, according to the lawsuit filed in federal court in Houston late Wednesday.

The lawsuit seeking return of his assets comes two years after the U.S. Securities and Exchange Commission filed civil charges against the former jet-setting Texas financier.

Stanford is accused of running a $7 billion Ponzi scheme centered on fraudulent certificates of deposit issued by Stanford International Bank in Antigua. He has pleaded not guilty to all charges in a 21-count indictment.

"Mr. Stanford was literally left with only the suit he was wearing at the time of the SEC Agents and U.S. Marshals seizure of property ..." the lawsuit said.

A spokeswoman for the U.S. Attorney in Houston declined to comment on the lawsuit, as did a spokesman for the U.S. Securities and Exchange Commission.

U.S. District Judge David Hittner, who ruled that Stanford is incompetent to stand trial in his current mental state, has also declared the former billionaire indigent.

Stanford, 60, is in the process of being transferred to a prison hospital for treatment of addiction to a powerful anti-anxiety medication he was prescribed while jailed.

The Bureau of Prisons website lists his current location as a federal transfer center in Oklahoma.

He may end up at a U.S. prison hospital in Butner, North Carolina. That is the same federal facility where Bernie Madoff is serving a 150-year sentence for conducting the largest financial fraud in history.

The case is R. Allen Stanford v Stephen Korotosh et al, U.S. District Court for the Southern District of Texas, No. 11-00582.