"What we have done is ensconce the biggest banks as if they are going to be around 50 or 100 years from now," Bullard said, a fact that may discourage innovation and leave regulations out of step with changes in the financial industry.

Federal agencies have designated a group of large banks as systemically important and subject to stricter oversight and capital rules, and forced them to prepare "living wills" to prepare for possible failure without need of a taxpayer bailout. But many have argued those firms still remain too big to fail, and would draw government support if they falter.

Bullard said he agreed the problem has not been resolved, and endorsed Minneapolis Fed President Neel Kashkari's decision to launch research on what Kashkari referred to as "transformational" ideas about how to deal with the largest financial firms.

After helping craft plans as a Treasury Department official to save the big banks during the financial crisis, Kashkari now thinks they should be broken up.

Bullard referred to other possible benefits, citing the wave of innovation triggered when AT&T's telecommunications monopoly was ended.

If banks remain large and heavily regulated, "how much innovation are you going to get?" Bullard said on CNBC. "We are writing all these laws about what they are doing today without much thought about what they do tomorrow."