LONDON (AP) — Former U.K. House of Commons Speaker John Bercow, who gained international fame refereeing the country’s political drama over Brexit, said Sunday he has quit the governing Conservatives to join the opposition Labour Party.

Bercow was a Conservative lawmaker for 12 years until being elected in 2009 to the neutral role of Speaker, responsible for running House of Commons business and interpreting Parliament's rules of procedure.

He enraged the Conservative governments of Prime Minister Theresa May and her successor, Boris Johnson, by giving legislators chances to stymie proposed legislation about the U.K.’s exit from the European Union.

He became a hero to opponents of Brexit and a villain to its supporters, and was recognized around the world for his loud ties and even louder calls of “Order!”

Bercow also was accused during his career of bullying members of his staff, claims he denied. He stepped down in 2019.

Bercow is a strong critic of Johnson, a champion of Brexit whose rise has coincided with an exodus of pro-EU politicians from the Conservative Party. Bercow told The Observer newspaper that the Conservatives under Johnson had become “reactionary, populist, nationalistic and sometimes even xenophobic.”

“The conclusion I have reached is that this government needs to be replaced,” he said. “The reality is that the Labour Party is the only vehicle that can achieve that objective.”

Justice Secretary Robert Buckland rejected the characterization of the party as xenophobic, and said Bercow’s decision to forgo political neutrality “actually has the effect of diminishing the force of his voice in politics.”