(Adds background, quotes)

By Patricia Zengerle

WASHINGTON, March 29 (Reuters) - U.S. President Barack Obama challenged Congress on Thursday to repeal billions of dollars in tax breaks for the biggest U.S. energy companies, saying they are raking in record profits while Americans struggle with higher gasoline prices.

"Today, members of Congress have a simple choice to make. They can stand with big oil companies, or they can stand with the American people," he said at an event in the White House Rose Garden highlighting one of the hottest issues in the U.S. presidential campaign.

"Last year, the three biggest U.S. oil companies took home more than $80 billion in profit. Exxon ( ) pocketed nearly $4.7 million every hour," Obama said.

Instead of rewarding oil companies, the United States should "double down" on clean energy, Obama said in his speech.

"Keep in mind, we can't just drill our way out of this problem," Obama said before a crowd of about 100 people, including representatives of environmental groups and Americans affected by the rising cost of gasoline.

The U.S. Senate was to vote on Thursday on legislation backed by Democrats that would eliminate more than $24 billion in tax deductions granted to big oil companies over the next decade. The measure would also extend some tax breaks intended to foster clean energy initiatives. (Additional reporting by Caren Bohan and Samson Reiny; Editing by Vicki Allen and Sandra Maler)