More and more consumer spending is coming from the wealthy, easing the middle class out of the picture.

Many big stores and restaurants are either trying to attract rich customers with a broader array of upscale goods and services, or focusing on bargain-basement prices to draw the growing number of less wealthy consumers, The New York Times reports.

"As a retailer or restaurant chain, if you're not at the really high level or the low level, that's a tough place to be," John Maxwell, head of the global retail and consumer practice at PricewaterhouseCoopers, told The Times. "You don't want to be stuck in the middle."

Editor’s Note: 250% Gains Bagged Using Secret Calendar (See Video)

In 2012, the top 5 percent of earners accounted for 38 percent of U.S. consumption, up from 28 percent in 1995, according to economists Steven Fazzari of Washington University and Barry Cynamon of the St. Louis Federal Reserve Bank.

Fazzari sees the trend as worrisome.

"It's going to be hard to maintain strong economic growth with such a large proportion of the population falling behind," he told The Times. "We might be able to muddle along, but can we really recover?"

A recent Pew Research Center/USA Today survey found that fewer Americans view themselves as middle class than in the past. A total of 44 percent now identify themselves as middle class, down from 53 percent in 2008, during the first months of the Great Recession.

Meanwhile, 40 percent of Americans now think they're lower class, up from 25 percent in 2008, and 15 percent view themselves as upper class, down from 21 percent.

Editor’s Note: 250% Gains Bagged Using Secret Calendar (See Video)

Related Stories:

• Americans Grapple With Income Inequality
• Morici: Inequality Is Replacing the American Dream