Former Treasury Secretary Larry Summers says the dismal start to financial markets this year and fears about China should keep the Federal Reserve from raising interest rates four times in 2016, as policymakers had forecast last month.

Summers, who also was an economic adviser to President Barack Obama and served as Treasury secretary under Bill Clinton, also warned that psychology in volatile markets can change quickly.

"Just as I thought it was a mistake to overreact to bits of strength in the middle of last fall, there's also a danger to overreacting to the weakness at present," Summers, now a Harvard professor, told CNBC.

Fed policymakers meet on Tuesday and Wednesday for the first time since raising interest rates in December. Roughly $2.5 trillion of stock market value wiped out in the past three weeks and a possible consumer pullback could throw the Fed off its course of gradual interest rate hikes, Reuters reports.

Policymakers continue to argue that the threat will pass, but the risk that the selloff will hit the main engine of U.S. economic growth — household spending — gets bigger the longer markets remain depressed.

Fed research and other studies estimate that up to 6 percent of any drop in household net worth gets passed through and results in less spending. It means that unless the market recovers soon, upwards of $150 billion in consumption will be lost in coming months — a drag of close to 1 percent of gross domestic product.

"I've thought consistently that it was not a confident bet that the economy could withstand four rate increases this year, and continue to growth robustly and continue to provide support for a very weak global economy," Summers said. "Certainly the way markets have moved this year has done nothing but support the view."

"The markets have never believed the Fed on the Fed's expansion [on rates]," he argued. "I think the Fed has quite been very unwise when it has criticized markets for not believing it because ... markets reflect the collective judgment of a number of people. It seems to me there was substantial grounds for concern," Summers said.

However, Newsmax Finance Insider Hans Parisis disagrees, forecasting four rate hikes.

“I think investors shouldn’t be surprised if the Fed would stick to its planned four rate hikes of 0.25 percent each for this year, which would really surprise the markets,” he predicts. “I personally do not expect the FOMC to signal they will place their planned rate hikes for 2016 on hold.”

(Newsmax wire services contributed to this report).

Related Stories:

• Larry Summers: Preparing for the Next Recession
• Harvard's Larry Summers: Global Economic Danger Most Severe Since 2008