Four of the largest U.S. mortgage servicers failed to adhere to some of the new standards set out in a landmark settlement meant to help consumers facing foreclosure, a court-appointed monitor said.

Bank of America, Citigroup, JPMorgan Chase and Wells Fargo need to make improvements in handling loan modifications, said Joseph Smith, who oversees the $25 billion settlement between banks and the U.S. government.

One of the main complaints documented by Smith's report was that banks failed to ensure a single point of contact is provided for struggling borrowers.

Smith on Wednesday issued his first report on how the financial firms are complying with 304 new rules to protect consumers as part of the settlement to correct foreclosure abuses.

Under the deal reached in February 2012 with 49 states and federal agencies, banks must implement new servicing rules and stricter oversight of foreclosure processing, provide some consumers cash payments and promise mortgage relief to help borrowers.

Smith tested the banks on 29 metrics to see whether the banks are adhering to the new servicing standards. Ally Financial was the only bank that passed all of them, the report said.

"There is still work to be done," Smith said. "The banks are now working to correct these errors and will be tested again to determine their level of improvement."

Consumer advocates have said banks are behind in complying with the settlement. New York Attorney General Eric Schneiderman announced in May that he would seek enforcement actions against Wells Fargo and Bank of America for problems servicing applications for loan modifications.

"These flagrant violations put homeowners in New York and across the nation at greater risk of foreclosure," Schneiderman said in a statement on Wednesday.

Bank of America failed in two areas, including loan modification document collection, according to Smith's report. Wells Fargo also failed in that area.

JPMorgan had problems with force-placed insurance and with timelines for notifying customers about loan modification decisions. Citi had three problem areas, including timelines for collecting documents on short sales and loan modifications.

A spokesman for Wells Fargo said the bank has a plan in place to improve its response to loan modification applications, and internal reviews show it is now meeting those requirements. Citi also said it was correcting flaws under Smith's direction.

Bank of America said in a statement that neither of its mistakes "resulted in inaccurate foreclosures or improper loan modification denials." The bank said it had resolved one problem area and would soon be in compliance on the other.

Amy Bonitatibus, a spokeswoman for JPMorgan, said they are "working to let customers know more quickly about modification decisions" and are also reviewing the timing of refunds for customers on insurance premiums.

Shaun Donovan, secretary of the Department of Housing and Urban Development, said on Wednesday that the servicers have improved their performance since ending the practice of so-called robo signing when banks filed foreclosure paperwork without proper review. The five banks also stopped charging borrowers a fee to process a loan modification request.

"Unfortunately, other abuses shamefully endure," he said. Most notably, the companies "consistently fail" to send notices and communicate decisions to stakeholders in a timely manner.

Nearly 60,000 complaints about mortgage servicers were registered between October of last year and March 31, Smith's report found. The main complaint, with about 12,000 filers contacting the banks, was that a single point of contact was not provided.

When a servicer fails in the same area after an improvement plan is implemented under Smith's review, eventually they could be hit with civil penalties and fines.