The government cannot do anything to make Rafael Aponte's home affordable.

Aponte has been unemployed for three years since being laid off after 17 years at National Display Co. That is how long he has owned his Northern Liberties rowhouse, now in foreclosure.

Protesting near the National Constitution Center, where a congressional panel yesterday tried to gauge the progress of homeowner-rescue programs, Aponte said his lender had agreed to modify his mortgage, then had withdrawn the offer.

"They said that without a job, how could I pay even that?" Aponte said. "They're right, but it isn't fair."

That opinion was shared by many speakers inside the center. While applauding the Obama administration for getting the Home Affordable Modification Program up and running, some questioned whether the $75 billion being paid to mortgage servicers would do more good if instead it was lent to homeowners, especially the growing ranks of the jobless.

"Getting a homeowner's DTI [debt-to-income] ratio to 31 percent won't help the unemployed, since 31 percent of their income is zero," said Paul Willen, a senior economist and policy adviser at the Federal Reserve Bank of Boston.

Irwin Trauss, supervising attorney at Philadelphia Legal Assistance, told the oversight panel that money repaid by lenders who took Troubled Asset Relief Program money might be used to structure a national program along the lines of the Pennsylvania Homeowners' Emergency Mortgage Assistance Program. That program provides loans to unemployed Pennsylvanians facing foreclosure to help them make mortgage payments until they return to work.

Trauss and others testifying on behalf of borrowers complained about the volume of paperwork and length of time the modification process takes - laying the blame on the loan servicers.

Eileen Fitzgerald, chief operating officer of the nonprofit NeighborWorks America, told the panel that working with the servicers "continues to be a challenge."

"Our counselors have difficulty communicating with servicers," she said. "Many of them are not following the guidelines, and some will not even work with us, and we often spend two or more hours on the phone being switched from one extension to the next."

The servicers defended their efforts.

Allen Jones, a Bank of America Corp. senior vice president, said the lender had increased trial modifications substantially and was "trying to make the process more consumer-friendly."

"We understand the urgency, but there are limits to what we can do," he said, citing the rising jobless rate and the "lack of interest in keeping up property" by those in foreclosure.

Outside, Jeff Eison of University City stood with protesters, holding another set of documents he had to send by FedEx to his lender in his six-month search for a modification.

"It has been real stressful," he said. "I have to get up at 2 a.m. to fax things to these people because the phone lines are jammed during the day."

Contact real estate writer Alan J. Heavens at 215-854-2472 or aheavens@phillynews.com.