WASHINGTON – The political future of Congress's longest-serving member, Rep. John Conyers Jr., D-Mich., appeared to be precarious early Wednesday as Democratic leaders pressured him to resign over allegations that he had sexually harassed multiple female aides.

House Minority Leader Nancy Pelosi, D-Calif., and members of the Congressional Black Caucus encouraged the veteran lawmaker to step down as soon as this week after a fourth accuser came forward Tuesday morning, according to a senior Democratic aide who spoke on condition of anonymity to describe private conversations.

Although Conyers denies wrongdoing and has insisted he will not resign, he stepped down as ranking Democrat of the House Judiciary Committee on Sunday. The move was seen as a concession to critics who said he should no longer occupy such a powerful perch as allegations against him mount.

The 88-year-old lawmaker flew back to Detroit from Washington on Tuesday night without explanation.

House Speaker Paul Ryan, R-Wis., meanwhile, faced a battery of questions about how he will respond to the reports of members' secret settlements with aides who accused them of harassment.

He declined to say Wednesday morning whether Conyers should resign. "Look, I know what I would do if this happened to me," Ryan said at a news conference. "I will leave it up to him to decide what he wants to do. I think he made the right decision in stepping down from his leadership position."

Conyers was not spotted at votes Tuesday evening, where members of the CBC held a rare huddle on the House floor. Several members of the group declined to say publicly whether Conyers should step down.

"Resignation is a personal matter," said Rep. Cedric Richmond, D-La., the chairman of the CBC. "That's a personal decision for him and his family." Richmond said in a statement Tuesday that he had "a very candid conversation" about the seriousness of the allegations against Conyers, "which he vehemently denies." He also said he urged his colleague to fully cooperate with the ethics investigation.

"Any decision to resign from office before the ethics investigation is complete is John's decision to make," Richmond said.

"I don't think we should rush to judgment on decisions," said Rep. Bennie Thompson of Mississippi, ranking Democrat on the House Homeland Security Committee. "It's his decision to make, and I would look forward to him at some point making that decision."

Arnold Reed, Conyers's attorney, said CBC members were not trying to get him to resign.

"The tenor of the meeting was to discuss how he was doing, how he was handling things. They were trying to determine how his family and how he is handling the allegations. They are not trying to force him to resign," he said.

"We will be having a conversation about the allegations and how we move forward. It may be tomorrow, it may be the next day, but it will occur," Reed said.

Conyers, an icon of liberal policymaking, has become a focus in discussions about sexual harassment in Congress as decades of misconduct and a pattern of secret settlements between lawmakers and staff members come to light on Capitol Hill. Although a growing number of female lawmakers are urging congressional leaders to respond swiftly, Conyers' seniority and participation in the civil rights movement have given some colleagues pause about calling for his resignation. Pelosi called him "an icon" on Sunday talk shows, only to face immediate criticism from women's rights advocates and others.

On Wednesday morning, Rep. Kathleen Rice, D-N.Y., called on Ryan to lift a confidentiality requirement affecting a woman who settled with Conyers after accusing him of harassment. "The accuser who attempted to seek help through a deeply flawed system should not continue to be silenced by the institution that failed to protect her in the first place," Rice wrote in a letter to Ryan.

Rice has urged Conyers and Sen. Al Franken, D-Minn., who has apologized after facing allegations that he groped several women, to resign.

On Tuesday, Rep. Pramila Jayapal, D-Wash., also said Conyers must resign, given the pattern of misconduct alleged by former aides. "This is a watershed moment where, finally, the country seems to be waking up and realizing we need to have a zero-tolerance policy toward sexual harassment," Jayapal, a member of the Judiciary Committee, said in a statement."I believe these women, I see the pattern, and there is only one conclusion: Mr. Conyers must resign."

A report published Tuesday in the Detroit News could make it harder for Conyers to defy calls to resign. The paper reported that Deanna Maher, who worked for Conyers between 1997 and 2005, said he propositioned her once and inappropriately touched her twice.

Multiple allegations have surfaced since last week, when BuzzFeed reported that Conyers reached a financial settlement in 2015 with a former employee who said she was fired for refusing his sexual advances.

In court documents filed earlier this year, another woman, Maria Reddick, accused Conyers of harassing her while she worked as his scheduler.

And in an interview with The Washington Post, a well-known lawyer specializing in congressional ethics accused Conyers of harassing and verbally abusing her when she worked for him in the 1990s.

The House Ethics Committee has launched an investigation into Conyers's behavior, and on Tuesday, Pelosi urged the panel to review the sexual harassment allegations "expeditiously as well as fairly."

"Should you need any additional resources to fairly and swiftly pursue these investigations, please make that need known," Pelosi wrote in a letter to the committee's leaders Tuesday afternoon.