Former New Jersey Gov. Chris Christie said Friday that he had told President Trump he does not want to be his next chief of staff.

It’s the latest sign that Trump, known as mercurial and difficult to manage, is struggling to fill a once-coveted job after announcing last week that John Kelly would leave the White House by the end of the year.

After Christie met with Trump at the White House on Thursday to discuss the position, speculation mounted that Christie was a leading contender.

But in a statement Friday afternoon, Christie said he had informed the president that “now is not the right time for me or my family to undertake this serious assignment.”

“It’s an honor to have the president consider me as he looks to choose a new White House chief of staff,” Christie said in the statement. The former governor added that he had asked Trump “to no longer keep me in any of his considerations for this post.”

Christie was an early supporter of Trump’s during the 2016 campaign, but was fired as head of the transition team after Trump was elected. He has said he has turned down several offers to join the administration.

One problem Christie may have faced had he taken the job: working with Jared Kushner, Trump’s son-in-law and senior adviser. During his tenure as U.S. attorney in New Jersey, Christie prosecuted Kushner’s father, Charles, for crimes including tax evasion and campaign finance violations.

(Christie’s forthcoming book is titled Let Me Finish: Trump, the Kushners, Bannon, New Jersey, and the Power of In-Your-Face Politics.)

The president’s reported top choice to replace Kelly, Nick Ayers, chief of staff to Vice President Pence, is also leaving the White House. Others reportedly under consideration for the job include Jared Kushner and Robert Lighthizer, the U.S. trade representative.