CLEVELAND (AP) — Two brothers of the Cleveland kidnapping suspect say they fear people still believe they had something to do with the three missing women found in his home.

Onil and Pedro Castro tell CNN that they've been getting death threats even after police decided not to charge them. The men say they're hiding in an undisclosed location, unable to go home.

Pedro Castro says he would have turned in his brother Ariel if he had known he was involved in the women's disappearance.

"But yeah, it's going to haunt me down because people going to think, yeah, Pedro got something to do with this and Pedro don't have nothing to do with this," Pedro Castro told CNN. "If I knew, I would have reported it, brother or no brother."

Ariel Castro is suspected of holding the women captive in his home for a decade. Authorities say he kidnapped all three, raped them and fathered a child with one.

The women were found May 6 after one escaped and called 911.

The brothers were initially taken into custody but released Thursday after investigators said there was no evidence against them.