It’s been interesting to watch my white media colleagues, who have been known to use indecisive words like racially-charged or bigoted, define President Trump’s recent tirades against four Congresswomen of color as racist. In truth, Trump’s push for these American citizens to “go back” to where they came from is about much more than race. It is about the American psyche.

Ours is a country of immigrants where many people pretend it is a country of natives. That’s why, when Donald Trump doubled down on his comments that four Congresswomen of color should “go back” to their “crime-infested" countries where “corrupt” and “broken” governments held sway, I was not surprised. Nor was I shocked when he doubled down on his original comments in tweets, in statements and at rallies.

But this is a little more complicated than it seems. In fact, it’s so complicated that I’ll paraphrase a line from my conservative adversaries: This is not (just) about race. It is about America’s historical penchant for defining people as outsiders and colluding to hold those people down.

The white folks who joined Trump at a North Carolina campaign rally to chant “send her back” when Trump mentioned Congresswoman Ilhan Omar — a former Somali refugee — should recognize this truth. That’s because many of them are likely descended from people who were victimized in the same way.

It’s easy now to look at these four Congresswomen and think that race is the only thing that defines such difference. After all, Alexandria Ocasio-Cortez is a Bronx native of Puerto Rican descent. Rashida Tlaib of Michigan is a Palestinian American. Ayanna Pressley of Massachusetts is African American. Ilhan Omar of Minnesota was born in Somalia and became an American citizen in 2000.

But this kind of victimization began long before Trump began his presidential campaign based on the bigoted claim that Mexico was sending criminals and rapists to America. In fact, that message goes back centuries, to a time when the people who were being targeted this way weren’t black and brown. They were white.

In the 1850s there was a group of immigrants that was often described as poor and diseased. They were criticized because their religion was a little different. They were frequently called criminals and rapists.

That group was the Irish. They were a people who were escaping a famine in their country. they came to America in a desperate quest for survival.

During the 1880s, the number of Eastern European Jews fleeing to America increased. They came to escape economic hardship and religious persecution. Still, they continued to face anti-Semitism here. By 1920, there was a political response to the millions of Jews who had come here as immigrants, and in 1921 and 1924, restrictive immigration laws were put in place.

In the 1890s a wave of Italian immigrants began arriving in America. They were met with fake scientific theories claiming that “Mediterranean” types were inferior to people from northern Europe. Images in the media helped to drive those theories home, often portraying Italians as childlike, subhuman criminals.

That kind of language should sound familiar to black and brown people. It’s the language that has continuously been used to define us. And the phrase “go back” to where you came from is as familiar to us as it would have been to the Irish, Italians and Eastern Europeans who came to America in the late 19th and early 20th centuries.

That's what makes this all so ironic.

Some of those at Trump’s rally yelling “send her back” about a black woman are very likely the descendants of the Irish, Italian and Eastern European immigrants who faced that same kind of hate when they arrived in America. Or perhaps they’re the progeny of impoverished Britons who arrived as indentured servants and were treated as slaves.

But maybe that's the greatest aspect of identifying oneself as white. You don't have to remember that your ancestors were once the victims of ethnic hatred. You don't have to look back at America's formative years and see that your people were once the underclass you now despise.

Whiteness allows too many to pretend that ethnic and racial hate is un-American. In truth, even a cursory look at history confirms that hate is in our country's DNA.

There is no simple solution to America's state of denial. However, there is a way to begin our journey toward truth.

The descendants of Irish immigrants have to remember where they came from. Eastern Europeans have to remember the hate they faced. Italians have to recall what it was like to be outsiders.

America has to know we can't go back.

Solomon Jones is the author of 10 books. Listen to him weekdays 7 a.m. to 10 on 900 AM/ 96.1 FM WURD.