The White House confirmed the fatality. One person was seriously injured, the White House said.
"The president has been fully briefed on the situation in Virginia and is receiving regular updates," White House press secretary Sarah Huckabee Sanders said. "There are no serious injuries among members of Congress or their staff."
Amtrak said there were no serious injuries to those aboard the train, which originated in Washington.
An official with Albemarle County Fire and Rescue declined to comment immediately on the number of injuries or fatalities but said the agency would provide more information once they know the extent of the casualties.
The train was carrying members of the House and Senate, some of their spouses and children and aides.
Rep. Tom MacArthur of New Jersey was on the train but unhurt.
"On the way to the annual GOP retreat, the train that my colleagues and I were traveling on was involved in an accident. I am ok and am thankful for the first responders for their quick action. My prayers are with those who were injured in this accident," he tweeted.
Rep. Leonard Lance of New Jersey also reported that he was okay.
"The train carrying lawmakers to our retreat had a collision west of Charlottesville, VA. I was not hurt. Capitol Police and first responders are helping to secure the scene," he tweeted.
"We appear to have hit a garbage truck," said Sen. Mike Lee (R., Utah), who was reached by telephone on board the train.
Rep. Brian Fitzpatrick of Pennsylvania was onboard but unijured, an aide said.
"It looks like it's been maybe cut in half. We can see garbage strewn around, and we can see the back half of the truck. There was a feeling of an impact and you could feel we had hit something. It took us maybe a quarter mile to stop."
Lee said there are a few people "roughed up. Most of us hit a knee or a head on the seat in front of us but nothing too serious on board. I'm not sure about the driver of the truck yet."
Rep. Thomas Massie (R., Ky.) called the crash "loud and surprising. Some minor bumps and bruises in this car. We saw debris go by the left side of the train. The part of the truck we can see was decimated. Very relieved when the train came to a stop and still on the tracks."
"It was a hard impact. It threw everybody up in the air a little. I don't think anybody on the train is seriously injured, but I may be wrong," said Sen. Jeff Flake (R., Ariz.).
Flake, Sen. Bill Cassidy (R., La.), and Rep. Brad Wenstrup (R., Ohio) joined medical personnel in carrying one of the injured passengers to an ambulance. Flake and Wenstrup did almost the exact same thing last summer in the shooting at a baseball field in Alexandria, Virginia.
Flake said his wife and 18-year old son are with him, but they are OK, he said.
Republican lawmakers were scheduled to arrive at the Greenbrier Resort in White Sulphur Springs, W.Va., later Wednesday and hear from Vice President Pence. President Trump was scheduled to address the group Thursday.
Crozet is about 110 miles southwest of Washington, and 90 miles east of White Sulphur Springs.
According to a statement from Amtrak, the train struck a vehicle on the track about 11:20 a.m.
The rail agency said local law enforcement officials were investigating "and inspecting the equipment for damage."
House Speaker Paul Ryan (R., Wis.) called the incident a "terrible tragedy." He wrote on Twitter: "We are grateful for the first responders who rushed to the scene and we pray for the victims and their families. May they all be in our thoughts right now."
Senate Majority Leader Mitch McConnell (R., Ky.) said in a statement that he was "deeply saddened" by the crash.
"My condolences, and those of the entire Senate family, go out to the victim's family, friends, and coworkers," McConnell said. "Our prayers are with the other accident victims who are fighting to recover from their injuries."