Demonstrators expressed anger and frustration over the death of 52-year-old Lorenzo Salgado Araujo, a husband and father from Guadalajara, Mexico.
"We're here because we're angry, we're sad, we're upset," one activist told the crowd during the rally.
According to the Department of Homeland Security, Salgado Araujo was shot on July 7 after failing to comply with verbal commands and allegedly attempting to use his van to run over ICE agents.
RELATED: Family says dad was scared and 'wanted to protect his tools, workers' before deadly ICE shooting
DHS said an officer fired a weapon in self-defense. Federal officials later acknowledged that Salgado Araujo was not the intended target of the enforcement operation, stating that he "resembled the target" agents were seeking.
The incident has drawn criticism from immigrant rights advocates, particularly because the ICE officers involved were not equipped with body-worn cameras.
Adding to the controversy, at least two passengers who were inside the van have disputed the federal government's account.
RELATED: Man fatally shot by federal officer in Houston was not ICE target, DHS says
They claim Salgado Araujo was shot through the passenger-side window and that no officer was in danger at the time of the shooting.
"An innocent man was killed, shot down," said Patricio Gonzalez, a longtime San Jose activist. Gonzalez and other members of the Community Service Organization are calling for accountability and a thorough investigation into the shooting.
"These ICE agents were looking for someone else," Gonzalez said. "They didn't take more care to find out if this was the actual person. We want ICE out of our communities."
San Jose resident Tara Lee, who attended the demonstration, said she supports immigration reform but opposes current immigration enforcement practices. "I'm all for humane and sensible immigration reform, but the current ICE program is horrible and horrific," Lee said.
The rally also highlighted local concerns about a proposed ICE detention facility near Gilroy. California Attorney General Rob Bonta has filed a lawsuit seeking to block construction of a potential detention center on a 24-acre site east of the Gilroy Premium Outlets.
RELATED: Mexican builder fatally shot by an ICE officer is mourned after making a life in the US
The lawsuit alleges the project violates state environmental and immigration laws.
"Do we need an ICE facility taking up a big swath of land as a detention camp to keep people from our communities in?" Gonzalez said. "No. No one was asked about this."
Meanwhile, Salgado Araujo's sons say their father had no criminal history. The family is calling for an independent investigation into the circumstances surrounding his death.
If you're on the ABC7 News app, click here to watch live