"From Palestine to Mexico, these border walls have got to go," the group chanted.
TAKE ACTION: Get help with your rights to justice, equality and civil liberties
As reported by the New York Times, the round-ups would target about 2,000 undocumented immigrants on deportation lists.
The raids would begin this Sunday in at least 10 major cities-- about three weeks after President Donald Trump warned this could happen.
"We began to receive unusual reports of ICE activity here starting Sunday," said Hamid Yazden Panah, an immigration attorney with the California Collaborative for Immigrant Justice.
RELATED: 'Siente mucho miedo': Potential ICE raids spark fear in San Francisco's Mission District
Panah says at least three immigrants were arrested by ICE in Contra Costa County last Sunday and that his group has been tracking reports of ICE arrests the whole week.
"In total, I don't have the exact number, but it seems a lot higher than normal, in the last few days."
But Bay Area ICE Spokesman, Paul Prince, told ABC7 that their activity this week has been, "normal - busy as usual. The agency's Enforcement and Removal Operations (ERO) component continues to conduct targeted immigration enforcement in compliance with federal law and agency policy."
Blanca Marroquin, a mother from Guatemala, came to the rally with her five and seven-year-old young sons. Marroquin says she fled to the U.S. from Guatemala five months ago-- leaving behind a life filled with violence and domestic abuse.
In Spanish, she said about ICE, "My biggest fear is that they would take my boys away and deport me back to my country."
RELATED: How immigration authorities make arrests, conduct raids
"I think ICE should be disbanded. I think ICE should be abolished," said State Assemblyman, David Chiu, who was at the rally.
Chiu says the California legislature is doing everything they can to protect California's immigrant communities.
"As a son of immigrants, as someone who represents a city that is a sanctuary city, in a state that is a sanctuary state, we are always going to stand by our immigrant families."
Immigration groups have been handing out cards advising people that they are not legally bound to open doors to ICE.
Meanwhile, ICE officials are unwilling to confirming this Sunday's raids.