Black Lives Matter protesters showed up Thursday outside Harvey's house and there was tension as some Trump supporters who live in the neighborhood showed up.
RELATED: Santa Rosa man's mannequins send social distancing message; 'Even a dummy can do it'
"That is not okay, that is not okay for him to do that," said Veronica Perez who went on to say "Brentwood is not racist, it's beautiful and we don't support that!"
Harvey took to social media Thursday and said, "Last night I was so pissed off I had a dummy from Halloween hanging by a noose so I made a cardboard sign, put it around it's neck that said, 'Sleepy Joe (Cheater)' on my second story roof.
Police said the Contra Costa District Attorney said the display was not criminal on the basis of offensive expression being protected under the first amendment.
The mannequin was hanging from the second story of the East Bay home when police officers arrived.
VIDEO: Video shows 'Sleepy Joe' mannequin at Brentwood home
No charges filed after 'Sleep Joe' mannequin displayed outside Brentwood home, police say
The mannequin was clothed in a baseball cap, long-sleeved shirt and blue jeans. A cardboard sign with the phrase "Sleepy Joe (Cheater)" was displayed on the mannequin's chest.
The dummy's face was covered in what appears to be a black hat or black cloth.
"It's not like he's trying to pretend that it's just Joe Biden, why didn't he put a white doll with Joe Biden on it? Why did it have to be a black person?" said Corena Diaz who protested outside Harvey's house Thursday night.
Police say the homeowner agreed to take the mannequin down.
Though the incident is not considered criminal, the investigation has now been forwarded to the U.S. Secret Service, Brentwood police said.
ELECTORAL MAP: How Pres. Trump, Biden could win the 2020 presidential election
Path to victory: How Trump, Biden can win presidential election