Four row houses got the brunt of the damage. Neighbors say the house where the fire began has been abandoned for years and has been a trouble spot.
Sunday morning services were called off at the Calvary Apostolic Church. The church was not damaged by flames, but the smoke was unbearable.
"So we called all of the members, let them know service was cancelled for today, let them know to go to another church across town," Pastor Jervis Williams said.
The fire was fierce when crews arrived and one neighbor says she was surprised at how quick it moved.
"It scared me because this whole block was lit up like it was daylight and the fire was intense where you could just feel it on your skin," Jaqueline Hunter said.
Outside the church, 71-year-old Maria Garcia waited to get into her house. She is one of the 30 people who have been displaced. She says she has her plastic bags ready to collect some clothes because all she has is what she's wearing. She lived right next door to the home that first caught fire.
Williams says the house where the fire started has been abandoned for some time and has become a big concern.
"When a building is empty, it attracts other things that shouldn't be there," he said. "So we have been concerned about it for a long time and the fire department and the police department knows about it too."
As the fire was blazing, one neighbor said he was not surprised because he's called city authorities over the years to complain about squatters at the property who could cause trouble.
"It's just something that the city is failing to act on and that is blight in the Bayview, it's a real shame, there are hundreds of blighted buildings in this neighborhood," Robert Davis said.
Arson investigators spent much of the day collecting clues from the back yard. It was crammed with stuff, including a trailer RV that looks like it was recently used. But fire crews are not yet saying where exactly the fire started or how it was sparked.
Things do take time; arson investigators will be out here all day working on it," San Francisco Fire Battalion Chief Mark Johnson said.
Neighbors say they may have seen someone running from that abandoned home as the fire started. San Francisco police were out at the scene asking those neighbors for details. But specifics in the ongoing investigation have not yet been released.