ABC7 News learned the owner was in legal trouble before it even started as he is being sued by the people who used to live here.
PHOTOS: Massive fire burns near Oakland's Lake Merritt
The owner of the building next door also says she had to hire an attorney at one point against him because of some demolition work he was doing.
RELATED: Attorney calls Oakland apartment fire 'suspicious'
Federal agents from the Department of Alcohol, Tobacco and Firearms are at the scene investigating what may have caused the fire.
Fire investigators have not said whether they think this was set on purpose, but former tenants have their suspicions as they are suing him for starting demolition work while they still lived here. They believe it was retaliation because they had made complaints about their living conditions.
The attorney representing the former tenants spoke to ABC7 News Tuesday morning. "In this particular instance where you've got a landlord that's willing to obtain demolition permits and start demolition on a large scale and bring in big machinery while people are still living in a unit," attorney Michael Bracamontes said.
The apartment building was under construction when the fire broke out and wasn't set to be completed until next year.
The fire damaged the home next door, displacing three people.
Piece of scaffolding that fell from a burning building pierced a car windshield in Oakland this morning. No one hurt. pic.twitter.com/aTXy9PqaDL— Amy Hollyfield (@amyhollyfield) October 31, 2016
Officials say no one was hurt.
ABC7 News reached out to the building owner Tuesday morning, but have not heard back. His son said his dad is a good guy and that prior business dealings had nothing to do with this fire.
In the meantime, residents on Lester Avenue are still inconvenienced and firefighters are still taking up space in the area, while nearby streets are closed while they look for cause and watch the building for hot spots and possible collapse.