Mayor Ed Lee's goal is to cut the timeline in half.
It's not the only planning department that deals with new construction. There are other city agencies that have to approve permits.
"You're talking about trees, sidewalks, water and sewer connections, gas and electric," said San Francisco Supervisor Ahsha Safai.
VIDEO: SF mayor pushing to speed up construction permit process
Quite often, those departments unnecessarily slow down the building process in San Francisco. Lee wants to see caps of 9, 12 and 18 months in the permit process to accelerate new construction. "I know that once I have given the proper directives and support to the departments to make the right decisions and to move the projects forward, we can cut it in half all the delays," he said.
The mayor's goal is to build 5,000 new units every year for the foreseeable future -- 30 to 35 percent of those would be permanently affordable.
Since 1986, San Francisco has built on average only 1,900 new units. Then in 2014, the mayor challenged the city to increase those numbers to get to 30,000 new and rehabilitated units by 2020.
"It's not enough and if we want to keep that pace going, this directive will excel and enable us to keep the pace -- 5,000 units a year," Lee said.
San Franciscans we talked to are in favor of building more affordable housing, but worry about the impact it will have on public transportation.
"I take the bus every day. I take the 9 San Bruno and there is more and more congestion with ride share and stuff that makes it really hard to even take the bus now, let alone drive in the city," San Francisco resident Emily Craig said.
The mayor admits a lot of the congestion is ironically caused by construction projects throughout the city.