But, according to new data released Tuesday, that could be starting to change ever so slightly.
MORE: SF metro area sees mass exodus during pandemic, Census Bureau says
In fact, according to one index, housing prices in the city by the Bay dropped faster in August than in any other city in America - a 4.3 percent decline since July.
"A couple of months ago, it was academic. We heard that there were changes coming, but I'm sensing on the street that folks are feeling, okay, it's here. It's changed," said realtor John Yen Wong.
There are numerous reasons behind the drop, according to economists.
"We've seen interest rates going up very quickly. Faster than they've gone up since the 1980s. The Federal Reserve has been pushing up short-term rates and mortgage rates have responded as well," said economist Daniel Altman.
VIDEO: Bay Area realtor, economist predict rents will rise 5% to 10% by end of the year
Experts predict rents will rise 5% to 10% by end of the year
On top of climbing interest rates and high inflation, the already astronomical prices of San Francisco real estate means costs have further to fall.
And, despite being off their peak, real estate prices are still hovering near all-time highs.
"Given the market that we had before where they were 10, 15 offers for a property, that was sort of an unreal market. Now it's moving into a more reasonable discussion between buyer and seller type of market," Wong said.
Experts say prices locally, like the rest of the country, will continue to fall in the immediate future.
MORE: Are Bay Area housing prices about to drop? Here's what experts say as interest rates increase
But most don't think the downturn will resemble anything like the crash we saw back in 2008.
"We don't have such a huge overhang of leverage this time, so I don't think that the crash, if there is one in housing, would be quite as bad as what we saw then," Altman said.
Because, despite its flaws, San Francisco still remains a highly desirable place to live for many.
"If you mention the name San Francisco anywhere in the world, and anywhere in this country, everybody knows of it," said Wong.