Since the start of the year, a billionaire venture capitalist has bought several of the buildings in this area.
And many of these business owners say they're being forced to leave.
"We're scared. We don't want to lose everything," said Steve Amano.
Amano is the co-owner of Ten-Ichi, a sushi bar that was opened by his Japanese immigrant parents back in 1978.
Amano feels they're being strong-armed into closing next month.
MORE: Are free concerts helping San Francisco's comeback?
"I signed that termination agreement because they were saying things like we're going to charge you this. All these numbers they were firing at us that we'd have to pay. That was freaking us out," said Amano.
Across the street at La Mediterranee, we were told a similar story by Vanick Der Bedrossian.
His restaurant was opened by his father, a Lebanese immigrant, around the same time as Ten-Ichi.
Their lease isn't up until May, Der Bedrossian says the new landlord has shown no interest in negotiating a new lease.
"We have 20 plus employees who work here. We have a very strong following here in the community. And we want to have clarity about our future," said Der Bedrossian.
The buildings that both Ten-Ichi and La Mediterranee operate out of have been bought by entities connected to venture capitalist Neil Mehta.
MORE: CA's new retail theft laws come at a time when SF small business owners say they need more support
Mehta could not be reached for comment but nightlife entrepreneur Cody Allen who has worked with the venture capitalist's nonprofit SF Reserve Foundation sent us this statement:
"Our investment in Upper Fillmore has been made on behalf of our nonprofit, SF Reserve Foundation, with the objective of bringing in more small businesses, rather than introducing chain or formula retail tenants. Our foundation's aim is to bring in an even wider variety of food and beverage operators; to revitalize the iconic Clay Theater; and to restore Fillmore storefronts."
On Monday, Supervisor and mayoral candidate Aaron Peskin held a news conference on Fillmore Street to announce a plan to introduce legislation making it harder to kick out legacy businesses.
"It's important for the family, in this case a multi-generational family that's been here for literally almost half a century. It's important for the community that sees this as a gathering space. It's important for the fabric of a neighborhood," Peskin said.
Welcome news for people like Amano, who says while he's trying to find a new location for Ten-Ichi, he doesn't know what will come next.
"I mean we have no option. There's no other option. But we're not going to lay down. I can't. We have to keep working," he said.