The fortress has historically kept people out, while Kink.com made adult movies beyond the brick walls.
But now, the owners want to invite as many as 4,000 people in for concerts until as late as 2 a.m. "I thought it was cool because they already had a couple headliners which I was interested in," Daniel Contreas said.
The venue has hosted parties and concerts in the past, which its allowed to do with one night event permits.
But some neighbors didn't like what they heard and saw. "I think the neighbors are having a really hard time trusting the venue operators and owners," Deputy Director of San Francisco Entertainment Commission Maggie Weiland said.
The San Francisco Entertainment Commission has received dozens of written complaints.
One woman who lives nearby wrote that she witnessed urination and sex in building entryways. Others said things got too loud. "We have a whole bunch of drunkies acting crazy and my little sister's trying to go to sleep," Ana Barahona said.
"I don't want to deal with people throwing up down the street, it's just a little bit much especially when you have a child and you're already trying to have a homey feel in this type of neighborhood, there's no way I'd be happy with that," Jessica Contreras said.
Heavy traffic and a lack of parking also have people concerned.
But, the planning commission and the police department have already signed off on the permit. "It's very unlikely that the commission would outright deny a permit application because they have the ability to condition the permit in many ways," Weiland said.
Conditions that could limit the days and hours of concerts.