Their local library has become a magnet for those living and using drugs on the streets.
Now some are asking the library to turn off the WiFi in an attempt to push them out.
ABC7 News reporter Lyanne Melendez visited 16th Street near Market across from the library to see what conditions were like.
RELATED: 50 Blocks: Stories from SF's Tenderloin neighborhood
She navigated her way through a homeless encampment where she saw families and many tents.
Thomas Ashwood told her why many gather there.
"It's because they're safe here and it's not violent here. When they go to sleep they don't have to worry about waking up when their tents are on fire or someone is cutting it open," explained Ashwood.
They are drawn to this spot because they're in front of a public building with access to the bathrooms. They've also learned how to generate power from the bicycle docks.
"When I come to the library, you almost always see someone smoking meth and people are doing fentanyl as well and it's comfortable, the weather is not too bad, no one bothers them," said long-time resident Richard Wigen.
VIDEO: 'Out of control': Amid SF homeless crisis, drug problem worsens in Castro District
'Out of control': Amid SF homeless crisis, drug problem worsens in Castro District
Another visible reason for congregating here is the access to free Wi-Fi.
It's no secret, that residents have asked the public library to shut down the Wi-Fi at night hoping that will drive them out.
"What are you going to do get all of them shut down,?" asked Ashwood.
The reality is that the free city Wi-Fi along the Market Street corridor is not reliable, but the Wi-Fi at the public library is much more robust and allows for streaming.
In fact, ABC7 News witnessed many of them doing just that, using the internet from outside, waiting for the library to open.
We reached out to the San Francisco Public Library. They said a 2017 study found that crime in the area fluctuated whether Wi-Fi was turned on or off. They also told us "they continue to seek to address the digital divide."
RELATED: 'I covered their eyes and we ran away': Family recounts what it's like living in SF's Tenderloin
Father shares struggles of raising children in SF's Tenderloin
"Obviously they aren't filing resumes," said Kathy Amendola, a resident of the Castro who offers historic tours of the district.
"Obviously you want to provide this service for your patrons and anyone is welcome inside but there is no reason to let it go on during the night," added Amendola.
The
re are signs of vandalism in the area. The library's parking lot surveillance camera was damaged and neighbors continue to report crimes like broken garage doors and Ring doorbells that have been tampered with.
"This is the public library, I live in this neighborhood. I won't let my daughter come here unaccompanied," said David Burke, the SFPD Public Safety Liaison for this district.
"We don't have a lot of sticks, we have even fewer carrots and if folks don't want help, they can refuse it which is very frustrating for those of us who live and work in the Castro," expressed Burke.
Burke says it's obvious they need help and treatment but they are one of the hardest groups to reach.