The group says the center should be part of the reparations process for slavery and discrimination against Black people in the city.
RELATED: Bay Area activist reflects on decades-long fight as CA, SF explore reparations for slavery
"I was born here. I would come here with my dad and I would see the magic that used to happen on the streets down here. Magic when I went up there. I saw people that looked like me. People who had businesses and were doing work," said Actor Danny Glover.
The Hollywood actor joined forces with activists and politicians demanding for the building to be given to the African American community as part of the reparations process.
"It's the location for the San Francisco Black Panther headquarters," said Executive Director of New Community Leadership Foundation Majeid Crawford.
In 2007, the Fillmore Heritage Center became a musical venue for Jazz and Blues, but decades before that activists say it was a key location for civil rights.
WATCH: The importance of building Black wealth for future generations
"Deliver justice, fairness, and inclusion for the culture. The history and the good sense of the African American community," said President of the San Francisco branch of the NAACP Rev. Amos Brown.
The center is located in the Fillmore District, once known as the "Harlem of the West" decades ago.
"This building is a right for us to return and have culture. There is nowhere in San Francisco we can enjoy ourselves. Not one place," said Gloria Barrett, The San Francisco Reparations Advisory Committee. "The reparations committee requests that this building is a priority because it's an urgent matter."
RELATED: Congress approves bill to make Juneteenth a federal holiday; Biden expected to sign
The San Francisco Reparations Committee is requesting for city officials to prioritize this building as rumors of its potential sale are circulating.
"There are some rumors that this building is going to be sold off to the highest bidder," said San Francisco Supervisor Matt Haney.
ABC7 News met Mayor London Breed at a different press conference where we questioned her about the site. She said it's complicated and added, "There are things we are obligated to do as a city under re-development disposition laws that will require money. Until we have a clear understanding of what that would entail and what it would mean we don't know if that is going to be possible."
When we asked Mayor Breed what she would like to see she responded, "I would like to see the venue become a huge success. I don't want to continue to see the venue be a financial drain to the city."