A three-judge panel of the Court of Appeal in San Francisco said the measure, known as Proposition H, conflicted with state laws regulating handguns.
The court ruled in a lawsuit filed by the National Rifle Association, four firearms rights groups and seven individuals to challenge the ban.
It upheld a similar ruling in which San Francisco Superior Court Judge James Warren found in 2006 that state laws pre-empted the local ordinance.
The measure, which never went into effect, would have barred almost all city residents from possessing handguns and prohibited all residents from selling, manufacturing or distributing the firearms within the city.
Exceptions would have been allowed for possession by law enforcement officers and others such as security guards who need guns for professional purposes.
Alexis Thompson, a spokeswoman for City Attorney Dennis Herrera, said no decision has been made on whether to appeal further to the California Supreme Court.
Thompson said, "We are disappointed in the decision, particularly with the continuing plague of handgun violence in San Francisco. We are currently looking at the ruling and evaluating our options."
The city argued that it was entitled to pass the measure to address the local problem of gun violence and protect residents' welfare and safety.
But Justice Ignazio Ruvolo wrote for the appeals panel that a "comprehensive montage" of state laws on gun sales, licensing and registration was intended to balance competing interests.
Ruvolo wrote, "These laws of statewide application reflect the Legislature's balancing of interests - on the one side, the interest of the general public to be protected from the criminal misuse of firearms, on the other, the interests of law-abiding citizens to purchase and use firearms to deter crime, to help police fight crime, to defend themselves and for hunting and certain recreational purposes."
The panel noted that the California Supreme Court has upheld some limited local regulation of guns, such as county bans on gun shows or sales on county fair property.
But the appeals court cautioned that "when it comes to regulating firearms, local governments are well advised to tread lightly."
