The 4:30 p.m. vigil was held on the steps of City Hall, the building where Moscone, 49, in his third year as mayor, and Milk, 48, the city's first openly gay supervisor in his first year in office, were fatally shot by former Supervisor Dan White on the morning on Nov. 27, 1978.
The memories of the slain city leaders were honored by members of their family speaking to an audience of about 150.
Jonathan Moscone, the former mayor's youngest son who was a teenager when his father was killed, asked for people to remember Moscone and Milk's lives more vividly than their deaths.
"I'm tired of remembering them on the worst days of their lives," he said. He suggested the city memorialize Moscone on his birthday on Nov. 24 rather than the day he died. "Let's get this straight: George and Harvey did not die heroically. It was a senseless act," he said.
The day of the shooting, White, angry that Moscone had turned down his request to reappoint him after he resigned as supervisor 17 days earlier, entered City Hall through a window in a side entrance and shot Moscone four times in his second floor office, reloaded his revolver, walked down the hall and fired five bullets into Milk.
White was tried on murder charges, but after his defense argued he suffered from diminished mental capacity, the trial jury chose a verdict of voluntary manslaughter in May 1979. The judgment sparked riots in the Castro District and at City Hall by protesters who thought the conviction was too light.
But rather than dwell on the details of their deaths, Moscone's son said that people should live their lives like the two fallen leaders, who were crusaders for equal rights. "We're all agents of change like George and Harvey were. All of us have a voice," Moscone said.
Mayor Ed Lee and former Mayor Willie Brown also spoke about the dreams of equality the two shared.
Lee said if Milk and Moscone were alive today, "They would smile. They would see that their efforts to make this city more equitable have already been accomplished."
Milk and Moscone "had been an incredible team," Brown said. "When I walk around the city... I see what George Moscone and Harvey Milk and what their existence inspired in all of us."
State Assemblyman Tom Ammiano also spoke, telling the audience, "You might take away the messengers, but you're not going to take away the message."
Supervisors Scott Wiener and Christina Olague, City Attorney Dennis Herrera, police Chief Greg Suhr and former Supervisor Bevan Dufty also attended the event.
The memorial continued with a performance by the San Francisco Gay Men's Chorus of the two songs, "Love Can Build a Bridge" and "Singing for our Lives."
The crowd started walking with candles toward the Castro District shortly before 6 p.m., the opposite direction a spontaneous march took the night of the shootings in 1978, accompanied by a small police escort.
The destination of the walk was Milk's old camera store in the Castro District at 575 Castro St., which is now a Human Rights Campaign Action Center and store.
The annual commemoration was sponsored by the Harvey Milk LGBT Democratic Club, the Harvey Milk Civil Rights Academy and the Harvey Milk Foundation. Events vary each year from small gatherings to bigger anniversary memorials.