The director was spotted on location at 14th Street and South Van Ness Avenue Tuesday morning.
A woman answering the phone at the family-owned Mexican restaurant Los Yaquis on South Van Ness Avenue between 14th and 15th streets said she had seen all the activity on the street corner.
She said the film crew had seemed nice and considerate of local businesses.
The San Francisco Film Commission said in the beginning of August that Allen would be filming throughout the Bay Area through the end of the month for the yet-to-be-named film, scheduled to be released next year.
Actors Cate Blanchett and Peter Sarsgaard, who are in the film, were spotted Friday filming in front of jewelry shop Shreve & Co. near Union Square. Other actors rumored to be in the film are Alec Baldwin and comedian Louis C.K.
Some filming has been done in Pacific Heights, and other locales for the movie include the Sunset and Marina districts and parts of Marin County.
The romantic comedy, produced by New York-based company Perdido Productions, is about a wealthy New York City woman who finds herself broke and in San Francisco, living with her sister and downsizing her life, according to the film commission.