The Contra Costa Fire Department says the Lafayette Tennis Club was destroyed in the blaze.
An outbuilding was also damaged and a residential structure received minor damage to its roof.
Fire officials issued an evacuation notice on Sunday afternoon. People were told evacuate to Springhill Elementary School.
All evacuations were lifted around 7 p.m.
Stay with ABC7 News for the latest developments.