The two-alarm fire was reported at 1300 Senter Road at about 8:45 a.m. by an employee at the Happy Hollow Park and Zoo, located within the park.
Firefighters fought the fire from the outside, and the front part of the building collapsed at about 9:05 a.m., fire Capt. Mary Gutierrez said.
As of 10:30 a.m., the flames were out but smoke could still be seen coming from the attic, and the fire was not yet considered under control.
Firefighters planned to remain at the scene through the afternoon extinguishing hot spots.
No injuries were reported.
The 5,600-square-foot, two-story Craftsman-style house was built in 1912 and is the last remaining historical structure associated with the Kelley family.
The land was once owned by Louise Kelley, who inherited it from her father, former San Jose mayor Lawrence Archer, according to the park's website.
Happy Hollow Zoo spokeswoman Vanessa Rogier said the house was heavily damaged in the Loma Prieta earthquake of 1989, which shook it off of its foundation.
It would have cost $8 million to renovate it, so the renovations weren't made, she said.
These days, the building was mostly empty and was boarded-up. It was being used for storage, and contained tables, chairs and historical artifacts.
"It was a real beauty in its time," Rogier said of the house.
Gutierrez said firefighters were able to recover some photos and an envelope containing negatives.
She said it appears the fire started in the front of the house, but the cause remains under investigation.
Fire Capt. Rob Brown said 50 to 60 percent of the structure was damaged.
Kelley Park is open to the public today, and visitors are not being charged admission, which is normally $12.