The blaze was reported at 7:52 a.m. at 545 N. Fourth St. It was controlled at about 8:45 a.m.
No injuries were reported.
Firefighters battled the flames from the outside because the structure was deemed unstable, San Jose fire Capt. Mary Gutierrez said.
The blaze destroyed the second floor of the two-story house, which was boarded up and contained little other than a sleeping bag, pillow, and an old TV set, Gutierrez said. The house is surrounded by a chain-link fence.
"A lot of those homes in Japantown have historical value," Gutierrez said. "We're still trying to locate the owner's information for the property."