Despite the tragedy, there is optimism for future of The Chabad House.
Rabbi Mendel Weinfeld says he got a call from someone he knows early Wednesday telling him the synagogue was on fire.
RELATED: Suspect in custody after 2nd fire erupts at Concord church in less than a week
"I came running to the scene, the one thing in my mind was, 'How do we save the Torah scrolls?'" Weinfeld said, "Torah scrolls are the most important, holiest part of our synagogue."
San Jose Fire Department crews were able to save those Torah scrolls and other important books.
The department says that they first got to the fire at 6:44 a.m.
They say the fire appears to have started outside and spread to a carport, from there it went through the attic and to the rest of the building. They're still investigating the cause of the fire.
VIDEO: Fire burns Saint James Cathedral of Faith in Pittsburg
The Rabbi shared surveillance images with ABC7 that they have of someone they believed caused the fire.
San Jose Police did an investigation into what happened and say that it determined that it was not a hate incident or racially motivated, something Rabbi Weinfeld says is a relief.
"People are relieved as well," he said, "I think people are very happy that this wasn't, God forbid, someone trying to hurt us."
RELATED: Police investigating San Francisco Armenian church fire as hate crime
Rabbi Weinfeld and his congregation had only been in this building for three months.
They'd done extensive renovations on it and had just celebrated Hanukkah there.
Members of his congregation have set up a GoFundMe account which has already raised thousands of dollars.
Rabbi Weinfeld says that for now, they'll be looking for a place to meet temporarily before rebuilding the synagogue.
"This fire not only won't stop us," he said, "It will make us bigger and stronger."