The fast-moving fire has forced hundreds of residents out of their homes.
At least one home and six structures have been destroyed, but no injuries have been reported.
New mandatory evacuation orders have been ordered for Little Uvas and Croy roads. Animals are also being evacuated.
Video from Sky 7 showed firefighters putting water on the flames next to a building and home on Casa Loma Road while several horses nervously paced back and forth.
RELATED: Horses, alpacas evacuated from Santa Cruz Mountains
It has been tough for firefighters to battle the flames in part because it is dry and hot - although it's cooled off significantly overnight, the terrain is steep and unfamiliar to fire crews as well.
Community members are trying to lean on one another for support. "They lost their house and their animals in front of their eyes, it was heart wrenching," Gilroy resident Jennifer Swifteagle said.
Eric and Danielle Mays just moved into their house in the hills two months ago. They weren't home when Loma Fire erupted and they weren't allowed to go home, which led to a frantic call to a new neighbor. "And he just takes off down the road, up the hill toward the fire, oh I'm getting chills, to go get my dog and my cat," Danielle said.
RELATED: Disaster preparedness resources
Danielle is grateful to have her pets safe and sound. She has no idea about her house. "I don't know if it's still standing, if it's covered in retardant, if I'm ever going to get back up there, if it's going to be a black hole," she said.
The estimated containment for the fire is October 3.
"A weather pattern change, the days are going to be continuously cooler as we progress, so we're hoping that coupled with the amount of resources we have coming in across the state will help fight against this fire," Cal Fire Cpt. Nick Wallingford said.
Just below Loma Prieta on Casa Loma Road many gathered to wait as crews worked to contain the fire. "You've got seniors that live up there, that lost everything, some of them are on medication, they didn't even have time to grab their medication out of their house or anything," Swifteagle said.
The smoke from the Loma Fire is dominating much of the Santa Clara Valley skyline.
One woman's childhood home was spared, but her grandmother's house just a few doors down didn't make it. "My dad's like, 'oh I'm going to stay here, I don't think it's going to hit us, but then all of a sudden it's like no it's coming, I could see flames from mini Pacheco,"' Morgan Hill resident Katie McPartland said.
Cal Fire says more than 500 personnel are on the ground with more to arrive in the coming days.
PHOTOS: Crews battle brush fire in Santa Cruz Mountains
Temperatures on the fire lines were anywhere between 90 degrees and 100 degrees Tuesday afternoon. "We have very, very, very dry fuel up there, we have steep terrain and we have territory that hasn't burned in 50-plus years, kind of making for a perfect storm for this even right now," Cal Fire Battalion Chief Jonathan Cox said.
Residents nearby say they're trying to keep their heads up. "You know that the community is here to support in any way they can and it's a wonderful feeling," Swifteagle said.
Evacuation centers have been set up at Soquel High School, the Jewish Community Center in Los Gatos and the Presbyterian church on De Witt Avenue in Morgan Hill.
For evacuees who want to get their large animals out of harm's way, two centers have been set up to take them in. Horses can be taken to the Graham Hill Showgrounds in Santa Cruz or the Santa Cruz County Fairgrounds in Watsonville.
If you have other animals, please call ahead to see if they can take them.
Mandatory evacuation orders for the Loma Prieta ridgeline area including all tributary roads along Summit Road from Soquel San Jose Road to Ormsby Fire Station. This includes Uvas Canyon County Park, Loma Chiquita, Casa Loma, Loma Prieta Way, Highland Road and Mount Bache Road. Portions of Croy Road are under an evacuation warning.
Loma Chiquita Road
Summit Road from Soquel San Jose Road to Pole Line Road
Mount Bache Road at Highland Road
Casa Loma Road at McKean/Uvas
Mount Madonna at Ormsby Road
Mount Madonna Road at Pole Line Road
Croy Road at Uvas Road is open to residents only.
Click here for full coverage on the Loma Fire.
I was just told more evacuations taking place right now for #LomaFire at Croy Road and Little Uvas. Animals also being evacuated from area.— Matt Keller (@MattKellerABC7) September 27, 2016