Police identified the suspect as a San Jose resident named Thomas Chalmes.
Officials said the girl was released earlier and is now safe.
The standoff prompted police to put nearby Hammer and Galarza elementary schools in a shelter-in-place as a precaution.
Police responded to a 911 call from the girl's mother at about 10 a.m., saying her daughter's father had come into her home in the area of Pine and Jonathan avenues with a gun and taken the girl hostage.
Police responded to the domestic dispute and then it turned into a SWAT stand-off that lasted for nearly six hours.
Police took the man into custody after he surrendered at 4 p.m.
ABC7 News met another man who didn't hesitate to put his life on the line for his neighbors. "You hear about people getting shot and he was like five feet away from me, all he had to do was lift up the gun and shoot me," Willow Glen resident Ron Vierra said.
Vierra said his neighbor across the street came rushing into his home, screaming that her soon-to-be ex-husband was trying to take off with their young daughter. Vierra rushed over to confront the man. "I looked at his right hand and he had a black pistol in it," he said.
Vierra shouted for help and told nearby construction crews to call 911 and then took it one step further. "I ran over, jumped in my truck and pulled my truck behind his truck, so that he wouldn't leave with the baby," Vierra said.
Ridgeway Consruction spokesperson Erica Ridgeway heard the commotion while working just a few houses down. "You're just standing there and realizing, well this isn't a good situation, I need to get down the street and empty out my job site and make sure everybody's safe," she said.
Sky7 HD was over the scene when law enforcement responded. Police and SWAT arrived within minutes of the call and were eventually able to negotiate the little girl's release. "Our technical negotiation team was able to establish a rapport with him and they were able to get him to release the child, so that was a great outcome," San Jose Police Department Lt. Mike King said.
Those nearby couldn't believe what was happening. "It's unfortunate to see something like this happen and I just hope that everything's OK. It's a very close-knit community, so I just hope that everybody's going to be alright," Willow Glenn resident Bronwyn Olmos said.
One neighbor's fatherly instinct that may have saved a family from tragedy. "All I knew was that I didn't want the baby to leave and I didn't want the baby to get hurt, and I did what I had to do," Vierra said.
The man is in custody and will be booked into jail on domestic violence charges.
#UPDATE: As we reported at 4/5pm, 34yo Thomas Chalmes of San Jose has surrendered. Police took him into custody. He'll be booked into jail.— Chris Nguyen ABC7 (@ChrisNguyenTV) March 11, 2016