Vicente Benavides Figueroa, 68, was in prison even though the state Supreme Court last month overturned his 1993 conviction on grounds that medical testimony at his trial was false.
Many doctors who testified to the cause of the girl's injuries recanted.
VIDEO: Rare look inside San Quentin Prison, home of death row
Kern County District Attorney Lisa Green announced Tuesday that prosecutors won't retry Benavides for first-degree murder and that without the medical testimony, it would be "difficult, if not impossible" to win a conviction for second-degree murder.
Even if Benavides was convicted on the lesser charge, he would be immediately eligible for parole, Green said.
ABC7 News cameras caught up with a Spanish-language reporter asking Benavides, "Are you feeling happy?"
He replied, "Si, muy feliz" which translates to, "yes, very happy."
VIDEO: Vicente Benavides walks free after 25 years on death row
Benavides was babysitting his girlfriend's nearly 2-year-old daughter, Consuelo Verdugo, in Delano in 1991 while the mother was at work.
The couple later brought the injured child to an emergency room. She died about a week later.
The couple told doctors that the girl had hit her head on a door but at trial the defense said the girl may have been struck by a car when she got out of the house and Benavides briefly lost sight of her.
A forensic pathologist concluded that the girl died from anal injuries from being sodomized, and several doctors testified that the girl's injuries were caused by sexual assault.
But nearly all later recanted, saying they hadn't seen her full medical records that indicated there was no evidence of sexual assault when the girl was first hospitalized.
They also said her genital and other injuries may have been caused by her medical treatment and some said the purported cause of death 'anatomically impossible,'" according to the state Supreme Court's ruling.
VIDEO: Golden State Warriors coaches, staff visit San Quentin State Prison
"Human beings make mistakes," says San Francisco based human rights attorney Robert Bryan, who has represented more than a hundred people who have faced the death penalty.
He says the Kern Couty DA's mistakes, past and present, are why Benavides should be free. "They don't have evidence beyond a reasonable doubt that he's guilty, so therefore he is innocent."
According to the Calfornia Department of Corrections and Rehabilitation, Benavides will be receiving reentry services.
According to the CDCR, under state law, people whose convictions were reversed based on insufficient evidence or who were exonerated are entitled to receive services like housing assistance, job training, help in obtaining valid identification card and other reentry services.
ABC7 News spoke with the victim's older sister on the phone Thursday night. She told us she is "shocked" and that she believes Benavides is guilty.
Benavides' representatives released a statement Thursday: "Mr Benavides will not be speaking today. Of course we are elated by the fact that he is released today. He was wrongfully convicted over 25 years ago...he will not be making any statements to the press."
The Associated Press contributed to this report.