Two years ago, Ayres went on trial, charged with molesting young boys he was treating. The trial ended in a hung jury. Plans for a new trial have been derailed because the prosecutor and defense attorney have decided he is not mentally competent.
It is a painful development for victims, who do not want to be identified by name or face.
One such victim, identified only as "Clark" says he was molested by Ayers as a teenage patient. At each therapy session, the doctor would use a phrase.
"You know the drill, and the drill was to remove my pants and underwear and sit in the chair in front of him naked from the waist down," "Clark" said.
Victims say they do not believe Ayres has dementia. Video shot by private investigators hired by the victims shows Ayres having lunch last Thursday in San Francisco.
"Dr. Ayres discussed everything from the Iowa straw polls to the future of the Republican Party," victims' advocate Victoria Balfour said. "He talked about a business venture he had in the 1980s. He talked about how in the future he's going to have to go to court."
The district attorney thinks the video is inconclusive.
"My view is that it shows somebody who was still engaged in obviously some aspects of life, but it doesn't change the fact that he is suffering from dementia," San Mateo District Attoryney Steve Wagstaffe said.
There will be no closure for the victims.
It is already too late for 48-year-old J.D., who committed suicide two weeks ago. His friends say he was never the same after being molested when he was 11.
"He went from a happy-go-lucky kid, many friends, to being withdrawn and very few friends," friend Jonathan Huddleston said.
The court will decide in two weeks about committing Ayres to a mental care facility.