Police arrested John Walker of Campbell on Thursday after investigating allegations of inappropriate conduct by the nanny.
"We are concerned that there may be other victims in this case," said police Chief Greg Finch.
The San Jose Police Department Child Exploits Team and the Campbell Police Department have identified multiple children victimized by Walker, police said.
Walker worked as a childcare provider for numerous families in Santa Clara County over the last 10 years. Police said they believe there is a high likelihood that he took advantage of other children.
"He has been entrusted by the parents for a number of years. He has taken advantage of that trust and the innocence of these children," police spokesman Sgt. Gary Berg said.
Victim resources have been set up and are available to victimized families, police said.
The Campbell Police Department is urging anybody with information regarding the case to contact the police at (408) 871-5190.