The dog, named Rambo, was taken from a home on the 300 block of Escuela Avenue.
A spokesperson for the police department told ABC7 the dog was running outside the home around 8:15 p.m. when a suspect allegedly took the dog.
RELATED: South Bay police on the lookout for dog stolen outside Los Gatos Safeway
Police say the suspect may have been driving a blue and gray Nissan sedan, according to witnesses.
"So we're just asking for the community's help to try and find this puppy and bring it home to their owners, one of whom is a little boy and so we're certainly concerned he's missing his companion right now," said Katie Nelson, a police spokesperson.
Those with information on the animal's whereabouts are asked to call the non-emergency police line at (650) 903-6344 and use the reference number 21-04926.
This is the first case in a "while" where a dog has been stolen in Mountain View, Nelson said.
"I can't imagine what that little boy is feeling like right now," she said. "We just want to get this puppy home to his family."