It happened around 3 pm on southbound Highway 85 north of Winchester Blvd.
Police say they were conducting a traffic stop here when a man in the passenger seat of the car, who was wanted on an outstanding warrant, pulled out a handgun.
That's when an officer fired, hitting the suspect at least once.
"The adult male suspect was pronounced deceased at the scene. No officers were injured," police said.
The driver of the car was not injured.
"The officer will be placed on routine paid administrative leave," police said.
The freeway was temporarily shut down because of the ongoing investigation.
Southbound Highway 85 will be closed until further notice from Saratoga Avenue to Winchester Blvd.
Anyone with information is asked to contact Detective Sergeant TJ Lewis or Detective Brian Meeker of the San Jose Police Department's Homicide Unit at 408-277-5283.
If you want to report information anonymously, you can call the Crime Stoppers Tip Line, 408-947-7867.