Police responded to the supermarket at 3002 Story Road at 8:37 p.m. on reports of a medical emergency.
The victim, a man in his 20s, had been stabbed at least once and was pronounced dead at the scene.
The suspect or suspects had fled the scene before police arrived at remain at large tonight, police said.
Police remain at the Safeway store tonight investigating. The killing is San Jose's 31st homicide of the year.
Anyone with information about this case is urged to contact the San Jose Police Department, Homicide Unit, at (408) 277-5283. Persons wishing to remain anonymous can call the Silicon Valley Crime Stoppers at (408) 947-STOP (7867), or may visit: svcrimestoppers.org.
KGO contributed to this report.