The leak was reported shortly before 1 p.m. in the 1900 block of Alum Rock Avenue.
PG&E spokeswoman Monica Tell said the leak occurred when a third-party crew doing digging struck and ruptured a half-inch gas line.
San Jose fire Capt. Cleo Doss said the crew was using a backhoe, and that the leak occurred outside the apartment building's parking garage.
"The way the prevailing winds were blowing, it was blowing the gas right into the underground parking area," he said.
The building was evacuated, but residents were let back in shortly after 2 p.m. when the leak was capped, Doss said. Hazardous materials crews also did a sweep to make sure there were no hidden pockets of gas, he said.
Tell said today's incident is a good reminder that anyone doing digging should call 811 to learn the location of underground gas lines before starting the work.