Warren Taylor of Sullivan County, Tenn., is being questioned and authorities do not yet have a motive, state police Sgt. Michael Conroy said. The hostages and suspect left the building in Wytheville after authorities ordered the man to surrender.
The standoff began at about 2:30 p.m. when shots were fired at the one-story brick post office in the mountain town in western Virginia. No one was injured, and relatives of two of the hostages said they were able to talk to their loved ones by phone.
It ended about 8 1/2 hours later without the dozens of SWAT members armed with automatic weapons having to fire a shot.
"We're just grateful it ended peacefully," Conroy said. "This is just the best outcome we could hope for."
Police in the town in the heart of the Blue Ridge Mountains told the Wytheville Enterprise the suspect had what appeared to be a common plastic explosive strapped to his chest. Conroy said police had found weapons and that shots were fired, but no explosives had been uncovered.
He said they were still searching the building late Wednesday, as well as Taylor's truck.
The suspect made no demands other than to ask for a pizza, said Pete Rendina, spokesman for the U.S. Postal Inspection Service.
An FBI negotiator had asked the several dozen SWAT members, police, fire and others surrounding the building to be quiet because authorities were talking with the suspect.
An officer in the early evening delivered food and drink to the front door at the request of the suspect, state police said. Earlier reports said the man holding the hostages was in a wheelchair, but state police said he entered the building pushing one.
Carlton Austin said his daughter, postal worker Margie Austin, was among the hostages. She managed to call a family friend around 4:30 p.m. and said she was fine. Later, her father said, family members were waiting to hear more.
"That's all we can do," he said.
Niki Oliver told the Enterprise that her brother, Jimmy Oliver, was one of the hostages and had been able to phone family members.
"We love you," she yelled to him as his mother was speaking to him on the phone.
Niki Oliver said her brother went to the post office to mail a Christmas gift to his son.
Postal worker Walt Korndoerfer said he was in the post office when he heard shots and a co-worker ran past. He called police and then ran himself.
His wife, Christine Korndoerfer, said he called around 3:30 p.m. to tell her he had gotten out safely.
"My husband is not one to get upset," she said. "When he called, I don't think I've ever heard him so upset."
Sutherland said the streets were filled with holiday shoppers at the time in the traditional-looking American town of 8,500 decked out for Christmas.
"All the stores are busy," he said.