Steve Lochte told The Associated Press by phone that his gold-medal-winning son called him Tuesday after arriving back in the United States and told him he was going to pick up his car and buy a new wallet to replace the one that he said was stolen during the robbery.
"I'm just happy he's safe," the elder Lochte said. "It was an unfortunate experience for him and the other three. I don't know what all the controversy is. They were basically taken out of the taxi and robbed. The main thing is he's very lucky that he's safe and that all they got was his cash and wallet."
The elder Lochte said his son's Olympic credential and cell phone weren't taken during the incident early Sunday morning. He said he was sure Ryan had his passport or he would not have been allowed to board a plane.
Feigen's whereabouts, however, could not immediately be confirmed.
The office of Judge Keyla Blank confirmed the order to seize their passports in a statement Wednesday.
The U.S. Olympic Committee said police went to the athletes village Wednesday morning to try to collect the passports, but the swim team had already moved out. Spokesman Patrick Sandusky declined to say whether Lochte and Feigen were still in the country.
"As part of our standard security protocol, we do not make athlete travel plans public and therefore cannot confirm the athletes' current location," Sandusky said. "We will continue to cooperate with Brazilian authorities."
The U.S. State Department issued a statement Wednesday encouraging those involved to cooperate with Brazilian law enforcement.
Lochte and three of his teammates say they were robbed at gunpoint in a taxi Sunday morning as they returned to the athletes village from a party, several hours after the last Olympic swimming events were held.
Police have found little evidence so far to support their accounts, and say the swimmers were unable to provide key details in police interviews.
"Why would anybody fabricate anything?" Steve Lochte said. "It's just ridiculous."
A police official with knowledge of the investigation told The Associated Press that police cannot find their taxi driver or witnesses. The official spoke on condition of anonymity because the investigation is ongoing.
Lochte's attorney, Jeff Ostrow, has said there is no question the robbery happened.
According to statements from Lochte and the USOC, the swimmers were returning to the athletes village by taxi after a night out at the French Olympic team's hospitality house in the Rodrigo de Freitas area in the upscale south zone of the city.
Traveling with Lochte and 26-year-old Feigen were 21-year-old Gunnar Bentz and 20-year-old Jack Conger. Lochte swam in two events at the Rio Games, winning gold in the 4x200-meter freestyle relay. He is a 12-time Olympic medalist.
Bentz' father, Steve Bentz, declined comment when reached by the AP.
The group did not call police, authorities said, and officers began investigating once they saw media reports in which Lochte's mother spoke about the robbery. Police interviewed Lochte and one other swimmer, who said they had been intoxicated and could not remember what type and color of taxi they rode in or where the robbery happened, the police official said. The swimmers also could not say what time the events occurred.
Later in the day, Lochte described the incident to NBC's "Today" show .
"We got pulled over, in the taxi, and these guys came out with a badge, a police badge, no lights, no nothing just a police badge and they pulled us over," Lochte said. "They pulled out their guns, they told the other swimmers to get down on the ground -- they got down on the ground. I refused, I was like we didn't do anything wrong, so -- I'm not getting down on the ground.
"And then the guy pulled out his gun, he cocked it, put it to my forehead and he said, 'Get down,' and I put my hands up, I was like 'whatever.' He took our money, he took my wallet -- he left my cellphone, he left my credentials."
Word of the robbery initially created confusion between Olympic and U.S. officials. An International Olympic Committee spokesman at first said reports of the robbery were "absolutely not true," then reversed himself, apologized and said he was relying on initial information from the USOC that was wrong.
Lochte told USA Today Sports that he and his teammates didn't initially tell the U.S. Olympic officials about the robbery "because we were afraid we'd get in trouble."
The Associated Press contributed to this report.
Fair to have doubts about Lochte's story?
Michael Smith and Jemele Hill discuss the Brazilian courts finding contradictions in Ryan Lochte's story about being robbed in Rio.