Jill Ellis, U.S. Soccer Federation's director of development, will serve as interim coach and take over the post on Thursday, when the U.S. faces China in San Diego.
"We want to thank Tom for his service over the past year and half, but we felt that we needed to go in a different direction at this time," U.S. Soccer president Sunil Gulati said via news release. "We will begin looking for a new coach immediately to guide our women's national team toward qualifying for the 2015 FIFA Women's World Cup."
Sermanni's sacking was announced a few hours after Lauren Holiday and Megan Rapinoe scored in the Americans' 2-0 win over China in Commerce City, Colo. The U.S. held China to one shot on goal while running its home unbeaten streak to 81 games (71 wins, 10 draws).
"I'm disappointed that things didn't work out, but I'd like to thank U.S. Soccer for the opportunity to have coached this team and also the staff and players for all their hard work,'' Sermanni said in the national governing body's statement.
The U.S. was unbeaten last year under Sermanni, winning 13 matches and drawing three. But it had a disappointing Algarve Cup in March, finishing with two losses and a draw in its three matches. The loss to Sweden in that event ended a 43-game unbeaten streak for the U.S. team.
Sermanni took over the coaching reins in October 2012 after Pia Sundhage left the program to coach the national team in her native Sweden.
Sunday's move caught many in the U.S. women's soccer community by off guard, including Sermanni. ESPN television analyst Julie Foudy reported via her Twitter account that Sermanni "had no idea" that the USSF was contemplating a switch.
Portland Thorns coach Paul Riley, who was among the finalists when Sermanni was appointed following Sundhage's resignation after the 2012 Olympics, also expressed surprise at the announcement.
"[It was] a big shock to all," Riley said via text message about Sermanni's dismissal. "Tom was very laid-back, but that was always his way. I thought he was excellent."
Sermanni was attempting to transition the team to a new generation of players, especially on defense, and it's unclear whether such changes were met with resistance by the players.
A conference call with Gulati is set for Monday.
ESPN FC's Jeff Carlisle and The Associated Press contributed to this report.
U.S. Women's Soccer Coach Fired
Julie Foudy discusses the reaction of Tom Sermanni after learning that he had been fired as coach of the U.S. women's national soccer team.