Harbaugh will not return for the final season of his $25 million, five-year contract he signed in January 2011.
VIDEO: 49ers fans address rumors about Jim Harbaugh
"For the last four seasons I have had the great privilege to coach one of the storied franchises in the history of football," said Harbaugh. "We accomplished many great things together as a team during this period, which is a tribute to the incredible efforts of some of the most dedicated players and coaches in the NFL. I will miss competing alongside this group of players and coaches, I have the utmost respect and admiration for their hard work and support. It has been my honor to share the sideline with these mighty men. I will always appreciate and remember fondly, the passion and support of our Faithful fans, and want to express my particular thanks to them."
The announcement came Sunday after a 20-17 win over the Arizona Cardinals and following a 8-8 season in which San Francisco had hoped to establish some momentum in the new $1.3 billion Levi's Stadium.
Instead, the 49ers saw thousands of empty seats at most games. San Francisco was eliminated from playoff contention with a 17-7 loss at Seattle on Dec. 14, its second defeat to the rival Seahawks in an 18-day stretch that helped seal Harbaugh's fate.
Harbaugh guided the 49ers to three straight NFC championship games and had a Super Bowl-or-bust mentality for this season that quickly turned sour.
"Jim and I have come to the conclusion that it is in our mutual best interest to move in different directions," said CEO Jed York. "We thank Jim for bringing a tremendous competitive nature and a great passion for the game to the 49ers. He and his staff restored a winning culture that has been the standard for our franchise throughout its history."
York added, "We are now squarely focused on finding the next head coach of the 49ers and I am very confident in Trent's ability to lead that process."
The 49ers came oh so close to their sixth Super Bowl championship after the 2012 season, losing 34-31 to Harbaugh's big brother, John, and the Baltimore Ravens.
Harbaugh had a 49-22-1 overall record in four years with San Francisco, which might look to promote from within to replace him. Defensive line coach Jim Tomsula's name has been mentioned.
From his signature outfit of khaki pants and a black 49ers pullover to his signature phrases such as "Who's got it better than us? No-body!" Harbaugh did things his way. And, largely, his way worked.
But not this season, when star linebacker Patrick Willis was one of several key players to go down with injuries on what had been one of the NFL's stingiest defenses.
Harbaugh's future was in doubt all season after he didn't receive the contract extension York publicly said he hoped to work out. Reports surfaced last January, when San Francisco lost by six points at Seattle for the NFC championship, that Cleveland inquired about trading for Harbaugh. York later confirmed the Browns did contact him.
Then, when the 49ers lost at home 19-3 on Thanksgiving night to Seattle, York turned to Twitter to express his discontent with the team - perhaps only a preview of a relationship that couldn't be repaired.
He wrote: "Thank you (hashtag)49ersfaithful for coming out strong tonight. This performance wasn't acceptable. I apologize for that."
There was some thought York believed he could trade Harbaugh this time and receive a first-round draft pick. Nobody expected Harbaugh to agree to such a swap that would hamper a potential new team.
Harbaugh said all season he would not be fazed by the constant chatter about his fate, or by those outside the organization questioning whether he had lost the locker room.
It's a breakup that seemed unthinkable when the 49ers nearly won it all less than two years ago in New Orleans.
49ers CEO Jed York and General Manager Trent Baalke will hold a press conference in the Levi's Stadium auditorium Monday at 1:00 p.m. to address the announcement.
ABC7 News contributed to this story.