The league also said the banned owner has engaged in other conduct that has impaired its relationship with fans and merchandising partners.
Sterling has asked for a three-month delay in the hearing, sources told ESPN.com. That request is expected to be rejected, sources said. The owners' vote will take place after the hearing, which will be held in New York.
Sterling's comments during a CNN interview May 12 with Anderson Cooper are included as part of the basis for the charge to remove him and the rest of the Clippers' ownership group. The charge relates to a TMZ audio recording made public last month and acts related to that recording, including the CNN interview.
"All of these acts provide grounds for termination under several provisions of the NBA constitution and related agreements," the league said in a statement.
Minnesota Timberwolves owner Glen Taylor, the chairman of the league's board of governors, will preside over the hearing, which is planned for two days before the start of the NBA Finals. If three-fourths of the owners vote to sustain the charge, Sterling will be forced to sell the team he has owned since 1981.
Commissioner Adam Silver has said he is confident he has the votes.
Sterling and his lawyer, Maxwell Blecher, were hand-delivered the written charges on Monday. Blecher then sent an email back to the league's general counsel, Rick Buchanon, asking for the three-month delay. The NBA constitution provides that, after being formally charged, five business days are granted.
Blecher has declined comment through his law firm.
His client was banned for life and fined $2.5 million by Silver after the release of the TMZ recording. He has until May 27 to respond to the charge, and he has the right to appear at the hearing and make a presentation in front of the board of governors. If Sterling does not respond by May 27, that would be grounds for termination.
Sterling told a female friend, V. Stiviano, not to bring black people to Clippers games during their conversation that was recorded. He had specifically mentioned Magic Johnson on that recording, and during his CNN interview with Cooper, Sterling criticized the NBA Hall of Famer, calling him a poor role model.
"Among other things, Mr. Sterling disparaged African-Americans and 'minorities'; directed a female acquaintance not to associate publicly with African-Americans or to bring African-Americans to Clippers games; and criticized African-Americans for not supporting their communities," the NBA said.
The league also charged Sterling with issuing a false and misleading news statement about the matter.
Article 13 of the NBA's constitution states that one of the conditions that could lead to termination is if an ownership fails or refuses "to fulfill its contractual obligations to the Association, its members, players, or any other third party in such a way as to affect the Association or its members adversely."
A number of sponsors suspended deals with the Clippers in the wake of Sterling's remarks, potentially hurting league revenues, and some players have said they would consider a boycott next season if Sterling still owned the team.
"Mr. Sterling's actions and positions significantly undermine the NBA's efforts to promote diversity and inclusion; damage the NBA's relationship with its fans; harm NBA owners, players and Clippers team personnel; and impair the NBA's relationship with marketing and merchandising partners, as well as with government and community leaders," the league said.
If Sterling does not respond to the charge within five business days, or if he does not appear at the hearing, it would be deemed an admission of the "total validity of the charges as presented," according to the constitution.
But even the players who want him out believe Sterling will fight, and his attorney sent a letter to the league last week informing it that Sterling wouldn't be paying the fine. Blecher also said Sterling will sue if not afforded due process.
Sterling's estranged wife, Shelly, has said she will fight to keep her 50 percent share of the team even if Donald Sterling is forced to sell, but the league said in its statement that "all ownership interests in the Clippers will be terminated" if the charge is upheld.
Shelly Sterling's lawyer, Pierce O'Donnell, released a statement on Monday.
"We have just received the voluminous charges and are beginning the process of carefully reviewing them," O'Donnell said. "Based on our initial assessment, we continue to believe there is no lawful basis for stripping Shelly Sterling of her 50 percent ownership interest in the Clippers. She is the innocent estranged spouse. We also continue to hope that we can resolve this dispute with the NBA for the good of all constituencies."
Leon Jenkins, former president of the Los Angeles chapter of the NAACP, told The Associated Press Monday that he called Sterling after last week's interview aired and said Sterling's comments about Johnson took away from the apology he was making for earlier racist comments. Sterling was "really distraught" and hoped the public would forgive him, Jenkins said.
"I said, Once you got off focus and started talking about Magic Johnson, whatever reconsideration some people would have, you kind of lost it," Jenkins recalled.
Jenkins resigned as chapter president May 1 following outrage over a decision he later reversed to honor Sterling with a lifetime achievement award. It would have been Sterling's second award -- the chapter also honored him in 2009. The same year Sterling agreed to pay $2.7 million to settle a U.S. Justice Department suit alleging that he refused to rent apartments to Hispanics and blacks.
Jenkins said Sterling's comments were "very, very ugly statements" but also said "I'm not really sure we have all the facts."
"You can never under any circumstances defend what he said," Jenkins said. "But there are some issues there like his right to privacy, are these private thoughts in a private home."
He urged people to also not forget about issues like crime and unemployment that impact the African-American community.
"I think a true racist deserves everything he gets," Jenkins said, but he wouldn't say if he considered Sterling a "true racist." He said: "I don't know, I don't know. You have to look at the body of work."
And he added, "Where's the forgiveness here?" before pausing -- "but then again, you've got to show redemption first."
Information from ESPN.com's Ramona Shelburne and Darren Rovell and The Associated Press was used in this report.