"We know we're doing the right thing, and I know I have the owners behind me," Silver said, one day after the NBA charged Sterling with damaging the league with his racist comments and set a hearing for June 3.
Sterling has until May 27 to respond to the charge, and he has the right to appear at the hearing and make a presentation in front of the board of governors. If Sterling does not respond by May 27, that will be grounds for termination.
Sterling has asked for a three-month delay in the hearing, sources told ESPN.com.
Silver said the timeline will not be extended.
"In terms of additional time, the answer has been no," Silver said. "The proceedings and the process is set out in our constitution, something they signed on for when they became owners in the league."
The owners' vote on whether to oust Sterling will take place after the hearing, which will be held in New York.
"I'll say that we've had strong leadership from Glen Taylor, who's the chairman of our board, and Peter Holt, who just recently stepped down as the former chairman or who had been the former chairman of the board," Silver said. "We have a very active advisory finance committee, which is 10 NBA owners who have been meeting on a regular basis to discuss these proceedings, and the timing is laid out in the NBA constitution. We're following it to the letter in terms of numbers of days that Mr. Sterling has to respond and then when the hearing will be held, and as I said, I know we're doing the right thing here. This is an unprecedented proceeding.
"Will there be bumps in the road? Presumably yes. Mr. Sterling on one hand at least in his CNN interview indicated a willingness to accept the judgment of his owner partners. His lawyers are saying otherwise, so we'll see.
"But this will all get worked out. I know we're pursuing the right course here and doing the right thing."
Silver, who previously released a statement castigating Sterling for further comments he made criticizing Magic Johnson during a CNN interview May 12 with Anderson Cooper, elaborated on his disappointment in having to deal with Sterling while the public's attention could be focused on positive events such as the playoffs, which have reached the conference finals, or the upcoming draft.
"Your question makes me think of Kevin Durant's MVP speech," Silver said. "I remember at one point Kevin Durant says, really in addressing his mother who was sitting in the audience at the end of the speech, I'm paraphrasing, I think, but he said something like, 'Mom, we weren't supposed to be here. The deck was stacked against us.' I get choked up a little bit just remembering watching him give that speech, and I think Kevin Durant as our most valuable player embodies what this league is all about, and frankly Mr. Sterling doesn't."
The Sterling saga reached day No. 26 on Tuesday since the audio tape of his racist remarks was released by TMZ. On a day when the league was hosting the draft lottery to determine the positioning for what's considered to be one of the deepest prospect pools in years, Sterling and the Clippers again dominated the conversation.
"So it's not just the performances on the court that it's a distraction from," Silver said. "And I think what made this moment bigger than basketball, certainly for everybody involved in the league, and that moment being that recording, was that it did come from within, that under David Stern and commissioners that came before him, barriers were broken with this league, and I think for those who say it's a slippery slope, and my God, what happens to the next player or the next owner who does something wrong, I'd only say there's something particular about race issues when it comes to sports, and maybe the NBA in particular.
"I mean, it's no secret we have a league that the majority of the players are African-American; the vast majority of the owners are not. But it's as egalitarian an institution as there is anywhere, at least that I know of. And I look at the track record in terms of hiring of coaches, general managers, front-office personnel, even increasingly in the ownership ranks, I think it's you know, it's beyond anger. It's sort of what I said earlier. There's a certain sadness, and you feel it, it's almost a malaise around the league.
"That's what I sensed when I first met with the Clippers. It was something deeper than anger.
"And again, it's that so many of our players in listening to Kevin Durant who had experienced discrimination in their lives, we're not a post-racial society, but at least within the boundaries of my authority, I feel an obligation to protect the people who are within this league, and so that's my reaction."
Silver dismissed the claims by Sterling's estranged wife, Shelly, who has said she will fight to keep her 50 percent share of the team even if Donald Sterling is forced to sell.
"As I understand the position of Mrs. Sterling's lawyers, in essence they would say we accept you can terminate Mr. Sterling, but somehow Mrs. Sterling comes with the team," Silver said. "I think even if that's not what it said in our constitution, it just doesn't make sense. The same way even if you had unrelated partners, if you terminated the franchise of the primary owner and that owner had several colleagues, cronies, who were also owners with him, it wouldn't make sense that under our constitution we could then go about selling the team, but those other partners would have to come along.
"So our position is once under the constitution, based on Mr. Sterling's conduct, if the owners ultimately decide that it's appropriate to terminate his franchise, the interest of all owners is terminated."
Silver encouraged Sterling to initiate a sale of the Clippers' franchise on his own accord before the league has to intervene.
"Mr. Sterling still owns the Los Angeles Clippers," Silver said. "Mrs. Sterling, as I understand it, through a trust owns 50 percent of the team as well. It is their team to sell, and so he knows what the league's point of view is, and so I'm sure if he wanted to sell the team on some reasonable timetable, I'd prefer he sell it than we go through this process."
Of course, if Sterling refuses and three-fourths of the league's owners vote to sustain the charge levied against him, Sterling will be forced to sell the team he has owned since 1981.
"I envision once we move through that process that we will then put the team in order, and presumably we will hire an investment banker and we will conduct an orderly process," Silver said. "We also have a fiduciary obligation to the Sterlings to ensure that we sell it for the highest possible price, and there's no doubt it's an incredibly valuable asset."
Silver made no mention of Sterling during his three-minute opening remarks to the media, instead focusing on his desire to raise the league's minimum age requirement from 19 to 20, his intention to improve the league's implementation of instant replay and his excitement for the upcoming draft, his first since taking over as commissioner in February.
"I'll just conclude by saying the state of the game is fantastic, and I think as I said, it's a renaissance for basketball in this country and increasingly globally, and I think our best years are ahead of us," Silver said.
Information from ESPN.com's Ramona Shelburne and Darren Rovell and The Associated Press was used in this report.