Rupf retired in January 2011 after 45 years of service at the sheriff's office, 18 of those years as sheriff, sheriff's office spokesman Jimmy Lee.
He assumed a range of titles at the sheriff's office and was elected to his first four-year term as sheriff in 1992.
He began his tenure as sheriff in the early 90s with just 523 employees and left a department of 1,000 employees in 2011, Lee said.
Rupf was known for being active in the community, with a focus on youth and service organizations. In 1998, he established Martinez-based community service organization Sheriff's Charities Inc.
In an email Rupf sent last month to friends and former colleagues after receiving his diagnosis, the sheriff wrote, "Some will say that I should have retired earlier and enjoyed the good life. I say: Poppycock, my life could not have been any better."
Sheriff David Livingston expressed his condolences in a prepared statement.
"Today we lost a great man, a great leader and a true statesman," Livingston said. "His first priority was always the public safety of the Contra Costa County community and supporting the people he worked with. My heart goes out to his wife Carole and all of his children and grandchildren during this difficult time."
Plans for Rupf's memorial service are underway.