The body was spotted by people near the marina around 9:05 a.m., who then helped pull the body out of the water, police said.
Police have only identified the body as that of a white man in his 60s.
Detectives are working with the Alameda County coroner's office to determine if the man's death was a homicide.
Anyone with information is asked to call Oakland police at (510) 238-3821.
