The whale didn't seem to be in any obvious distress, though it was getting pretty close to shore. It swam directly under several paddleboarders and kayakers.
Some onlookers said they saw the whale as early as 7 a.m. Tuesday. By mid-afternoon, the young marine mammal was putting on quite a show.
Experts at the Ocean Institute said the juvenile whale was likely chasing food when it got lost once it made its way into the harbor.
Orange County Sheriff's Department officials along with civilian paddleboarders worked together to usher the whale out of the harbor.
The unusual experience was surreal for many beachgoers. "Unreal. Unreal. It was really cool," said Richard Cespedes.
RELATED: Whale 'dances' beautifully in Great Barrier Reef
"I've had turtles and dolphins come around me, but never a whale, ever. So, it was pretty crazy. Absolute close encounter," Paddleboarder Christy Ruiz said,
It appears the same whale may have also been spotted Monday as it surfaced in a Carlsbad lagoon.
Residents on the beach and paths spotted the 15-foot juvenile gray whale swimming in the afternoon.
They said the baby seemed to be playing in the shallow waters. It eventually drew a crowd of nearly 300 spectators.
SeaWorld experts went to the area and said the whale appeared to be fine.
After hanging out in the lagoon for a while, the whale swam back out to sea.
Click here to read more stories about whales.