His ties to the place were too strong. It's where his father, Dave Wilcox, played before a Hall of Fame NFL career. It's where he and his brother, Josh, served as a ball boys in the late 1980s. After games, the brothers would get to play Nerf football on the field, and later they were teammates during standout careers for Oregon.
"I was really fortunate to have the opportunity to do that and it was a great experience in my life," Wilcox said.
Coaching at six other schools over the past 19 years, though, has diluted the significance that a return to Eugene, Oregon, might once have had. Since his first full-time coaching gig in 2003, Wilcox has coached against Oregon 11 times. Saturday's game between No. 13 Oregon and California will mark Wilcox's seventh return trip to Autzen and his second as Cal's head coach.
Been there, done that.
"I'm really appreciative, but I guess when you're in this profession, especially, you got to be so locked in," Wilcox said. "You're so invested in the group you're with."
As Wilcox's career has progressed, his family's rooting habits have adjusted accordingly. His parents and brother are still regulars at Autzen Stadium, where they usually sit in former Oregon athletic director Pat Kilkenny's box, but there is no question where their loyalty will be this week.
"It's family. Mom, Dad and myself we'll all sit where we do for Ducks games," Josh said. "And we'll be wearing Cal gear."
After a standout career as a tight end for Oregon, Josh spent a couple of seasons in the NFL and had stops in arena football, NFL Europe and the XFL. He also dabbled in professional wrestling.
"It's always kinda fun [when Justin coaches against Oregon]," Josh said. "It's like being the heel in wrestling. You can walk in there and everyone is looking you."
For the past decade, though, the results haven't been so much fun. Wilcox's teams have gone 0-6 against Oregon and every game was decided by at least three scores. He hasn't been on the winning side against his alma mater since 2009, when he was defensive coordinator at Boise State and the 14th-ranked Broncos beat No. 16 Oregon 19-6.
That Boise State win, infamous for Oregon running back LeGarrette Blount's postgame punch, was the type of defensive struggle Cal will likely need if it expects to beat Oregon (4-1, 1-0 Pac-12) on Saturday (8 p.m. ET, Fox). Even before starting quarterback Chase Garbers got hurt in last week's loss to Arizona State, Cal's offense wasn't exactly humming.
The Bears went into that game 4-0 and No. 15 in the AP poll having scored no more than 28 points in a game. The defense, led by standout linebacker Evan Weaver and one of the nation's best secondaries, did the heavy lifting.
"I think you got to credit the entire defense," Oregon coach Mario Cristobal said. "They're hard-nosed, they're tough, they're disciplined.
"They're also very talented. I think that gets lost a little bit. People just say, 'Well, they're tough.' They're really, really talented and they're coached really well. They've got a great team."
Oregon's success to this point has come in a similar fashion. The Ducks have been dominant on defense -- they haven't allowed a touchdown in three games -- but are seemingly just skimming the surface on what their potential might be in talented quarterback Justin Herbert's final season.
Cal announced earlier this week that Garbers will be out for a significant amount of time with the upper-body injury he suffered against ASU. It's possible he could return by the end of the season, but it's unclear how likely that is.
"At Ole Miss and then really the first half [against ASU], he was playing the best football since he's been here," Wilcox said. "Really in a rhythm. So you really feel for him there. ... It's hard in college football and he'll bounce back and we have other guys prepared to step up."
Devon Modster, who backed up Josh Rosen before transferring from UCLA, will make his first Pac-12 start. In his only start for the Bruins, a 35-17 loss to Kansas State in the 2017 Cactus Bowl, Modster showed promise, completing 21 of 34 passes for 295 yards with a pair of touchdown passes, but he struggled in his first meaningful action for Cal last week.
For the Bears, the hope is that with a week of practice and a game plan tailored to his strengths, he'll play much better than against ASU, when he completed just 5 of 14 passes for 23 yards with an interception.
In Wilcox's short tenure at Cal, he has picked up some significant wins -- including two against Washington -- but a win this week might be his biggest to date. That would certainly be the case for his family.
"From a Duck fan perspective, if I can put those blinders on, it's fun to watch one of your own come back," Josh Wilcox said. "Even one of my old teammates, taking the brother part out, just coming back as a head coach and playing against the team he played for is cool.
"And then for it to be my brother, I'm sure for my father it's especially pretty special."
Garbers goes down with shoulder injury in first half
Cal quarterback Chase Garbers scrambles and comes down hard on his right shoulder, forcing him out of the game against Arizona State.