Roman, 44, was the NFL's highest-paid offensive coordinator. The Bills appointed assistant head coach/running backs coach Anthony Lynn as their new offensive coordinator.
Roman's firing comes less than 24 hours after the Bills fell to 0-2 with Thursday night's 37-31 loss to the Jets. Buffalo's offense gained 393 total yards in the game, bouncing back from a 160-yard outing in a season-opening 13-7 loss to the Baltimore Ravens.
The Bills hired Roman in January 2015 after naming Rex Ryan as their coach. Roman was among about a dozen candidates to interview for the Bills' head-coaching vacancy following the sudden departure of Doug Marrone.
Roman served as the 49ers' offensive coordinator from 2011 to 2014 after following then-49ers coach Jim Harbaugh from Stanford.
Lynn, 47, served as Ryan's running backs coach with the Jets from 2009 to 2014 before joining Ryan in Buffalo last season. The former NFL running back for the Broncos and 49ers interviewed for the Dolphins' head-coaching vacancy and San Francisco's offensive coordinator job this past offseason.
Though Buffalo scored 31 points on Thursday night, seven came as a result of safety Nickell Robey-Coleman returning Jalin Marshall's fumble 36 yards for a touchdown.
The running attack has particularly struggled a year after leading the NFL with 2,432 yards. Buffalo has combined for just 151 yards rushing in two games.
That's not what was counted on from a unit that returned mostly intact and was expected to be more comfortable with Roman's philosophies and playbook.
Quarterback Tyrod Taylor went 8-6 as a first-time starter last year. The Bills believe Taylor has the opportunity to be their long-time starter after signing him to a six-year extension last month. The Bills, however, can opt out of the contract in each of the next two years.
Roman's dismissal is the latest in a series of shuffles Ryan has made to his staff.
In January, he hired his twin brother, Rob, to be an assistant head coach with a specific responsibility for defense. In March, defensive line coach Karl Dunbar was fired and replaced by John Blake.
Information from The Associated Press was used in this report.