Rapoport added that Don Martindale, Baltimore's linebackers coach, will get a "strong look" to take over.
Pees, 68, has been with the Ravens since 2010 when he was hired as the linebackers coach. He was promoted to defensive coordinator two years later, winning the Super Bowl in his first season leading that group.
The Ravens only finished outside the top 12 in scoring defense once in the past six seasons. Their 18.9 points allowed per game in 2017 ranked sixth in the NFL.
Baltimore finished this season 9-7, missing the playoffs after Tyler Boyd's 49-yard touchdown reception with 44 seconds remaining.
Pees won two Super Bowl titles as an assistant coach with the Ravens and New England Patriots in 2004.