Week 5 signals the (general) end of nonconference play, and as teams transition into their league schedule, it becomes harder and harder for fraudulent records to survive.
A bad or mediocre team can hide itself in the early going, and a handful of them do every year. By scheduling easy and catching a lucky break here or there, they can stay undefeated longer than their skill and coaching merit.
The cream always—or at least usually—rises, though, and as the season goes on, it tends to leave non-dairy teams like that in the dust.
One can only fake being good for so long.