A goal from Michy Batshuayi opened the scoring for the Blues after 55 minutes, but a dramatic 94th-minute equaliser from Jamal Lewis took the game into extra-time.
The Premier League champions prevailed 5-3 in a penalty shootout, edging out the Canaries in a game of small margins.
The victory sets up Chelsea for a home clash with Newcastle United in the next stage of the competition.
Blues manager Antonio Conte gave run-outs to many of his reserves, as goalkeeper Willy Caballero, Ethan Ampadu and Davide Zappacosta were selected.
Danny Drinkwater and Kenedy both featured in midfield for the Premier League champions, with Batshuayi leading the forward line.
Canaries star midfielder Alex Pritchard recently departed in a major transfer to Huddersfield Town, but his absence did not affect the visitors' confidence or performance.
The hosts should have taken the lead in the opening moments after Cesar Azpilicueta was just short of the ball as he slid in from close range.
Drinkwater then slammed his effort against the crossbar after 25 minutes, with replays showing Norwich stopper Angus Gunn getting a fingertip on the ball.
However, an error from Batshuayi almost gave the visitors the lead. Nelson Oliveira's dipping effort bounced off the top of the crossbar, as Chelsea fans held their breath after the half-hour mark.
Norwich were the happier side as Chelsea gave a lacklustre display, and Conte appeared glum, with his players lacking any sort of momentum.
The EFL Championship side remained disciplined until the interval, and there was silence from the home stands as the teams trudged off the pitch.
Fooball.London writer Oliver Harbord gave a straight verdict on the game:
Oliver Harbord @ojharbord
Half-time: Chelsea 0-0 Norwich. Announcement at the break that the District Line is now running from Fulham Broadway, just in case any fans want to leave now...2018-1-17 20:46:31
Chelsea finally broke the deadlock nine minutes into the second half as Batshuayi slotted home from close range.
The relief was palpable as the Belgium international scored after excellent work from Kenedy on the left wing.
Batshuayi had cut a frustrated figure throughout the contest and was clearly overjoyed to earn a rare moment of personal success.
The effort woke up the home fans who had been very quiet throughout the game, but Norwich responded by going on the attack.
A mistake from both Caballero and Azpilicueta almost cost the Blues a goal, with Josh Murphy striking the post as his shot went through a crowd of players in the box.
Drinkwater should have put the game beyond Norwich's reach after failing to hit the target after excellent work from Willian, and the match became stretched in the final stages.
Substitute Alvaro Morata watched his glancing header flash just wide in the last minutes of regulation, but Chelsea appeared content with a slender win against an inferior opponent who had battled hard.
The result was turned on its head in the fourth minute of injury time. Norwich's Lewis ghosted into the box and found a dramatic header to level the scores at the death.
Conte was distraught after the failure of his side, and the Blues were shellshocked as they were forced to play 30 minutes of extra time.
The hosts pushed forward in an attempt to steal a winner, but they had two men sent off as Pedro and Morata were given their marching orders for second yellows in a heated period.
Norwich earned their deserved chance to progress via a shootout, but the hosts showed their technical quality from the penalty spot.
Willian and David Luiz handed Chelsea control with good penalties, and a save by Caballero on Oliveira's attempt gave the Premier League giants the advantage.
Conte's men wrapped up the win as Azpilicueta, N'Golo Kante and Eden Hazard made easy work of their spot kicks, but plenty of credit goes to Norwich for their valiant display.