A frantic opening five minutes set the tone for a sensational match, as Alexandre Lacazette's debut goal for the Gunners was cancelled out by Shinji Okazaki. Jamie Vardy's 29th-minute strike put the Foxes ahead, before Danny Welbeck levelled again in first-half stoppage time.
Leicester came again after the break, with Vardy heading home a Riyad Mahrez corner to put the visitors ahead for a second time after 56 minutes.
It looked to be enough for the Foxes for long spells, but two substitutes hauled the Gunners back from the brink. First Aaron Ramsey smashed home an equaliser in the 83rd minute, before Giroud powered a header into the top corner a couple of minutes later to complete an incredible fightback.
Arsenal fans heading to the Emirates would have been desperate to see Lacazette get off to a fast start at his new club. They didn't have to wait long.
With less than two minutes on the clock, Mohamed Elneny swung a cross into the penalty area and the new Arsenal No. 9 steered a pinpoint header beyond Kasper Schmeichel. Lacazette stood and soaked up the adulation of his new supporters.
Per the Match of the Day Twitter account, there's never been a goal scored earlier in a Premier League season:
Just when those in red and white may have thought things were set to change, some familiar pangs would have rumbled in their gut.
Less than three minutes after going ahead, their good work was undone, as the Gunners were caught off guard at a set piece. Harry Maguire stole in at the back post, nodded the ball back across goal to Okazaki and he applied the finish from close range.
There was a collective sigh in the stadium, and while Arsenal did respond well to that equaliser in an attacking sense, their defensive play was ropey. Just before the half-hour mark, Leicester and Vardy capitalised on those hesitancies.
A loose ball from Granit Xhaka found its way to Marc Albrighton, who guided a magnificent cross into the penalty area from the left flank. The England striker had timed his run brilliantly and thudded home an emphatic finish.
Per Squawka Football, Vardy, once linked with Arsenal, loves scoring against the Gunners:
The Emirates seemed to be readying itself for a collective booing at the break, as their team continued to toil. But they were given a major boost, as Welbeck was able to level things up.
That was after Lacazette and fellow summer signing Sead Kolasinac linked up in the area, with the latter squaring to Welbeck to fire into the unguarded goal. It was the last action of a pulsating first period.
The second started at breakneck pace too, and it was Leicester, unmoved by Welbeck's equaliser, who edged back in front again. And once again, Arsenal were statuesque at a set piece, allowing Vardy to bustle to the front post and head past Petr Cech.
Journalist Miguel Delaney wasn't surprised to see the Gunners defence struggling in those situations:
The home fans stirred with angst once again, with another opening day defeat seemingly on the cards. Manager Arsene Wenger looked to the bench in an attempt to liven his side up, with Giroud and Ramsey introduced.
James McNicholas commented on what was a peculiar new setup for the Gunners after those changes:
But the alterations eventually went from inexplicable to inspired in a chaotic finish to the game, as Arsenal roared back into the contest.
First, Ramsey put the Gunners back on level terms in the 83rd minute, as the Welshman was picked out brilliantly by Xhaka at the far post; he expertly controlled and finished from the corner of the six-yard box. Suddenly the momentum was with the hosts and the home supporters were believing.
With Leicester retreating into a deeper defensive shape, it was left to Giroud, as he so frequently does, to make a late impact. The forward bullied Maguire in a standing battle from a corner, got his head to the ball first and arrowed it off the crossbar, beyond Schmeichel. The roof lifted off the Emirates.
The manner of the victory makes it huge for Arsenal. And it's a win that'll leave those Arsenal supporters hoping, once again, that things will be different this year after all.