Arsene Wenger's side scored twice in the last 10 minutes as Aaron Ramsey equalised before Olivier Giroud bagged the victor at the Emirates Stadium.
Arsenal looked a beaten team when Leicester led 3-2 early in the second half after Vardy grabbed his second goal before the substitutions gave them new vigour in attack.
But Giroud made his mark on Arsenal's new campaign on Friday as his late header secured a 4-3 win at home to Leicester City, who had led when the forward came off the bench.
They have won trophies in each of their last two competitive games, beating Chelsea in the FA Cup final last season then in the season opening Community Shield at Wembley. We see players like Hector Bellerin, Alex Iwobi coming through, Reiss Nelson and Joe Willock have had a fantastic pre-season.
North Korea Tensions Hit Global Stock Markets
The company posted better-than-expected quarterly earnings but its sales missed expectations. USA stocks closed lower on Tuesday after Trump's comments sparked a late afternoon selling.
"Previously in the Premier League we've not got off to the best of starts, so it meant a lot for us to get a good start and get three points today".
Once they are back and available, Welbeck is hopeful that future results can be earned in less trying circumstances. Shinji Okazaki scored another header to make it 1-1 three minutes later, as the Japanese forward latched on to a Harry Maguire flick-on from Marc Albrighton's cross.
Arsenal welcomed Leicester City into Emirates Stadium on Friday to get the festivities rolling this season, and it appeared as if everyone would get a chance to use their Arsenal jokes for at least the next week.
"It was a great game for the fans to watch but it is disappointing for ourselves", he told LCFC TV.
"We had some avoidable goals which we conceded and overall we have to work on that", he said referring to poor marking in the by his team's defence.