The hosts went in front from the penalty spot just before half-time after Shkodran Mustafi brought down Cheikhou Kouyate. Captain Milivojevic made no mistake, as he sent goalkeeper Bernd Leno the wrong way.
Arsenal equalised after the break with a terrific free-kick from Granit Xhaka. The Switzerland international powered a fierce shot past Wayne Hennessey and into the far corner.
Pierre-Emerick Aubameyang put Arsenal ahead five minutes later. The striker poked home at the far post after Alexandre Lacazette appeared to have handled Xhaka's free-kick into the box.
Arsenal appeared to be headed for a 12th consecutive win in all competitions. However, Xhaka conceded a late penalty by bringing down Wilfried Zaha, allowing Milivojevic to snatch a point.
Mustafi Must Be Dropped for Sokratis
Unai Emery opted to partner Mustafi with Rob Holding in central defence, and the German was culpable for Crystal Palace's opening goal.
The defender, with his back to goal, lunged in on Kouyate to concede the penalty from which Milivojevic fired Palace into the lead.
It was a clumsy error from Mustafi, who had no need to go to ground in such reckless fashion.
James Benge at the Evening Standard said he must take the blame:
Mustafi is known for making costly errors, and his latest mistake shows it's time for Unai Emery to bring Sokratis Papastathopoulos back into the defence alongside Holding.
The Greek international has made nine appearances this season and looks to be an astute acquisition.
He put in a dominant performance against Sporting in Arsenal's 1-0 UEFA Europa League win Thursday:
The 30-year-old has looked good alongside Holding when the duo have been paired together, and they seem to be Arsenal's strongest centre-back pairing.
Arsenal's next Premier League outing is a visit from Liverpool. The Reds have scored 24 goals already in the top flight this season, and the Gunners will need to get it right at the back if they are to take all three points.