Two-time Tour champion Vingegaard crashed during the fifth stage on Thursday.
"Our medical staff decided it would be best for him to recover at home and concentrate on the rest of his season," his Visma-Lease a Bike team said in a statement.
Vingegaard finished second in last year's Tour, when his preparations for the sport's biggest race were thrown into disarray after a horrific crash in the Tour of the Basque Country.
