The French Professional League (LFP), the sports' domestic governing body, had called for Ligue 1 players to wear a shirt with a rainbow-coloured number on the back at this weekend's matches ahead of Wednesday's International Day against Homophobia, Transphobia and Biphobia.
However, a number of players declined.
The National Union of Professional Footballers (UNFP) said it was not up to the players to convey "collective messages". It was astonished that clubs should ask them to do so, it said in a statement.
Toulouse left out the players who did not want to wear the shirt. But the club's Moroccan defender Zakaria Aboukhlal (23) defended his choice not to take part in the campaign.
"Respect is a value that I hold in great esteem. It extends to others, but it also encompasses respect for my own personal beliefs. Hence, I don't believe I am the most suitable person to participate in this campaign," he wrote on Twitter.
Brest coach Eric Roy (55) called the gesture "catastrophic", a comment that drew widespread criticism on social media.
"You can see that there are players who have a problem with it. Everyone is free to express their opinions. Personally, I have no problem with it. But there are players who may have a problem with it," he said.
He also said it was wrong to schedule the campaign at a time when players were fighting to avoid relegation.
His Rennes counterpart Bruno Genesio (56) said that while he was against any kind of discrimination, he was "not sure it was a good idea to organise a day against homophobia".
Sports Minister Amelie Oudea-Castera weighed in on the controversy in support of the campaign.
"I think it is the responsibility of the clubs to take sanctions," she said. "During such an operation, which involves all the clubs on the basic subject of non-discrimination, you have to show up."
The LFP and the French Footbal Federation (FFF) have not commented.
At last year's World Cup in Qatar, world governing body FIFA threatened seven European teams with sanctions if they wore the "One Love" armband symbolising diversity and tolerance.
