"Our family has made the decision to part ways with Donda Sports," reigning Super Bowl champion Donald and his wife Erica said in a statement.
"The recent comments and displays of hate and antisemitism are the exact opposite of how we choose to live our lives and raise our children."
Brown, who led the Celtics to the NBA Finals last season, on Monday told the Boston Globe he planned to stay with Donda Sports but on Tuesday said he had a change of heart.
"I have always, and will always, continue to stand strongly against antisemitism, hate speech, misrepresentation, and oppressive rhetoric of any kind," he said in a statement.
The pair are the highest-profile athletes associated with Donda Sports, which represents the athletes' interests outside the field and plays no part in negotiating their contracts with their teams, according to Yahoo Sports.
West, who legally changed his name to simply Ye last year, could not be immediately reached for comment.
The multiple Grammy award-winning artist has been locked out of several of his social media accounts following a rash of offensive behavior by the rapper and fashion designer.
Earlier this month he said he was "going death con 3 on Jewish people" in a tweet that the platform later removed.
Athletic apparel brand Adidas also terminated its partnership with West on Tuesday, a move that knocked the musician off the Forbes list of the world's billionaires.
"Adidas does not tolerate antisemitism and any other sort of hate speech," the German company said.
"Ye's recent comments and actions have been unacceptable, hateful and dangerous, and they violate the company's values of diversity and inclusion, mutual respect and fairness."
