The alleged incident happened during the first half of the game on Sunday night when the Valencia defender Diakhaby challenged Juan Cala for a ball.
As Diakhaby was heading back to his position, he suddenly turned and angrily ran back towards Cala.
Teammates Gabriel Paulista and Kevin Gameiro tried to calm the 24-year-old down and pull him away from Cala.
After referee David Medié Jiménez had booked Diakhaby despite his explanations, the Valencia skipper José Luis Gayà led the team off the ptich at the Estadio Ramon de Carranza.
Suspension
Cadiz eventually followed suit as match officials attempted to sort out the situation.
Ximo Puig, president of the Valencia’s regional government, wrote on Twitter: “Equality and rejection of racism. That is the most important match. A hug to Diakhaby. This is his house, his land.”
Injured Valencia teammate Eliaquim Mangala tweeted that Diakhaby would not have stopped playing unless there was a good reason.
Valencia claimed that Jiménez told them of the possible sanctions that awaited them if they did not return to the pitch.
They went back out after 20 minutes in the dressing room without Diakhaby who was replaced by Hugo Guillamon.
“We were told that if we refused to return to the game, we could lose the three points or more," said Gaya.
“Diakhaby then asked us to go back on the field and fight for the club. So we listened to him. Without his permission, we would not have resumed the match.
"He was very down and didn't feel strong enough to play. Seeing him so badly, I have no doubt that he was really a victim of racist insults.”
Denial
Cala, who scored the opening goal in the 2-1 victory, told the referee he had done nothing wrong and gave the same explanation to his coach Alvaro Cervera.
"We did not know how to react,” said Cervera, “This is the first time this has happened to me in my career. Cala assured me that at no time had he insulted anyone and I believe him."
Cadiz said after the game that it was against all forms of racism or xenophobia.
“Anyone guilty of such an offence, whether or not they belong to our team, must be punished,” it added in a statement.
A statement from Valencia said: “We are proud of the support that Diakhaby received from his teammates and of the decision taken to leave the field of play together. We trust that the matter will be investigated.”
