Sampaoli, 60, quit his post as boss of Atletico Mineiro in Brazil to replace André Villas-Boas who left after disagreeing with the club's transfer policies.
His arrival at the Vélodrome comes at a gloomy moment.
A strong side including seasoned peformers such Arkadiusz Milik, Dimitri Payet and Valère Germain started the last-32 tie on Sunday night at the Stade Brutus.
But it was a tough night for the vaunted visitors.
Jérémy Posteraro gave Canet a 21st-minute lead with a brilliant free-kick which beat veteran goalkeeper Yohann Pélé.
Parity
Poland striker Milik levelled seven minutes before the break with his third goal since joining Marseille from Napoli.
But home winger Yohan Bai scored 19 minutes from time, dinking the ball over Pele, and the hosts held on on for a famous victory.
"My players were heroic, courageous," said Canet coach Farid Fouzari. "In the stands ... the stewards turned into our supporters."
The Coupe de France has proved a slough of despond for Marseille. They have only been past the quarter-finals once in the last 14 years. That milestone came in 2016 when they reached the final and lost to 4-2 to bitter rivals Paris Saint-Germain.
"I wanted to put the team in the hands of the new coach with a victory," admitted interim coach Nasser Larguet.
"It's a big personal disappointment ... The players did not understand that it was necessary to make even more effort. I thought that after the equaliser we would improve our level of play. We did not."
⏱️ 90+3 | #CRFCOM 2⃣-1⃣
— Olympique de Marseille (@OM_Officiel) March 7, 2021
Fin du match. Place au championnat mercredi avec la réception de Rennes. pic.twitter.com/1FxnOYm04h
Sampaoli, who has led the Chile and Argentina national teams during an 18-year coaching career, will be in charge for Marseille's next match - a home game in Ligue 1 against Rennes.
