Former Tory leader Iain Duncan Smith has said Theresa May must set an "immediate date for departure" following the party's disastrous performance at the local elections.
Mr Smith, a Brexiteer who has regularly stood at odds to Mrs May, said she appears to be a "caretaker" Prime Minister.
He urged the embattled Prime Minister to either set a date for her exit or for senior Tories to do so for her.
"We have to make a change," he told LBC, after stating the results were a "shockwave at the leadership".
It is believed the results were a sign of voters expressing frustration at Mrs May's failure to have taken Britain out of the European Union as of yet.
Anger at the mounting scale of losses saw Tory leader Theresa May heckled as she gave a speech in Wales, with a man shouting: "Why don't you resign? We don't want you."
However, despite their being criticism levelled at Mrs May, some ministers called for unity within the party.
Justice Secretary David Gauke branded the losses a "punishment" for the Tory's response to Brexit and said that should be their focus.
Mr Gauke told BBC Breakfast: "What we need to be doing is addressing the big issue in front of us, which is Brexit. We would have had a much better set of election results had we managed to get the Prime Minister's meaningful vote through earlier this year and we left the European Union on March 29.
"I think we can look at those local election results as a punishment for both the Labour Party and the Conservative Party for failing to find a way through that situation."
Health Secretary Matt Hancock said the party needed to listen to the results and "be in a mood for compromise", while Foreign Secretary Jeremy Hunt said "purists" in the party were putting the Brexit "in peril".
When asked who was responsible for the losses, Mr Hunt told reporters in Africa: "You can look at lots of different groups of people - you can look at Brexit purists in my party who have consistently refused to compromise and put Brexit in peril.
"You can for sure look at Government - I'm sure that there are things we could have done differently in the course of the negotiations. And you can look at the Labour Party who have played politics consistently."
Mrs May admitted she had always expected the elections to be tough for the Tories as counts in England were coming in on Friday.
Mrs May said: "These were always going to be difficult elections for us with us nine years into a government.
"Of course there is the added dimension we haven't got the Brexit deal over the line."
She also said the elections as a whole carried a "simple message" for both the Conservative and Labour parties: "Just get on and deliver Brexit."
