Meghan Markle returned to Canada just hours after the bombshell announcement that she and husband Prince Harry are taking a "step back" as senior royals.
According to newspaper reports, eight-month-old Archie stayed with his nanny in the country while his parents flew back to London.
The Queen and other members of the family were said to be "hurt" by Harry and Meghan's decision to issue a personal statement, which outlined their future lives as financially independent royals who will divide their time between the UK and North America.
The Queen, the Prince of Wales and the Duke of Cambridge have ordered their teams to find a "workable solution" over the future role of the Duke and Duchess of Sussex within the Royal Family.
A Buckingham Palace source said the senior royals have asked aides to work "at pace" with governments and The Sussex Household and an outcome was expected to take "days, not weeks".
The PA news agency understands the Royal Family will talk to multiple governments which suggests US and Canadian officials may be involved.
If they were to meet in person, the royals seem to have the next five days free of any public engagements.
The Duke of Cambridge is due to be in Yorkshire for a number of appearances next Wednesday, alongside the Duchess of Cambridge.
Meanwhile, Harry is not expected to be seen in public again until next Thursday, when he will do the draw for the Rugby League World Cup in 2021.
Given the Queen's timetable of "days" it is likely they will want any talks concluded before the Prince of Wales travels to Israel and the Occupied Palestinian Territories at the end of the month, to mark the 75th anniversary of the liberation of the former Nazi concentration camp, Auschwitz-Birkenau.
The latest developments reveal the divisions within the heart of the British monarchy which has already been rocked by the Duke of York's television interview about his relationship with convicted sex offender Jeffrey Epstein.
The couple's statement - and a new official Sussexroyal.com website - have thrown up important questions about funding for their round-the-clock security, media access to their royal events and how they will pay for their future lifestyles.
The Duke and Duchess, who have only recently returned from a six-week stay in Canada, said in their statement: "After many months of reflection and internal discussions, we have chosen to make a transition this year in starting to carve out a progressive new role within this institution.
"We intend to step back as 'senior' members of the royal family and work to become financially independent, while continuing to fully support Her Majesty the Queen."
Meghan received a public boost from her close friend Jessica Mulroney, a Canadian stylist and TV star, who posted a quote attributed to Gina Carey, which said: "A strong woman looks a challenge in the eye and gives it a wink."
The couple was also supported by writer and journalist Bryony Gordon, who defended the Duke and Duchess in a column in The Daily Telegraph.
"Their decision to 'step back' can only be positive, a move that will enable a young family to live their lives properly while still flying the flag for Britain," Ms Gordon wrote.
"While some have chosen to interpret the news as the couple wanting to enjoy a lavish lifestyle without any of the scrutiny, a more accurate reading of the situation is that they want to be able to express their views and pursue their campaigning passions without coming under constant fire for doing so."
However a series of commentators have strongly criticised their decision with public relations and crisis consultant Mark Borkowski saying about the Sussexes "It does feel like a royal couple in exile".
It is already known the Queen, Prince of Wales and other senior royals were not consulted about the content of the couple's statement or knew it was to be issued.
But the Evening Standard reports that Charles and the Duke of Cambridge only received it 10 minutes before it was released.
Buckingham Palace would not comment on the latest developments.
