The historic Beefeater guards are facing layoffs for what is believed to be the first time in their 500-year history.
The redundancies are part of "heartbreaking" measures to cut costs at the Tower of London following the coronavirus pandemic.
The global crisis has seen the temporary closure of six sites run by Historic Royal Palaces (HRP), which all rely heavily on visitor income.
A spokesperson for HRP confirmed a voluntary redundancy scheme had been introduced last month and that staff had been warned a compulsory redundancy scheme was likely to follow.
HRP believes it is the first time the guards, recognisable for their decorative red and black uniforms, have faced redundancy in their long history - having been formed by Henry VII in 1485.
PA reports at least two of the 37 Yeoman Warders, nicknamed Beefeaters, who guard the Crown Jewels, have reportedly taken voluntary redundancy already.
HRP chief executive John Barnes said the organisation had "simply had no choice" but to make the cuts.
He said: "Historic Royal Palaces is a self-funded charity. We depend on visitors for 80% of our income.
"The closure of our six sites for almost four months has dealt a devastating blow to our finances, which we expect to continue for the rest of the financial year and to be compounded by the slow recovery of international tourism.
"We have taken every possible measure to secure our financial position, but we need to do more to survive in the long term.
"We simply have no choice but to reduce our payroll costs."
Mr Barnes added the Beefeaters remained a valued part of the Tower of London and would "continue to be part of the Tower's story in the years to come".
He added: "We are heartbroken that it has come to this."
HRP added there would still be "sufficient staff" to run the site.
Keep up to date with coronavirus cases in your area by adding your postcode below:
The Tower of London is HRP's largest paid for attraction by some margin and normally attracts around three million visitors per year.
The site reopened on July 10 but can now welcome fewer than 1,000 people each day due to new safety measures.
Provisions to prevent further spread of Covid-19 include one-way routes and hand-sanitising stations throughout.
Tourists wanting a photograph with a Beefeater now have to do so while standing two metres away.
