The “heirs to the skinheads and the Paki-bashers of old” were among those who took to the streets of London for the Unite the Kingdom rally, the Home Secretary said.
“While not everyone chanted racist slogans, some did. Clear that in their view of this country, I have no place,” Shabana Mahmood said.
Between 110,000 and 150,000 people turned out for the rally organised by far-right activist Tommy Robinson on September 13, according to the Metropolitan Police.
At the Labour Party conference, Ms Mahmood said that “division within this country will grow” if her party fails to address immigration concerns.
In her speech, the Home Secretary set out a series of conditions for those seeking indefinite leave to remain status.
Under proposed changes, migrants who want to remain in the UK will have to learn English to a high standard, have a clean criminal record and volunteer in their community.
They will also need to be working, paying national insurance and not be claiming benefits.
Speaking on Monday, Ms Mahmood told the conference: “Just days into this job, on September 13, 150,000 people marched through London.
“They did so under the banner of a convicted criminal and a former BNP (British National Party) member.
“While not everyone was violent, some were, 26 police officers were injured as they tried to keep the peace.
“And while not everyone chanted racist slogans, some did. Clear that in their view of this country, I have no place.
“It would be easy to dismiss this as nothing but an angry minority, heirs to the skinheads and the Paki-bashers of old, and make no mistake, some were.
“But to dismiss what happened that day would be to ignore something bigger, something broader, that is happening across this country.
“The story of who we are is contested.”
Ms Mahmood said that working-class communities will turn away from Labour and “seek solace in the false promises of (Reform UK leader Nigel) Farage” if the Government fails to act on migration concerns.
She said: “They will turn towards something smaller, something narrower, something less welcoming, and the division within this country will grow.
“So the challenge we now face is this, not just to win the next election but to keep the country together and to fight for our belief in a greater Britain, not a littler England.”
Ms Mahmood also said “patriotism, a force for good, is turning into something smaller, something more like ethno-nationalism”.
The minister defended the Government’s decision to place conditionality on indefinite leave to remain, arguing it is “right” because it is “a two-way street”, when speaking at a fringe event.
Ms Mahmood said: “I think sometimes people might feel that adding conditionality maybe creates two tiers of citizenship potentially, or it’s asking too much of people.
“But what I would say is, first, I think it’s right that, as a country, we decide that before you are allowed to remain here for good, there’s some things that you have to do.
“And it’s a two-way street, there’s a reciprocal relationship that’s being created here.”
A consultation on the changes to the indefinite leave to remain status will be launched later this year.
In its white paper published in May, the Government also pledged to increase the amount of time migrants have to wait before they can apply for indefinite leave to remain in the UK, to 10 years.
Currently, most migrants who come to Britain on time-limited work visas can make an application for indefinite leave to remain after five years.
Speaking at a Spectator fringe event hosted by Lord Michael Gove, Ms Mahmood said anti-Muslim hatred is now “on a scale that I’ve never known in my lifetime”.
“When I was a child, I think I was seven or eight years old, that’s the first time I heard the word ‘Paki’… so it’s not as if I haven’t been racially abused before,” she said.
Ms Mahmood added: “But what is happening now is something much deeper and much more pervasive, and it does feel like it’s everywhere at the moment.
“Members of my own family, just in the last couple of weeks, you know, a handful of them have been called ‘f****** Paki’ in Birmingham, in places that I go to regularly with my family.”
The Refugee Council has urged the Government “to rethink this policy, which punishes refugees for needing help, and tells them that no matter how hard they work in the future, they’ll never have a safe permanent home in Britain”.
Chief executive Enver Solomon said: “These proposed changes disproportionately impact refugees – and far from encouraging integration and contribution, will erect even more barriers for those who have come here in search of safety. Many refugees have come here after fleeing war and persecution in places like Afghanistan or Sudan and they want more than anything to feel part of and give back to Britain.
“We see in our frontline services how so many refugees have received their indefinite leave to remain and gone on to build lives as doctors, entrepreneurs and other professionals. But that may not have been possible without a safety net.
“It is much harder for a refugee to find their feet straight away, especially when they are not allowed to work while waiting for their asylum decision, and if they have been given less than 30 days to find somewhere to live.”
