A Brexit event and an anti-racism event are due to take place in Swansea this Saturday.
South Wales Police is aware of what is planned on April 27 and, while acknowledging the right to peaceful and lawful protest, said "necessary action" would be taken against anyone acting illegally.
The party For Britain said one of its representatives had worked tirelessly to put the Brexit event on.
For Britain had inquired about staging it at Castle Square, but the Stand Up To Racism event is due to be staged there instead.
Concerns have been raised about leaflets denoting Swansea and Gower's three MPs as "traitors", which appear to have come from a different group altogether.
Speaking at a meeting of full council on April 25, council leader Rob Stewart warned of a "far right" element, and urged councillors who wished to support the Stand Up For Racism event to be vigilant.
"There is potential for disruption or for other issues to occur," said Cllr Stewart.
17-year-old charged with wounding and possession of offensive weapon after alleged Swansea assault
A spokesman for For Britain said it was his understanding that a wide range of people would be attending and supporting "this important event".
He said For Britain was concerned about the lack of the implementation of Brexit since the 2016 referendum, but that it was not a single issue party.
He claimed "left wing" Swansea Council opposed free speech by not allowing the event at the square and had gone into "full sulk mode".
Cllr Stewart said the council had not prevented For Britain coming into the city, but that it was not able to hold a political rally at the square. He said Stand Up To Racism's event was a community cohesion event.
This 'despicable' racist graffiti was sprayed on a Swansea mum-of-three's home
In a statement, South Wales Police said it was aware of an event planned in Swansea on April 27.
A spokeswoman said: “We acknowledge the right to peaceful and lawful protest and we are working to engage with organisers to facilitate this, while minimising disruption to the community and ensuring public safety.
“We will not hesitate to take necessary action against anyone who deliberately chooses to act outside the law.”
A council spokesman added: "Swansea is a city of sanctuary, a welcoming city that supports all our communities.
"We would condemn any behaviour or threats that aim to undermine our reputation for tolerance and support the strong message from the police that they will take any necessary action against anyone who chooses to act outside the law.”
