Intentionally waving a Nazi flag in NSW or displaying memorabilia bearing swastikas can land a person in jail for up to a year, along with a fine of $11,000.
The Crimes Amendment (Prohibition on display of Nazi symbols) Bill 2022 swiftly passed in the upper house on Thursday with unanimous support.
It comes after an inquiry earlier this year recommended a ban on the public display of Nazi symbols in a bid to tackle rising anti-Semitism.
NSW becomes the second state in Australia to pass the landmark legislation after Victoria in June.
NSW Jewish Board of Deputies CEO Darren Bark described the passing of the law as a historic day for NSW.
"Nazi symbols are a gateway to violence and are used as a recruitment tool by extremists," he said.
"Banning their display is a long-overdue and much-needed law in our state. The perpetrators will finally be held to account".
The Holocaust is the genocide perpetrated by the Nazi regime in Germany that killed some six million Jews and other minority groups including homosexuals, Black people and Roma people during World War II.
NSW Attorney General Mark Speakman said the passing of the bill marks a significant moment in the state's repudiation of hateful ideology.
"This new offence sends a clear message that the display of Nazi symbols, and the hatred and bigotry they represent will not, and should not, be tolerated," he said.
"This new criminal offence will provide important, additional safeguards against hate speech and vilification in our state."
A section in the bill allows for the swastika symbol to be used in academic, historical or educational settings, thereby paving the way for its display by Hindus, Buddhists and Jains in which it holds religious significance.
"For too long, the Hindu community has not felt comfortable to display our symbol of peace because it resembled a symbol of evil. This is no longer," said Hindu Council of Australia national vice-president Surinder Jain.
Mr Bark noted the legislation is also "a game-changer in tackling online hate" and called on tech companies to ramp up efforts to remove imagery and symbols associated with Nazism.
Labor's Walt Secord, a member of the parliamentary committee examining the ban of Nazi symbols and an ardent advocate for the bill, said 31 incidents of displaying the Nazi flag were reported to the police in 2020.
Many members from the government and opposition in the upper house recounted personal stories of their families' lived experiences enduring the Holocaust while others warned of the dangers of rising neo-Nazi trends.
On Thursday, Mr Secord referred to a NSW man arrested by counter-terrorism police in September found to be in possession of a Nazi flag and a map of the state on his bedroom wall with plans to make a 3D-printed gun.
ASIO said in 2020 that far-right violent extremism with its emphasis on neo-Nazi ideology makes up around 40 per cent of its counter-terrorism caseload.