The state's peak farming body says it is "hugely unlikely" Port Botany will "ever be as attractive or efficient as the Port of Newcastle" when it comes to agriculture exports.
NSW Farmers has lodged a submission to a government-initiated independent review of ports legislation and its Port Botany Landside Improvement Strategy.
The farming body has criticised controversial port privatisation deals in NSW which penalise Port of Newcastle if it develops a container terminal rival to Port Botany.
It argued in its submission to the review that, without significant improvement in rail access to Botany, supply chain challenges would increase over time and reduce the competitiveness of NSW farming exports.
NSW Farmers president Xavier Martin said on Tuesday that farmers wanted the "obvious artificial constraints" of the Newcastle container penalty removed.
He said government attempts to improve freight operations at Botany could be doomed to fail.
"It doesn't make a lot of sense to go to the eastern side of our largest city in this nation. It can't take big trains, and trucks just add to the congestion," he said.
"We have three ports, and we need to get them up to scratch. Port Botany's got significant limitations that Kembla and Newcastle don't have."
Mr Martin said the NSW Farmers submission highlighted potential improvements at Botany but "they've got an artificial constraint in terms of how they've built out the place with the city".
"It's hugely unlikely to be as attractive or efficient as the port of Newcastle," he said.
The Newcastle penalty was "costing the state serious money".
"It's a real problem in this era of inflation and need for our economy to recover from its COVID hit.
"We've got agriculture as an absolute dynamo primed with moisture to grow healthy animals and healthy plants, and yet we can't get the product efficiently to market, whether it's here or overseas.
"We're getting this cost-price squeeze. The inflation's hit the inputs and made them very costly, yet we can't get our exports out efficiently.
"Very quickly the prosperity that was promised vaporises. If we get the settings right, agriculture can absolutely fly."
Mr Martin welcomed the arrival of two mobile harbour cranes in Newcastle on Tuesday as the port attempts to diversify its operations within the constraints of the container penalty.
He said both sides of politics should commit before the March state election to "strategic investment" to achieve "significant improvement" in the freight network.
"We need to see the service quality improved and the costs reduced."
