Labor does not believe stripping Christian Porter of some responsibilities will be enough for the attorney-general to return to work.
Opposition Leader Anthony Albanese is not satisfied with a proposal to delegate Mr Porter's functions relating to the Federal Court and ABC while he pursues defamation action over historical rape allegations.
Mr Albanese said there were a range of other duties, including overseeing the Sex Discrimination Act, the attorney-general would still need to fulfil.
"It is going to be very difficult for the attorney-general to just return to work and to pretend that all of us can unsee and unhear what has been said over recent weeks," he told ABC radio on Wednesday.
He said the prime minister must satisfy himself and the Australian people that Mr Porter was a fit and proper person to serve as a cabinet minister.
"We think that there needs to be a proper examination of the allegations that have been made," Mr Albanese said.
"That hasn't been possible by the NSW Police due to the circumstances of the woman taking her own life."
The government has rejected calls for an independent inquiry into the allegations.
Mr Albanese fears allowing Mr Porter to return would undermine confidence in Australia's system of government.
Mr Porter is seeking aggravated damages over a story that detailed historical rape allegation against a cabinet minister.
His lawyers allege the story - which did not name Mr Porter - was defamatory because it imputed he raped a 16-year-old girl in 1988 and that contributed to her taking her own life.
The minister identified himself as the subject of the article almost two weeks ago.
His lawyer said he was forced to go public after a series of news articles, social media posts and interviews made him easily identifiable to many Australians.
Mr Porter is due to return to work after mental health leave on March 31.
Prime Minister Scott Morrison has proposed to delegate some of Mr Porter's responsibilities while the court case is afoot.
Mr Morrison told parliament Mr Porter would step back from certain aspects of the role out of an abundance of caution and to avoid any perceptions of a conflict of interest.
