Millwall and Rowett parted ways on Wednesday after four years together, with the club saying in a statement the decision was mutual.
Rowett had guided the Championship club to top-half finishes in each of his fours years, but failed to make the play-offs. Millwall had made a mixed start to this season and are currently sat 15th in the Championship.
“If I’m being honest, it just felt the right time,” Rowett told talkSPORT on Thursday morning.
“You know as a manager and you know as a club - it just feels like it needs something a little different.
“That was kind of fundamental to the decision. It was a mutual thing. I know sometimes people say: ‘Oh, he’s been sacked and that’s how they have put it out’ - but it genuinely was [mutual].
“I spoke to the chairman last night and I’ve got a good relationship with everyone at the club.”
Current assistant manager Adam Barrett will take charge of the team for Saturday’s trip to Preston, but the search for Rowett’s successor is already underway.
Millwall have already been contacted by interested applicants and will not rush to get someone in place.
Former Millwall midfielder Kevin Muscat is the current bookmakers’ favourite. He guided Yokohama F. Marinos to the title in the Japanese top-flight last season after succeeding Tottenham boss Ange Postecoglou.
Muscat, who captained Millwall to the FA Cup final in 2004, was interviewed for the Rangers job earlier this month but missed out to Philippe Clement.
John Eustace is also an early frontrunner after being sacked by Birmingham City earlier this month, with the Blues hiring former England captain Wayne Rooney.