In a new interview, the star defended the pair over their decision to step down from royal duties.
Currently hosting the Late Late Show in the US, James spoke out as he prepares to leave the hit programme next year.
While speaking with The Sun, James supported the pair's choice to leave the UK, reports Birmingham Live.
He also revealed that he and his wife have play dates with Harry and Meghan.
James said: "I have. It was more a sort of play date scenario [with the kids]. Then we went out for dinner after, it was lovely. Obviously I’m not telling you any more. Obviously."
He went on to say: "Look, I’m a huge fan of both of them. Obviously I know Harry better than I know Meghan but I think it’s really hard to judge, or be judgmental. People process things and deal with things in different ways and that’s OK, and it’s all right to think whatever you think about somebody.
“I don’t know if it necessarily needs to be voiced the whole time but my experience of them has been nothing but positive. I have a huge amount of respect and admiration for Harry, and anyone who has been around him and spent time with him would feel exactly the same way."
He added: “I think he is a devoted and loving husband and father and I think what they have done is incredibly brave and I’ll always be in their corner. I have nothing but admiration and respect for them. I think it’s impossible to judge any of it really.”
Get the latest celebrity gossip and telly news sent straight to your inbox. Sign up to our weekly Showbiz newsletter here .