Adnan Syed, whose murder trial was featured in the 2014 podcast Serial, has been told he will not get a new trial.
The Court of Appeals of Maryland, the state's highest court, on Friday reversed an earlier decision ordering a retrial.
Syed's lawyer Justin Brown said they were "devastated" by the decision but "will not give up".
The case for a retrial is centred on a witness who was not called in the initial trial.
But judges said that the witness would not have affected the outcome of the proceeding.
The decision was carried by four votes to three.
Mr Brown said: "We are devastated by the Court of Appeals’ decision but we will not give up on Adnan Syed. Unfortunately we live in a binary criminal justice system in which you either win or you lose. Today we lost by a 4-3 vote.
He added: "Our criminal justice system is desperately in need of reform. The obstacles to getting a new trial are simply too great.
"There was a credible alibi witness who was with Adnan at the precise time of the murder and now the Court of Appeals has said that witness would not have affected the outcome of the proceeding.
"We think just the opposite is true. From the perspective of the defendant, there is no stronger evidence than an alibi witness."
The lawyer later tweeted that Syed would continue trying to clear his name.
The Adnan Syed case was featured in the first season of the Serial podcast.
US TV channel HBO is due to air a documentary series called The Case Against Adnan Syed.
