Nottingham Forest goalkeeper Aro Muric helped Kosovo maintain their run as Europe's surprise package.
Kosovo are now unbeaten in 14 games - the longest unbeaten run on the continent - and top of Group A after beating the Czech Republic at home on Saturday.
The visitors took the lead after 16 minutes when Patrik Schick gave Muric no chance with a low finish.
It was 1-1 two minutes later though through Vedat Muriqi. Mergim Vojvoda got the winner for Kosovo after 66 minutes.
Muric was playing for the first time since Forest's draw at Chalton last month and he showed one sign of rustiness when he came for a cross and was missed the ball.
Fortunately for him, it was cleared off the line as Kosovo held on, meaning they go into Tuesday's game against England full of confidence.
