Dr Falke introduces himself to the audience before the story begins. Vienna in 1955 is still divided into four zones, controlled by the UK, USA, France and Russia. A Russian newsreel shows how Falke, dressed in a bat costume, was arrested after being found asleep in the arms of a Soviet statue. Falke explains that at a costume party he passed out after drinking too much, and that Rosalinda and her husband, Colonel Eisenstein of the French army, placed him on the statue. Falke is released and Eisenstein arrested.
At Eisentein's hotel, Captain Alfred of the US Air Force checks in and recognises Rosalinda from their liaison in Paris some years before. He takes the room next to hers in the hope of rekindling their romance. Eisenstein returns and tells Rosalinda that he will have to spend eight days in the international barracks since Falke was able to produce photographic evidence of the practical joke.
Falke goes to see Eisenstein and admits that he re-staged the joke for the cameras to get his own back. To placate him, he invites Eisenstein to General Orlovsky's party, telling him that he will make arrangements with Major Frank so that he can enter that barracks at midnight. Falke calls Frank, but instead only invites him to the party. He also gives an invitation to Rosalinda's maid, Adele.
Eisenstein goes to the restaurant, pretending that he will go directly to the barracks after having a final meal with Rosalinda. Alfred climbs over his balcony and asks Rosalinda to have a drink with him. She refuses, but isn't completely discouraging. Eisenstein and Rosalinda have their farewell dinner, served to them by Alfred, who has disguised himself as a waiter. They both sing of their imminent parting, both thinking of their alternate plans for the evening.
Eisenstein leaves, and Alfred asks Rosalinda to have a drink with him, having put on Eisenstein's robe. Frank arrives and mistakes Alfred for Eisenstein. Rosalinda convinces Alfred to go along with the charade for the sake of her reputation. Falke phones Rosalinda and tells her that a woman at the party is wearing one of her dresses and that she is dancing with a French gentleman who has escaped from the International barracks. Rosalinda is furious and goes to the party in disguise.
Eisenstein is outraged when he recognises Adele in one of Rosalinda's dresses, but she pretends to be an actress and, since he is also incognito, he is forced to let the matter drop. The General appears at the party and sings a song to peace before dancing with Adele, who has also attracted the eye of Frank.
Rosalinda arrives and Eisenstein is immediately drawn to her. He pursues and seduces her. She runs away with his watch, but Falke tells him that it will turn up again at home. At midnight, Frank and Eisenstein say good night to each other, unaware of their true identities, and head separately for the international barracks. Frank is seeing double as he is the worse for wear and is told by his assistant Frosh that there are two Eisensteins.
Alfred gets talking to Eisenstein, who quickly realises that the American is wearing his robe and was dallying with Rosalinda. In a fit of passion he rushes out and confronts Rosalinda. She retorts that he is a charming man but is frequently mentally unfaithful. He asks for proof, and she shows him his watch under her bed covers. He begs for forgiveness, unaware that Falke and the rest of Orlovsky's party are beneath their window listening to them.
Falke admits to Eisenstein that the whole scheme was of his doing. Alfred lets Eisenstein think that he was also part of the practical joke. Orlovsky sings, asking that they all drink a toast to love and wine. Falke addresses Alfred, Eisenstein, Orlovsky and Frank, reminds them that it is late and that they should all go home. The Allies have now been in Vienna for quite long enough and should return soon, but only as guests. Alfred lets American soldiers arrest him in Eisenstein's place so that Eisenstein and Rosalinda can be reconciled, while everyone dances and sings a toast to friendship.
