When Victor and Mrs Warboys get locked out in the rain, they wait at Mr Swainey's house until Margaret gets home. However, as Margaret does arrive home, Victor lets himself and Mrs Warboys in via the back door and asks Margaret where she has been - meaning he could have got into the house all along.
At approx 22 minutes, when Jean is talking to Victor, to the left of screen you can see a crew member's finger closing the front door.