His first wife Joan was a dancer, who used the stage name Zelda Burns/Barry; together they had a son, Gilbert. He also had a son,
Nick Emery, with his third wife Iris, and two children,
Michael Emery and
Eliza Emery with his fourth wife, Vicki. For the last 4 years of his life he lived with
Fay Hillier.