I'm equally alarmed, Mozz, that TL doesn't seem to know the difference between 'transvestite' and 'transexual'. It's a very big difference too!
The solution is simple and that's to abandon the idea of male and female 'safe spaces' altogether. Most swimming pools and leisure centres no longer have male and female changing rooms; they simply have 'changing villages', where both men and women use individual cubicles to change. Similarly many public venues (such as Somerset House) no longer have men's and women's toilets. They simply have toilet areas consisting of individual cubicles (with no urinals) and shared hand-washing facilities. (It's hardly a new idea. Many smaller restaurants, such as the majority of Indian restaurants, have only ever had single toilets anyway).