Equalities minister allegedly referred to trans women as 'me