You're right. Last time it happened, I was standing in a stall taking a leak when a woman walked in. When she saw me, she gasped "sir, this is for women only." Without missing a beat, I replied "so is this, but every now and then I need to run some water through it."
