I'm perfectly OK with people wearing whichever they choose, just as long as they know the difference and don't call a cap a hat.
https://www.merriam-webster.com/thesaurus/cap
Synonyms for cap
chapeau, hat, headdress, headgear, headpiece, lid [slang]
Merriam Webster seems to thing it's a suitable synonym.
Nanner nanner nanner!