Pants could also be exclusively feminine clothing, depending on the region and time period. Also, the colour pink used to be considered masculine (as we’re all shades of red).

What I’m saying is, you’re absolutely correct, and gender expectations are completely reliant on the culture factor.