As someone who does hit the gym, in my experience the majority of people you’re impressing with your physique are men. Sure I’d say most women like it if you have it (assuming you aren’t too big), but generalising that it’s the main attraction would be a mistake.