I'll admit that I used to be one of those judgemental people, before I thought about the fact that everyone does their own kind of grooming in different ways. (i.e. makeup, hair gel, tanning, hair removal, dental care etc.)
On the one hand I can respect people who visit the gym regularly, as I don't have the same motivation. On the other hand, I can't imagine wanting to put in the time to do so, since those 1-2 hours several times a week are what I spend honing other skills like art and music, which will last me a lifetime.
Plus, when it comes to physical attraction, I'm generally not pulled towards the idea of people putting "effort" in. If they're clean and in clean clothes, they're reet by me. Muscles/a toned physique don't do anything for me, but that's just personal preference, and I still respect the dedication.