I'm not sure how relevant this is as I wouldn't consider myself a schmoe, but anyway:
I can only speak for myself, but I've been attracted to in-shape people for most of my life, but it really ramped up in my mid-late 20s. I used to be very sporty myself. I played football/soccer, I did jiujitsu, competed in sailing on a national level, and more. Borderline chronic injuries due to wear and tear, improper training, warm-ups, or just bad genes have resulted in me not being able to be as active as I'd like. I'm no couch potato, and I still try to take care of myself as best I can, but there are periods where I'm able to do very little, which I've had to come to terms with. I've found solace in video games and books, but I would prefer to be out there playing sunday league football or something on the regular x)
That said, internet-dwellers don't represent everybody who's into muscle girls, and as such, the views and behaviour of these individuals do not represent the views and behaviour of the masses. The "schmoes" that you speak of are a minority. A very vocal minority, but a minority nonetheless, but it's worth keeping in mind that they're also the people who allow the muscle girl content creators to keep creating content by spending a fortune on said content, so it's very much a double-edged sword, lol.
Your local gymbros are probably on IG/TT and use them liberally like other people do, but most of them are more than likely to be nice/regular people who don't seek out these women with selfish and malicious/predatory intent.