Why, indeed? In my case, I think I always had the sense in that my desires were a weird fetish. Felt embarrassment every time I time went to the register with a muscle magazine. When my freshman hallmates sorted it out -- I guess it was a muscle mag under my bed that maybe was open to the 'wrong' page, or it was clear which pages I was looking at -- they made fun out of it, suddenly rushing me into someone's room to watch a Ms. Olympia that was on TV. I learned to hide it.
Keeping it from a partner isn't easy in our technological age, with browser histories and such, but I usually tried. Maybe out of fear I'd be rejected, but also because it would probably be misunderstood, given that few women I've dated were the least bit athletic. Why say you're turned on by women who are so different and that you masturbate to them? Unless you have reason to think that your partner might be willing to work towards that, it can't help the relationship. I mean, it's not like a gf telling you that she needs a bigger dick to be satisfied (you can't change that), but it's close.
I'm sorry your marriage has lost its intimacy. I can't comment on why, but maybe the fetish is an issue? Perhaps not, if you're new to the forum, maybe it just hasn't been a big deal in your life, as it has been for many of us. Sessions can be fun, but would never be worth wrecking a relationship for, especially if you have hope of restoring that intimacy.
It's weird -- it's not like being gay, or suddenly realizing one has homosexual interests. There's a whole community out there that is supportive and real chance of finding intimacy with a range of partners that fill that piece of your sexuality. If you're into women bodybuilders, there's not much chance of finding partners that fit the bill there.