The disconnect is thinking that you can't love them if you also don't have sex with them. I'll explain.
Of course you love your wife for more than sex - you wouldn't have married her otherwise (I hope). There's a lot they bring to the relationship. HOWEVER, us men being men, we will eventually be unhappy without steady sex.
Why does the person giving you sex have to be the same person you partner with? I'm not talking some open relationship etc in general (although whatever people do is fine) I'm just saying when clearly the other person doesn't want or can't have sex, why does that mean the end of the relationship? Why can't she see her way clear to understanding your needs and either seeing to it that they get fulfilled herself by bringing girls home and being a participant, or allowing you to fulfill your needs on your own?
The conventional wisdom is that you're an asshole if you step out or just say "I need sex", but we are only human. You will only get one life, and your opportunities to get your dick wet will dwindle as you get older. It's like saying "I simply choose not to travel to 90% of the world, because I love my wife and she won't travel. I can't go without her either, because of social norms, so just fuck the rest of the world. I won't experience it. The end." ?? What?