No, your instincts are correct here. That's just a shitty thing to do. I really hope you say something to him the next time you see him. That's hurtful, and he needs to know it's hurtful. Just something simple like "thanks for the invite to the fishing trip I suggested." Then walk away. Might seem passive aggressive, but he is a dick and he deserves to feel uncomfortable/badly for something like this.
I'm having the devil of a time with my parents in law. They both have been diagnosed with Alzheimers. Last time I saw them my father in law didn't remember who I was, which hurt, not because he was being a dick, but because he and I had a really great relationship. It's hard to see him disappearing because of the disease. He got very agitated and I had to leave their condo to get him to calm down. Half hour later I came back and he greeted me joyfully. He then proceeded to start up a discussion about a piece of music he and I discussed to death about 15 years ago, picking up that conversation like it had been happening ten minutes before.
My mother in law has always been less than pleasant. Now she's just mostly silent. She seems to be more aware than her husband of the freedoms that are being taken from them. No car or license, no bank or credit cards (she was online buying so much that it was a massive problem), no ability to just get up and go on a vacation or anywhere. She's just getting angry, and she is focusing that on me. She remembers that I was there to move them into their assisted living place, and she's blaming me for her having to be there.
Alzheimers is a really, really fucking shitty thing. The only positive that has come out of this is that my wife and her brother have become a lot closer as they together deal with it. Another down side is that the other brother has become a total dick in this, saying that the other kids are excluding him in decisions (they are, because he's a waste of skin and tries to override decisions by feeding fear and descension to his parents in their confused state).
You don't get to pick your own family. And really, you don't get to pick the family you marry in to. It's tough out there. Be good to yourself, don't let parents drag you down.