It's totally possible to support and respect a cause without wearing a token. It's a very nice gesture and I think all the people who put time into making displays or even just finding a poppy to wear are brilliant. But it's not the only way to remember. Respect doesn't have to be waved around or pinned to your shirt.
As time goes on things like this become less directly relevant to people. Poppies are usually associated with the older wars (although I believe proceeds go to newer service people too) and there's far less of a connection between those events and younger generations. Pride is something that is directly affecting a lot of people right now so it makes sense that it and other things like it would get more in the way of obvious support. It doesn't mean people don't care about it or respect it, but it isn't necessarily something that's relevant enough to them to insight much support. There are a lot of other causes with or without days dedicated to them that people support. Very few involve a physical symbol. Nowadays rememberace in the physical sense (poppies, parades etc) is more of a tradition than anything else.
Also it does have quite a specific focus (British soldiers) while I think nowadays we've started to widen our view and think about historical events effecting people from countries other than our own. And if you're doing that then putting more effort into one specific aspect doesn't make as much sense.
Personally I've done a fair bit in the past for remembrance day, but now I tend to honour it less openly. I'll think about the significance, but I tend not to wear a poppy or make any obvious show of my support. I just don't feel like it's necessary. There's a lot of very valid causes and thigns to think about. I can't wear a pin for all of them and I think the thought and lesson learnt from it is far more important than the display. That's where my focus goes and I think it's quite dismissive to assume that people don't care just because they don't have some red paper pinned to their shirt.
Also even when I was doing more to actively support rememberace I wasn't always wearing a poppy. People change their clothes and sometimes they forget to move the poppy over, it falls of, gets hidden under a coat etc.
Do whatever feels right to you. Don't worry about what other people think. It's fine to think in private about thigns. It's fine to decorate your whole house in its honour. It's not fine to judge other people by one very small aspect of their lives.