I wouldn't judge anyone who did.. I guess you're just attracted to the traits that your ex gf's parents had, which they passed onto both of their daughters.. It's not as if you're trying to replace her shortly after with the next best thing to ease the pain. It's been 2 years and you've found yourself now naturally attracted to her. Maybe 'society' might not think it's right, but when you think about it, you're not gonna be telling everybody the story behind it. The only people who you'd eventually tell would be your friends and they could see that you care for each other and are happy, so I don't think they'd care.
No one can really put themselves in that situation and try to imagine it, it's the kinda thing that happens over time. You can't just say "I have a boyfriend but I wonder if I'd be able to go out with his brother", that doesn't really answer the question. If it would make you both happy, go for it and don't worry what people say. It's not exactly unethical.