my sister is happily married with kids with a husband smaller than her. She actually feels bad for being taller and tries to make herself smaller on photos where they are together haha... but seriously, dont worry too much about height.
Everyone has an aspect of their body than they dont like/are concerned about, and people will generally just notice/talk about it if you are the one mentioning your own worries.