You genuinely do not have to worry. So long as you're healthy, that's all that matters.
I'm about 5ft 7, clothes size vary between 8 and 12 (depending on shop) and I'm about 9st 9.
Everyone (myself included) worries about their weight, some more than others. But if you're getting complimented and you're a healthy weight, then I wouldn't worry too much.