I think the best way to tell if you are very worried might be by a visit to your GP. I'm no expert and I suppose most people here on this forum aren't. My guess is that the BMI doesn't give you a very good picture of what is going on, though it is a rough guide. Overweight people could be damaging their health in the long run. What you want to know is whether you are healthy or not, and some numbers can't really tell you that unless you end up right off the scale. This isn't the case though, but if you are worried and don't want to take any risks, you could ask your GP.
Don't just let him weigh you and take your height and look it up on his own little BMI chart (that's what the doctors do every time they're checking up on whether I'm still a healthy weight or not!).
Make a list of what you'd generally eat in a day or week (and be brutally honest) and how much exercise you do (again, be honest) and make a note of anything else in your lifestyle that might be relevant to your health (smoking, alcohol etc). Then ask the doctor what he predicts for your health if you contiune your lifestyle and stay that weight for decades. Would it be harmful, or can he give you the all-clear? Cause that's what's really important, your health, not some silly number on a chart.