does it technically matter in the long run- no, not really, it doesn't count toward anything. also, it doesn't matter what your teachers think because (and this is a bit mean but true) you've already got predicted grades and they can't be reversed.
should you be worried- the harsh answer is probably yes. it shows you where you are currently at in terms of if you gave the exams with your current knowledge, that is the grade you will get. people say that it's just mocks and that they'll revise for the real thing, but what i've found is the people who said that for GCSE actually didn't end up doing much for the real thing either, and then severely regretted it.
however, it's over now, so there's no need to stress over it. take the hit and learn from it! are you applying to university? if so, imagine your dream university and decide that with an E you're not getting anywhere.
you have four months! people can go from an E to an B/A in that much time if they work hard enough! so a bit of worry will feel awful, but i think will probably be a good thing to keep you motivated.