english lit NEA AQA a level coursework

is there a penalty for going below or over the 2,500 advised word count?

There is no penalty, no. However, it can be self-defeating to be too far under it or too far over. Try to stick to within 10% of that target.
From AQA's Report on the Examination (June 2023):
"Word Counts
Though there is no automatic penalty for exceeding the word count, we expect responses to be within the 2,500 upper word limit, excluding quotations. Responses do not gain an advantage from exceeding this total, and such excess is often self-penalising as overlong work often loses coherence, can be repetitious and often strays into irrelevance. It is expected that every piece of work will be accompanied by an accurate word count shown both on the CRF and at the end of the student’s work. A useful practice has emerged over the last few years: one word count including quotations, and another indicating the total once quotations had been deducted. Moderators do check word counts and centres should stress this fact to their students."

I was 1,850 ish words in total but then my teacher told me that I have to be 1,500 words and told me to cut it down. But she didn’t exclude the 4 critics and the quotes that I used (which i used quite a good amount) and now as I am 1,500 total including the critics and quotes so that means without the quotes and critics my word count might be quite less, should I be worried?

Ah, that's interesting. AQA actually offer two A levels in English Literature, which have different requirements for the NEA.
English Literature A (7712)
"Comparative critical study of two texts, at least one of which must have been written pre-1900.
One extended essay (2500 words) and a bibliography."
English Literature B (7717)
Study of two texts: one poetry and one prose text, informed by study of the Critical Anthology.
Two essays of 1250–1500 words, each responding to a different text and linking to a different
aspect of the Critical anthology.
One essay can be re-creative. The re-creative piece will be accompanied by a commentary.
You originally referred to "the 2,500 advised word count" - where did you get that from? Could you have read that from the specification for English Literature A even though you're doing English Literature B?

I’m doing English literature B, I got the “2,500 advised word count” from my teacher. Also I didn’t know it could be 1250 words as my teacher stressed on it being 1,500 words.

Well, if you're doing English Literature B then the requirement is as stated above. Note that it's a range with 1500 being the upper limit.
You can also read it for yourself in the specification. Despite the fact that you say you got the 2,500 word count from your teacher, it's clear that they know the requirement is for 1250-1500 words, which is why she "told me that I have to be 1,500 words and told me to cut it down".
If you don't want to wade through the whole specification:

I think she probably said 3,000 in total I probably just got the 2,500 from somewhere else , thinking about it 😅😅