As above, your department or uni as a whole will usually give guidelines on what counts towards your word count, and what the deductions may be for going over (if any). Your module assessment information should give details on any specific formats required, and otherwise you should follow whatever the departmental formatting guidelines are (which will usually also be published alongside details about word counts and so on). It is worth checking for the formal guidelines (as if they do differ from what you were penalised for, you could appeal on that basis; make sure to check both departmental/university wide guidelines as well as any guidance in the module itself and for the particular piece of assessment submitted though, as the latter could override the former) to know for certain, and also to see if there is any leeway in word count (some of the unis I've studied at allow you up to 10% over the word count, whereas others strictly stop at the word count indicated). Although it was an unfortunate way for you to discover this, at least you know now for future reference.
Everywhere I've studied has included in-text citations in the word count (i.e. Harvard style referencing), though. The only things normally excluded from word counts in my experience are title pages and abstracts, footnotes, tables and figures, bibliographies/reference lists, and appendices (which are not strictly marked or read normally). Everything else counts usually (captions for tables and figures is a bit more variable); even if you type "word count: x" at the start of your assignment, but not as part of the title page, that usually counts as three words in my experience! It's best to assume things count towards your word limit unless it's specifically and unambiguously stated in the assessment criteria that something doesn't count. Formatting as above is really a matter of figuring out the standard format for your department and then following that unless specific guidance is given otherwise (which may sometimes be the case, e.g. if they want you to submit an academic journal style article that follows the style guide of a journal in your field).
I have been lucky however as I've not known anywhere to specifically penalise you for such things (also most of the unis I've studied with have allowed up to 10% over the word limit as well). My own experience has been if you are over the word count, the marker is not obliged to read or mark anything beyond the word limit, and formatting is really more a matter of convention than marking unless it is part of what is being assessed (such as the above example of a journal style article, which was a specific assignment I had once with marks specifically allocated for following the style guide for the journal in question!). But evidently it does vary between unis so it's important to find out what is the case for your uni/department/course/module!