I know where you are coming from, because I have panic disorder and have had the same problems. If you really can't do it, instead of completely quitting, could you take some time away from uni, and then go back and finish your final year when you are ready? You can recover from panic disorder. I have recovered massively compared to how I used to be, so maybe it would be better to go back when you feel better?
For lectures, I used to sit near the door, which reduced my panic a little. I knew I could leave at any point if it got too much. For exams, I also sit them in a separate room. Have you tried any medication, such as beta-blockers? I can't take them (due to asthma), but my doctor actually prescribed me some diazepam (they're not keen on prescribing it though), which helps massively in the short term for panic. This or beta-blockers can really calm you down and may help you be able to sit your exams. Maybe talk to the doctors about getting some medication that can help in the short term to reduce the panic symptoms. It may be just what you need to get through exams. If you want some longer term medication, you could also ask about that.
That invigilator sounds absolutely appalling and you were right to complain. They were obviously very ignorant to mental health problems, as are some people on this thread! Some people here have no idea how disabling panic disorder can be. Of course the OP already knows it makes sense to stay at uni for the final year, but the panic disorder is incredibly hard to cope with. It's not just a case of her being a bit nervous about finishing her final year.