Only a handful of universities require 4 AS, but if you don't do as well as expected in one subject's exam then you will be in a very tricky situation next year.
I would personally reconsider how many hours you're working. School should always be top priority, and if your ability at school is affected by work then you really need to think about your working hours.
And many people struggle at the start of AS with the workload. How about sticking with it for a bit longer?