You've had the benefit of two really good replies so far. It's difficult to know exactly where the problem lies here, but it's likely to be multi faceted. It's certainly too early to say that the career is not for you. I'm about 15 years practice, but I recognise a lot of the issues you've described, both in my own practice and in the practices of the barristers that I work closely with.
The bad news is that some of this is the reality of the job. You will always receive cases that you dread. I am generally pretty confident in my own ability, but even I get instructions now and again where I just don't want to open the Word document to start drafting. That happens to everyone, albeit probably not in the same way and to the same degree that you're describing, because your situation does seem to be compounded by other issues. It is certainly the case that some of this is likely unavoidable for someone of your level of call. I'll get on to the work/life balance aspect in a minute, but the reality is that as a new tenant you are the one who gets what's left over, who is sent to do the work that no one else wants to do, and who will often need to be called upon at the last minute. You'll perceive some of that as clerks not having respect for your work/life balance, but a lot of it may well just be that you are the one that this sort of work will fall to. Whilst that doesn't help right now, it does mean it is temporary. I will admit that even when you become more senior, the Bar does still have work/life balance issues. The severity of those depends on a range of things, but they do still happen.
The good news is that much of this can or will get better in time. Being the person that all the rubbish work falls to is tough, but it is a process that allows you to strengthen your own skills, build relationships with clerks and others, and learn things about yourself in terms of how you'd like things to operate when you become senior enough to have more control over it. I'm not saying that as a throwaway consolation prize. I do genuinely think that there is some value in that, albeit I don't think you're in a position to acknowledge that right now. But the general process of becoming more comfortable with the work, building relationships with and therefore having more sway with the clerks (and in turn them understanding your needs better), and becoming more able and confident in managing your own practice so that you can strike a better balance (which very much includes thinking about the work when you're not doing it), are all things that come in time.
Having a good work life balance is very important to me, but it has taken me every second of those 15 years to get to a point where I'm broadly happy with it, and even then, there are times when the work just crushes me. Case in point. I've spent a lot of time off work in the last six weeks. I have done work, but I've taken time out, both for holidays and so I'm not busy when family is visiting. And I have largely not been thinking about work when I'm not doing it, but that's a skill that takes time to hone. The end result of that is that I know I have a lot of work on, but I actually don't know exactly how much. I sat down at my desk about ten minutes ago to figure that out, and that's what I'm going to do after I submit this post. All I know is that even though I've proactively booked two papers days over the next two weeks, these next two weeks are going to be pretty horrendous, because all of the paperwork I've pushed back and conferences I've said I couldn't do in the last two or three weeks are happening in the next two weeks. But I know that, my wife knows that, and we'll get through it. Is that really a good situation to be in? I think it is. This is a high pressure job that comes with high pressure periods, but it's one that I'm well remunerated for. I've spent the majority of the last six weeks spending quality time with my wife, kids and extended family, and that has been very valuable. The trade off is that for the next two weeks I'm going to have to effectively play catch up, but that really is ok.
I realise that the experience of a barrister of 15 years practice may not seem directly relevant to you right now, but I have explained all of that it give you some hope that this can get and should get better. The exact path to that and what looks like depends on a lot of variables, including who you are, what your practice is like, and what your clerks are like, but either way, it will get better.
One thing I will say though, is that there is always a chance that aspects of your practice or you Chambers aren't fitting with you. There is always the possibility that a change in your practice, or a change of Chambers is what you need to get more satisfaction from this job. The one thing I disagree with Blayze about is how easy is it to move sets. It's much easier than most people think, and easier that he suggests. It's actually not difficult at all to move set at your level of call. You're ultimately a new tenant that chambers doesn't need to put any sort of resources into to train, and you can hit the ground running with work straightaway, but you won't come with established expectations of what your practice will look like. So you're actually a pretty attractive proposition for other sets. However, I wouldn't rush into either changing your practice or moving sets. Keep them as options, but for now I would just see how things develop in the short term. It may be that they improve quicker than you think. If they don't, I would talk either to another member of chambers that you feel you can confide in, including potentially your pupil master but also possibly other junior members of chambers, and/or speak to your senior clerk. I know both of those things are difficult when you're so junior, but I hope that your chambers has at least made you feel like you can do that and be listened to rather than judged. Not all sets do that, but I like to think most do, and hopefully yours has.
Alternatively, or in addition, Blayze has offered for you to PM him, and I'm very happy for you to do the same, or I can make time for a phone call if you want (yes, even during the upcoming horrendous two weeks). In my time on this site I've spoken to plenty of barristers in positions similar to yours who have wanted advice on dealing with issues with their practice. I can't always offer all of the answers, but I can give you the benefit of my experience, and I can be someone who you can talk to in confidence who does understand these issues (unlike most of your friends and family) without the risk of me telling people that you work with (unlike other barristers in your set). So if you do want to talk, send me a PM with your mobile number and we can.
EDIT: One more thing. I really like the therapy recommendation. It's something that so many people have done but no one talks about. I'm not even talking about it as treatment for your anxiety and depression. Even aside from that it's pretty valuable and can really help you to work through issues like this. I would keep that on the table as an option.