There isn't anything in theory stopping you from doing that. You'll need to be clear on the tax consequences of actually living in a different country, but whilst there are restrictions on witnesses giving sworn evidence from abroad, those same restrictions do not apply to advocates, who can cross examine, make submissions etc from abroad without restrictions. The only other thing that occurs to me when I'm writing this is whether a practising certificate could be issued to a barrister who is not resident in the jurisdiction, but I don't immediately see why not. It'd just be something to check.
However, you may find it to be difficult in practice to live abroad permanently whilst practising as a barrister. You're absolutely right that you can do paperwork from abroad, and you can also attend remote hearings from abroad. But it would be a very rare practice that wouldn't require you to attend hearings or conferences in person on a reasonably regular basis. I say this as someone who actually probably has one of the more remote practices that a barrister can have. I spend a clear majority (probably over 75%) of my working days at home, doing paperwork, remote conferences or remote hearings, because all final hearings in one of the jurisdictions that I practise in most of the time are fully remote. However, those days that I spend in court are still important, because it is either longer trials or work that I'm doing for solicitors that send me my better work. It's very difficult from a practical perspective to build relationships with solicitors, and for them to see you as a reliable option for their work, if you don't do any or many in person hearings for them.
As much as I'm probably in one of the better positions to live abroad and still practise as a barrister, I'm not sure it would be feasible. If it was, I expect it would come at the cost of notable sacrifice for my practice. That may still be palatable for me (and you) from an earnings perspective, but I'm not sure how palatable it would be for Chambers. I have spent a long time building my practice up in Chambers and have an excellent relationship with my clerks, so if I went to them and said I wanted to do this, I expect they would facilitate it. It would probably be more difficult to do when you're first starting out though (say in the first five years or so), when you don't have an established practice in the same way.
If you were going to do this, you could only do it with the backing and support of your clerks, so the first port of call would be to speak with them, even if it is a hypothetical conversation at this stage. If you were to do this there would be a lot of practical considerations to figure out, and you would need the clerks to work through those. You obviously wouldn't have to commit to anything, so there shouldn't really be any harm in having that discussion and seeing where it takes you.
As an alternative, it is worth highlighting that as a barrister you can still take a lot of time off work whilst earning enough money to live comfortable and do a lot of other things like travelling. I have taken the decision in the last couple of years to take a lot of time off for holidays (both actually going on holiday and spending time at home) because I have small children, and I am going to continue to do that over the next few years at least. Despite taking at least 25% of my possible working time off (and probably more), my earnings are still comfortably sufficient for us to maintain our lifestyle and go on several holidays a year. Again, my practice is established enough that I do earn plenty of money for the work I do, even if I'm not strictly full time in a sense, and that again may be a little different when you're just starting out. It is an option to consider though, and whilst I appreciate you have that desire to actually move abroad, it's worth noting that an alternative would be just to take a lot of holidays.