Who has said you can't leave the UK during term time? Is it a condition of your visa, or is it a university policy?
I suspect most universities have a policy that you're expected to be in attendance for the whole of term time -- in particular you're expected to attend all academic commitments: lectures, seminars, practical sessions, etc. There's probably a process by which you can request permission to be absent.
If you need advice regarding your visa, you need to get that from your university's visa team. They should have the necessary knowledge and training to answer your question.