It's worth noting, you will do some programming regardless - in any engineering course. Further, the nature of the Oxford engineering course allows you to specialise in e.g. electronic, information, or computer engineering. Thus, specialising in the fundamental hardware aspects of computing, or some elements of the communications of computer systems, or implementation of complex computer systems - which you may not be personally writing the code for, but you're coordinating how all these separate machines with different code written by different people work together. From this background it's entirely feasible to move into a software development role, or pursue a masters or higher in computer science and then do so.
If you really like the very abstract stuff then, you're probably going to do a PhD and go into academia anyway, and usually they don't care so much about your degree title as with what you've been doing with it - and if you are specialising in a related area, and doing projects and theses on related topics to these more abstract areas of computer science, within the scope of engineering, that will go a long way to at least being considered. Also, Imperial have a "conversion" CS masters course anyway - although they'd probably suggest you go onto their intermediary suite of specialised CS courses, rather than the base conversion one (or potentially the more research oriented "Advanced CS" masters - since they'll be aware that you're likely to have a better than average preparation for the nature of the course coming from Oxford engineering).
You can take a year out, but honestly I'd suggest beginning the course - if you really feel it's not for you, you can try to transfer after beginning by discussion with your director of studies (and if you had an exemplary first year performance they may well be amenable to that, although of course there is no guarantee). If not, then...you know what to do!