If its a passion of yours and you are well read i do not see a problem.
A level History is in depth and a challenge , i did A level after GCSE.
However i did Ancient History without the corresponding GCSE knowing next to nothing about the period.
Though like yourself History has always been a passion and i have read literally hundreds of books.
So yes, i would say perfectly doable, and i am sure you can do it. Go for it!