It depends on where you live. It's strange - since the M4 goes right into the centre of London, it's actually quicker to get from central London - Bristol than it is to get from the suburbs - Bristol a lot of the time, surprisingly.
I'd leave three hours.
Traffic congestion in Bristol is notoriously bad, and it only gets worse on the open days, I'd expect.
And yeah, the MegaBus rules!