To be honest, I haven't really thought about the question that much to come up with some really good replies.
Think along the lines of: changes in the regulations - accountancy firms effectively feed off regulation. That's what gives them business - increase in regulation will mean increases in fees, decreases in regulation will usually mean the opposite.
Oligopolies - at the moment the big 4 are so far ahead in turnover/size and quality/breadth of service that no other firm is deemed capable of auditing a FTSE100 company - the government may well interact in order to stop this, think forcing the big 4 to sell off their audit arms/make their audit business seperate to the other parts of the business etc.
The only other thing I can think of is the inherent nature of what auditing is. You effectively assume responsibility for the legitimacy of a companies financial statements. If you make a mistake you can be sued, and nobody is perfect. The problem is made more difficult when somebody wants to hide something from you, and they can often be succesful. Investor make choices based on your audits, and if they lose money because you cock-up, they can sue you for their losses. This can cause big problems - Anderson were forced to close down because of enron, a PwC member firm in Japan recently had to close due to a blunder on the audit of a Japanese company etc.
I think some of the big 4 firms operate a policy where each firm is responsible for the other ie. if somebody in america makes a huge cock up that might also mean that the UK firm is closed down if they don't have the money to cover the damages and legal fees.