Lone pairs are conjugated into the benzene ring. This explains the reactivity.....
1. As the lone pairs are conjugated they are not as available for nucleophillic attack of an electrophile.
2. Because the lone pairs are conjugated into the ring, they can donate into the ring. This increases the electron density in the benzene ring making it more nucleophillic.
3. The acid catalyst may form a lewis acid adduct with the NH2 group, making it non nucleophillic. This effect IS present with a protic acid, forming a positively charged R-NH3 + group which is no longer nucleophillic.
Also, to correct a statement above.
"well nitrogen is less electronegative than oxygen so there should be a greater electron density around the oxygen and hence oxygen SHOULD react with acyl chlorides / acid anhydrides."
As the oxygen is more electronegative, the energy of the lone pairs are lower than for N. They are therefore les available for nucleophillic attack. This is why Nitrogen nucleophiles are more reactive than Oxygen nucleophiles, and why ammonia is more basic than water