I've spent two years in Australia on a Working Holiday Visa, so if there's any specific questions or advice you need then let me know.
I've not been to New Zealand but from what I understand, their requirements on WHV holders are certainly more lax than Australia's. However, I'm not sure how easy it is to find work there in comparison with Australia.
I know of one friend who went to NZ after he was finished in Australia and still has an admin type job in Christchurch. I can get in touch with him on Facebook with any questions you might have.
Regarding the tax in Australia, you should still be able to reclaim some of it assuming you're going to travel in between jobs - I was able to for each of the two years. On top of that there's superannuation (your pension fund which you earn as you work in Australia) which you'll be able to claim once you leave the country (make sure you have a departure stamp on your passport).