If you plan ahead you can save a lot of money by using the university library and, if you are a postgraduate student (for future reference), you can borrow from other universities too. I always tend to use public libraries too. For example, the Bristol city library is fairly good, but there is an entire network of libraries in the south west that I can borrow from as well; if I order online then I get charged 50p to reserve the book, but that is far cheaper than actually buying the book, or using the inter-library loan service at most universities (and much quicker too). This book might come from a huge distance so 50p is nothing really.
Public libraries often have "books they should not have"; you will be surprised just how many academic texts there are in a library's reserve stock etc.. For example, I am borrowing an expensive book on Dostoevsky from a library in Somerset, again, for only 50p. And because it is a public book, and obviously one that someone is unlikely to want, I can renew it continuously until I find time to actually read it. In a lot of universities you cannot do this as you have limited borrowing rights, and your loan length is often cut short by other users wanting to read the same text.
Another issue no one has mentioned is note-taking. If you buy a text then you can make notes in the text. If you borrow a text from a library you can make notes in the text and write them down separately; for instance, you can highlight relevant paragraphs in pencil and have page numbers so if you do decide to write on a particular text you have already done half the work.
The Kindle would be nice for a casual reader but superfluous in most cases for a literature student in my opinion.