I don't know if there's any reason to expect an infinite amount of time, but if there is then I suppose what you say sounds about right.
Also if you had a machine that could go back any amount of time, you would need something that could somehow transport you through any amount of space too. Otherwise, if you stepped into your time machine today and tried to go back even a short time, you'd just emerge in what at that time would be some lonely place in space, just waiting for the earth to catch up as it travels through the galaxy. I definitely don't think it's possible in the hollywood sense.