It's perfectly normal(ish) to feel like that for the first few weeks. I had similar experiences on amitriptyline/nortriptyline/gabapentin/epilim and a few others. Some people are just more sensitive to side effects than others. I'd feel incredibly out of touch with reality at times and not really understand what's going on around me. Slow in movement, speech was slurring and I couldn't remember much. It's quite freaky so I can relate to you being a little upset.
My only advice is to stop if you feel that you can't carry on. Side effects will usually disappear once you adjust to it, but that will take some time. Do you live on your own? If not, make sure whoever you live with is aware of how you feel so they can keep an eye out for you. But if you really are not comfortable taking it, seek GP advice ASAP.
On a lighter note... it can be quite funny. I remember whilst walking to the shops and I couldn't move my legs when I got half way. I was so embarrassed it was really funny. I was just standing there on the side of the road and I just couldn't move my legs at all! Other times I've felt like I was lifting off of the sofa and a lot of the time I'd just feel a bit stoned, chilled out and just laughing at stuff. Other times I'd be absolutely buzzing. Cheaper than alcohol anyway!