If you're scared about squatting, two things will probably be key - technique and experience. Experience is pretty obvious - the more you squat, the more accurate your understanding of what you're able to handle. Technique is another big one because if you haven't got it right, you'll be less consistent and you'll wobble around more. If you have good technique, even when you're under a bar that's near the limit of what you can do, you should be able to drive it up and remain steady. If you don't, then when you're doing high effort work you'll be prone to little wobbles that throw you off, and if you feel that way that will obviously knock your confidence.
My advice would probably be to really drop your squat weight down for now (and keep doing your heavier exercise with the assistance lifts you've been doing) so you can work on technique. Watch some videos of good squats and try to drill the cues into your mind while you're at a weight low enough that you can focus on the technique rather than on whether you can complete the lift. Maybe in the end you'll still decide you don't want to squat, but it's that useful a lift that I wouldn't drop it because you lack confidence.