Aw OP, I was afraid to drive as well. I only got my license last year because I wanted to buy a house and knew it would save me money to look outside of the downtown core. I hope you have an experienced driving instructor as well as friends. The cars they use to teach in are specially insured to cover things like this and they can explain in a way that even experienced drivers may not be able to. I'd gone out with friends to learn how to drive in the past and nothing was as helpful as my actual paid lessons.
Don't let this discourage you, keep going! I regret not learning how to drive sooner. The independence is amazing.