@Juicy you are entitled to your feelings. You don't have to justify them and they don't have to be excused.
That said you should try to understand where they're coming from. Has he given you are reason to doubt him, is your gut telling you that you shouldn't trust him or is this about you?
Figuring out the cause and understanding where these feelings are coming from is where you'll find the answer to your question.
But once you say things out loud you can't take them back so think very carefully about what you say to him.
You sound like you'd be a good mother