It's not silly at all. I think that's how a lot of less experienced players are when they first start out. It's something you have to train yourself to worry about less.

I mean, maybe you could try improvizing a song? Start with a scale and just play around with notes? I think that would be a good way to start training that worry out of you.