"I think I might come across too full on" you don't say!
Wtaf! You're a NEW gf (not even that really - he is someone you're dating) and he has an ex he's able to deal with in a friendly grown up manner, he's open and honest with you about his dealings with her and she doesn't hate him so even though they've split she still thinks he's a decent guy.
He sounds a catch.
You sound like you need to grow up, work on your self esteem and controlling urges and not be in a relationship until you can.
How old are you? You sound VERY young.
6 weeks is no time at all and the "not putting labels on things"'sounds like he has been clear in NOT making a commitment to you - you cannot make assumptions (either that he is or isn't seeing you exclusively) you certainly have no right to be jealous!
You clearly need someone who is more committed and not leaning toward more "open" arrangements - which there is absolutely nothing wrong with, it's how I tend to be now, as long as everyone involved knows the score and is agreeing willingly that is fine. But it's not how you operate in your romantic life.
He is not the one for you, he still sounds a catch to me but not for you.