I'm loathe to post as a professional but as an SLT I'd say have him referred. All SLT departments offer an open referral system so you can actually ring them up and request an assessment yourself - cut out the middle man.
Waiting times for initial will vary between areas (wrong, I know) could be as little as three weeks, might be eighteen months. In the meantime, have his hearing checked - probably a shorter waiting list, on average.
If he needs help with speech sounds, I wouldn't work with him until he's at least 4. There are reasons for this which are too long to go into here, but he probably wouldn't be ready, is the gist. There's no guarantee he will be ready at 4 either, the SLT will judge when the time is right.
The speech sounds you describe aren't too worrying, but I'd want him checked by a trained ear, if you don't feel offended by the suggestion that your ear isn't as well trained! In my experience, parents pick up some of the difficulties but not all of them. Also, it's worth noticing whether the sounds you feel concerned about are developing at other "word positions" so for example can a child say a sound if it's at the end of a word but not the beginning. If so, it's a good sign that the sound is at least developing somewhere in the child's system.
Of course the SLT will look at language as well to rule that out. Sometimes children with delayed speech also have delayed language. In most cases, they don't. Try not to worry, just have him checked!