38.1 is a bit of a fever yes, but not terribly high, please don't panic. It's so hard to judge when it's your first, I remember it well.
A bit of a temp after jabs (esp men b) is normal and not anything to worry about. Just keep an eye on him and as pps have said, give him some infant paracetamol and regular feeds. It's not recommended any more to strip them right off or put them in a cool bath or anything as it can stop them regulating their temperature effectively, but you might want to reduce the number of layers he's wearing and the amount of bedding he's sleeping under.
To give a bit of context, doctors often won't really consider it a fever unless it's 38.5 or over, and even then won't give any advice other than the above unless there are any other concerning features.
Some children just get warmer than others (I have one who only goes above 37 when really poorly and one who hits 39 with the merest hint of a sniffle) so once he's better, you may want to take his temp regularly for a bit to establish what is 'normal' for him.
In terms of when to seek medical advice, I'd say as a guide do so if you are worried (for your benefit usually rather than his!) no doctor will mind checking such a young baby, if he is presenting any other concerning symptoms eg rash, abnormal moaning/inconsolable crying/high pitched wailing, if he is not alert when awake or is excessively sleepy/difficult to rouse from sleep in circumstances when he would normally be awake. For such a young baby I would also call 111 for advice of his temp goes above 39 (even if none of the above symptoms applies), and I'd always seek advice for a child with a temp hitting 40+.
Hope that helps as a general (non medical but fellow mum) guide.