Absolutely - it's a recommended but not an essential.
It's all to do with where your main arteries and veins travel through your body. These essentially travel close to your spine at the back of your abdominal cavity, slightly to the left or right of your spine depending on which vessel you're interested in.
Lying on your left side reduces the pressure on your main arterial vessels, improving their efficiency. So if you can always sleep on your left side without hip/back ache or moving into other positions during the night then that's gold standard perfection. However, get past 20wks and it becomes an uncomfortable impossibility for most people! Lying on your right side does put a tiny bit of pressure on your vessels, but really does not compress them that much at all, so it's perfectly acceptable as well.
Not lying on your front is because after a certain point, you obviously can't, something kinda gets in the way!
Lying on your back does not affect your baby, but it can affect you. What essentially happen is the weight of your uterus puts pressure on the main arterys and veins which supply your legs which, as I mentioned earlier, travel close to your spinal column at the back of your abdomen. What happens then is like when you get a dead leg from crossing them for too long, but on both sides and your whole leg length. You essentially reduce the blood flow to your legs significantly enough that they can start getting numb, tingly and when you do move from your back you can feel very light headed from the sudden rush of blood again. It can also increase chances of fluid accumulation. Therefore recommended no more than 10 - 15 minutes on your back once you get past 16wks.
So overall - left side gold standard but difficult (impossible!) to always do, right side is still absolutely fine, front is uncomfortable but not an issue until you can't do it anymore and back can have some effects on yourself, but not your baby
Hope that helps!