I'm so sorry, 4 weeks is really so hard :( but it does get better, please believe it.
How does he sleep during the day? Does he tolerate being put down for naps or do you have to hold him? Try to get as much sleep as you can during the day, nothing else is important right now.
At 4 wks he's really so small and has no idea about night and day yet - this is normal. Try to take him out every day and expose him to natural light, whilst keeping the lighting soft in the evening. Don't dim the room for day naps.
Babies are unfortunately born with an immature digestive system meaning they strain a lot to process food, and it takes a few months for their guts to mature.
Probiotic drops really made a difference for my DS, and I definitely recommend trying them. I believe they've helped him digest food better and also helped with reflux.
If you're breastfeeding, check if his latch is OK so he's not swallowing air.
Has he been checked for tongue tie? It can cause a lot of feeding issues leading to digestion problems. It can be corrected with a simple procedure.
Throwing up is usually normal unless the baby's not putting on weight or clearly finds it painful. Reflux medication sometimes has a nasty side effect of causing more gas or constipation so I'd want to be sure that the baby is troubled by reflux and not just wind before getting him on meds.