Oh, rollercoaster1, you will get LOTS of support re your problems/query!!
Have a glass of wine, it will do you good and him no harm. I was you 5 years ago after DS1 (now looking after 10 month old DS3).
You sound like you a doing a phatastic job! Do not waste precious nervous energy on doubting doing what feels right to you.
At 4 months, your son is tiny and probably just going through a growth spurt, often happens at 16-18 weeks when babies want to feed more often, for longer and generally can be v restless.
You cannot spoil a young baby - I hate that phrase "spoil": what, ruin him for life??! Things will settle again, they will, it is just hard to see that when the going is really rough and you are physically tired and mentally exhausted.
"Routine" - IMO there is nothing wrong with following babies clues, ie recognizing when he is tired, making a mental note of that time, and looking out for tired signs the next day around the same time. GF very emphatically would not have been for me either (I have a friend who was so upset by her inability to follow the routine she ended up burning the book in her garded ), but I have friends who loved that approach. I thing it very much depends how organised a person you are at the best of times (I am not very ), and how much crying you are prepared to listen to - at least in the short run. I did not want to teach my children that when they cried, nobody came.
Sorry, about the long ramble, you just really struck a chord. Repeat after me: it does get easier, you are doing a great job, try to roll with it rather than striving for unattainable goals.
Continue to BF on demand, follow your babies leads, nurture him and make time to look after yourself, too! In the words of people wiser than me: this too shall pass. He will develop a routine of his own.