Unless you've reason to think you'll have a small bump and/or not be out in the cold much, yes it is worth it. Realistically you are likely to be too large for your normal coat in December, January and February. The coldest months. Mine was the best £40 I've ever spent. Or £13 really, since I flogged it afterwards.
However, my opinion is influenced by:
a) living in the north
b) having been on a subsequent pregnancy and fucking massive when I had my winter baby. First one, I managed to button my pre-preg coat until 25 weeks, which is when it became too warm to need it. Second one, that fucker was done by about 18.
c) walking a lot. I can see that if you primarily drive, or you live next to a tube station and get everywhere that way, you could manage. I don't.
d) taking a while to shrink back down after my winter baby. Sounds like this is your first, so you'll probably be quicker, but regardless you should assume a month or two. People do leave the hospital in pre pregnancy skinnies, but not something you can plan to do.
I would also see if you can borrow one from anyone. It's the sort of thing people often have hanging around.