I would also add in some budget for business cards and a logo (c£100-200, logo could be part of the website brief). And money to attend networking events depending on your audience. If local networking type businesses I'd say £25 a go.
But if you are in a niche area there might be must-go type conferences/events that will be good to attend. Once you start paying for these things yourself (as opposed to as an employee) you realise how pricey they can be!
Ideally you want an email package to send out messages (I assume) www.mailchimp.com is free and does a nice line in standard templates.
If you can work from home you'll have no premises out goings but will need stationery etc.