Your plans sound fine, really nice in fact 
. A midweek wedding would not be to everyone's tastes but so long as you give people sufficient notice and it's a small wedding anyway, it should be fine IMO and your reasons for it are very sensible. I don't think there are any really hard and fast rules for what is 'expected' at a wedding these days, yes lots of people still do the £££££ big white wedding in a castle with 3 course sit down breakfast and disco after, but lots of people do very informal/small gatherings at registry office and afternoon tea/lunch/BBQ/some other informal meal after, and every shade in between, so don't feel obliged to spend more than you are comfortable with to meet some imaginary 'must haves'! On the other hand (unpopular MN opinion alert!) I do think it's nice to make a bit of a fuss and effort for your big day, after all you only plan on doing it once and so it's fine to indulge yourself a bit on frivolities so long as you genuinely are doing it to give yourselves or your guests joy rather than because someone else thinks you 'ought' to...
3pm totally normal ceremony time, getting to be more common than the previously more fashionable mid morning start time in my experience, and it's generally easier to manage, means any guests who are travelling don't have such an early start and everyone has time for a proper lunch which is nice, you are then not drinking the canape reception drinks on an empty stomach and aren't absolutely ravenous by dinner, both of which are a good thing for happy/harmonious guests!
Buffet dinner again great, lots of weddings I've been to have done something that isn't a trad sit down formal meal (everything from hog roast to BBQ to mediterrean style sharing platters of meats, cheeses, antipasti to more 'English' finger buffet of sandwiches, pies, cold meats etc). So long as there is enough food (nothing is worse than stinginess/being left hungry!) and dietary preferences/needs are taken into account (including at least some food suitable for children if any attending), then a buffet is a great no-fuss option. For personal preference I would like there to be at least some hot items particularly if it's in autumn/winter and I'd like to be able to sit down with my food rather than have to balance it and a drink in my hands whilst desperately trying not to spill down my fancy wedding guest attire but neither are absolutely essential!
Have you thought about things like:
-Will you have a best man/bridesmaids/page boys - completely non compulsory of course but can be a nice way to involve children in the wedding.
-Transport to/from the venues, for yourself and guests. Can be a nice touch to arrange taxis or a minibus if you need to move between venues that aren't walking distance away from each other, particularly if you have any elderly guests, although of course some people may prefer to come in their own cars anyway.
- Entertainment/music at the reception - I know you have said small/casual so maybe you don't want anything at all which would be fine, but perhaps you could put together a playlist of your favourite songs together and get pub to play it over speakers? People may want to dance even in a small space so it's nice to have small dancefloor area if you have a 'dancey' family/friends!
-Speeches/toasts, do you want any? Not compulsory at all of course and some people loathe them and/or they can become unhelpful family battleground so understandable if you want to skip them - just make sure you let anyone who might be expecting to make a speech know in that case - you don't want a surprise speech . But all other things being equal, I think so long as they are kept reasonably short and sweet (no-one wants to sit through hour long renditions of the groom's sexual history or all the bride's educational and career achievements since the age of 5!), they are a lovely addition to even a small wedding. It doesn't have to be the 'traditional' thing of just the men speaking, at recent weddings I've been to there have been lovely speeches from the bridesmaids, sister of the groom, step-mother of the bride and bride herself alongside the father of the bride, best man etc. - not all in one wedding I hasten to add!).
-Flowers/favours/venue decoration/table settings/photographer/cake/orders of service/fancy invitations - can all be completely disregarded if not important to you but worth thinking now if you do want these things (and it's fine if you do!) so you can seek out good deals or DIY some of it or perhaps talented friends/family could provide as a wedding gift?
Good luck!