I think as others have said YANBU but then neither is your fiance BU in wanting some kind of celebration or marking of the event, obviously legally you are just as much married if you go to a registry office in jeans with 2 witnesses pulled off the street as if you have the full shebang complete with 200 guests in a castle, huge meringue dress, speeches, favours, 6 figure budget etc etc so none of those things should be compulsory. What you need to do is give some thought to what the aspects of "a wedding" are which fill you with dread and on the other hand what your fiance feels he actually wants from "a wedding", as just saying he wants one and you don't really doesn't help you communicate and agree. Hopefully that will eliminate a lot of your concerns and help you identify where you can both make compromises. Unpopular opinion coming up but I do think also it's right to consider your family and close friends in this, it's natural and normal to want to celebrate a major life event of the people you love with them, and also to look forward to weddings as a chance to get together on a happy occasion with people you otherwise don't see frequently, before I'm jumped on of course this shouldn't come above the feelings of the bride and/or groom if this will cause genuine distress or unhappiness, nor should this mean you feel you have to spend more than you can afford or waste lots of money on fripperies like favours or fancy outfits just to keep other people happy, but all other things being equal I do think it is a good thing to do to offer your loved ones the chance to celebrate with you in some form or other.
Some examples of really lovely non-wedding-y, non traditional weddings I've attended recently-ish to give you an idea of how it could work for you? Not that any of these are how you SHOULD do things, just how it COULD work? One was where the bride and groom had a short private civil ceremony at a local hotel with just their parents in attendance in the afternoon, then invited a wider circle including some other family members and very close friends (about 20 people I think) to a nice local restaurant for dinner -the bride did wear a wedding dress albeit a low key one but it wouldn't have been odd at all if she'd worn a normal dress, there were no speeches or readings although the bride's father (very briefly!) proposed a toast to the happy couple, no 'top table' as such as people were sat informally on benches, no real 'centre of attention' on the couple although of course everyone was delighted for them and congratulated them in a low key way etc. As it happens the groom's family stealthily paid for everyone's meals which was a nice surprise as the guests were fully expecting to pay for themselves so it needn't have cost the couple anything at all...
Another was where for religious reasons the ceremony happened separately (I think some religious family members attended but not many), then there was a reception/gathering in the bride's parents back garden with a marquee in case of bad weather, buffet food, champagne and mingling for again probably 30 - 50 guests, people dropped in and out throughout the day so there weren't that many at any one time, bride and groom wore traditional costume for this one but again could have worn anything without it seeming odd. No speeches, there was some music and dancing but again that was a cultural thing, there didn't need to be to make the day run IFSWIM. I expect this one actually cost quite a bit but that's because bride's mum went a bit mad providing very nice champagne and lots and lots of traditional food, I'm sure something similar could be done very cheaply with a BBQ or afternoon tea style catering and BYOB and still be lovely?
Another one there were some family conflict issues, bride and groom legally married privately at a register office, but invited friends only to a bar in the evening for a party, everyone dressed very casually and they did a fun 'home made' unofficial ceremony, 'officiated' by their best friends and with 'speeches' (very casual!) from anyone who wanted to make one, games, quizzes etc. This was very fun and personal, felt more like a hen/stag night than a 'wedding', although this was very much all eyes on the couple as they are very extroverted people, again something similar could 100% be done in a more low key way with the same sense of fun and informality...