First of all, OP, a massive well done for actually finishing your book - it's far rarer than you think. I did a very well-respected MA, and I don't think even a quarter of my cohort actually finished their books - and that's off the back of spending a lot of money on a course.
A few thoughts...
Firstly, you say you've been working on the book for years - was that just writing it, or have you done a lot of editing? Most books aren't ready to be submitted until they've been through multiple edits. If you haven't worked on it much since actually finishing it, I would strongly suggest holding off on submitting to other agents at the moment, and give it at least two full run-throughs with the red pen. Yes, it's more time before submitting, but you'll be giving it the best possible chance.
Even if it's all edited and ready to go, I would now wait until after LBF. Agents will come back to very full inboxes, so it might be worth giving them time to get through their backlog before you submit.
The email you got does sound like one of the standard rejections - 'read with interest' is a classic. But that doesn't mean they hated it. They may have read the covering email and realised there was no way it was for them. They may have read the first page and come to the same realisation. Form responses can mean a huge number of things, right from 'this actually looks quite interesting but I'm too busy to give a personalised response' to 'OMG this is dreadful' and, at this early stage, you have no idea where, on that spectrum, yours fell. If you'd had a completely damning personalised response, that might be a little more worrying, but still only one person's opinion. So put this rejection out of your mind - there's no way of knowing what, if anything, it says about your work.
I am slightly concerned about what you say about the quite niche nature of your book, and about your reluctance to make changes. Niche is difficult to sell. Crossover is fairly difficult to sell. I'm published, and in discussions about my next book which could probably fit into more than one category, and this has been raised as a possible issue, even though the category it probably best sits in best has seen a fair bit of blurring of genre boundaries recently. It might be worth thinking about if you can make some changes that reduce the number of genres you are trying to straddle - two might be workable, while three or four might be tricky.
In terms of willingness to make changes, I think you need to have a very hard think about what you want from the process. If you want to be traditionally published, via an agent, you are going to need to be open to anything from small changes to a complete re-write, using some of the themes/characters, but cutting others. If it's this book that you want to put out there, then, yes, self-publishing might be a better option. Getting an agent is hard, and most of them won't take on something that needs a complete rewrite. What is more likely to happen is that you'll get a response along the lines of 'There's something here, but I couldn't sell it in its current form. Happy to have a look at it if you were minded to rework it by [insert suggestion]'. If it has a very strong USP and the writing is sound, you might just get someone willing to work with you on it, but that would be unusual. You need to work out just how much you love the book in its current form, and how willing you are to sacrifice some of that love for the sake of making it into something that has a chance of being picked up. I have considerable sympathy for that dilemma - I've recently re-read my first unpublished novel, which had a reasonable amount of interest, but which didn't get anywhere, and I've realised how much I still love it - far more so than anything else I've written, including the one that got me published. The writing is fairly bad - I've learned a lot since I wrote it - but I think there's something there. I can see how I could rework it, but I'm not entirely sure I'm willing to do that kind of hatchet job on something I'm so attached to. So it might have to sit in a drawer and just mean something to me and nobody else.
So have a good think about it all - with as much detachment as you can muster - before you take the next steps. When you do decide to submit to more agents, it might be worth getting the Writers and Artists Yearbook and making a list of likely agents. And follow lots of them on twitter - they often talk about their likes and dislikes.
Good luck - you've done the really hard bit. Nothing from here is as hard as writing the bloody thing in the first place!