The government can be based anywhere, and there are strong arguments for it not be based in the largest, wealthiest city. Especially when said city is both geographically and culturally a long way from much of the country. Somewhere more central is a perfectly sensible. It's also perfectly sensible to have governmental contingents focussed on different areas of the countries. (Sort of like mayors, I suppose...)
However, I don't think there's any suggestion of moving the whole government, just a few people. Not sure we know which people. Just Burnham's favourites, presumably. Like his own glittering little court, fit for the 'king of the north'. It's unclear as to what the benefit of this would be, at least to anyone other than him.
If there had been a full consultation to investigate which departments should be moved to where, which took into account all sorts of considerations from transport links to office prices, that would be one thing. There would be a detailed report of the rationale, and the cost vs benefits. This is not that. As far as I know, there has been announcement as to why Manchester rather than than say Liverpool, or Nottingham, or Sheffield etc. Just that he's from there, and doesn't really want to bother moving.