This global Microsoft Outage

[SSpratt]

https://www.bbc.co.uk/news/live/cnk4jdwp49et

Any worries? It is chaos out there by the look of the news.

My experience is that I’m not able to work today and had trouble using my debit card this morning. The transaction eventually went through but it’s not showing on my account.

No, not every single person can. But lots are very much able to, but don’t. And if those people do, then in a crisis they might be able to help those who can’t. It’s expected of us to help neighbour’s and friends who will need it. These are clear instrunctions from our Government, to not just help yourself but others too. It’s what our society is built on. Yours seem to be a bit different, but I am quite certain that many more people could do a little bit if they believed it was important.

And yes, quite a few people do resort to laugh about it and talk about hoarding and loo rolls, instead of their own responsibility to society.

It doesn't have to be "too time consuming" to have some kind of backup/failsafe system that will enable people to muddle through for a short time whilst waiting for the network/cloud computers to be restored.

Why immediately shred any paperwork as soon as it's been scanned? Instead of typing straight into a Word document as a file note, why not write by hand on a file and scan/photograph it into the computer?

For things like lists of appointments, what's the harm in the system automatically printing a "day sheet" at close of business the day before "just in case" something bad happens overnight?

Why can't "cloud" based systems be set up to automatically do a data dump to a local PC or hard drive at close of business every day, so at least a local PC could be used the next day if the network/cloud goes down.

These are the kinds of things I've implemented in my 40 years or working as for a decade or two during that time, I was responsible for IT in the firms where I worked! It's just common sense to me to have backups in place. Back then, I was acutely aware of our network servers failing, hence backups to local PCs, but these days it would be cloud storage that is vulnerable. I also ensured that we never had "automatic" updates switched on, on any computers nor servers - when it was time to do updates, we rolled out slowly, checking every step of the way, across all the computers in the firm.

I run a small accountancy practice and have dual redundancy for all our records. When it's done efficiently, it barely takes any time. I could go into work on Monday, not turn on any computers at all, and do a full days' work, just from basic paper files that take next to no time to keep updated and relevant. Yes, it would be slower, but I'd not be letting any clients down. The main time drain would be updating the computers on Tuesday when I switched them on again, but the chances of any of it happening are very low, so I'm not bothered about it. I'm more concerned that it would be as close to "business as usual" as possible on the day of an outage, and I'd worry about catching up and updating afterwards.

There was so much scaremongering. On the radio this morning, it was saying all Microsoft computers were affected. Us in the emergency services were fine! Expected chaos and no issues at all.

You seem to be very sensible, and I agree. It’s something to wish for, and your experience in this is probably more important than we think. I grew up working in my parents store from an early age, before computers and scanners. People with no money could shop and we wrote what they owed in a black notebook. It worked just fine. There was definitely a system. Things have changed.

MS HQ right now...

I used Microsoft hardware and software all day to do my job and wouldn’t have known there was an issue until I saw this thread this afternoon.

I think it's the difference between "all the affected computers ran Microsoft software" and "all Microsoft computers were affected".

Whether that was a misunderstanding at the presenter's end or the listener's end is up for debate.

Which is fine for companies that still have on site IT support.

How unfortunate for all those companies who decided to save money by outsourcing their IT support to offshore companies. (Says someone who has TWICE been made redundant when this has happened.)

How unfortunate for all those companies who decided to save money by outsourcing their IT support to offshore companies. (Says someone who has TWICE been made redundant when this has happened.)

That's not how it works. Companies who don't have onsite presence in their data centre, use remote consoles which can do everything you can do as if you were stood in front of it.

Yes, but it means the person on the remote console can be anywhere in the world. I've logged on to consoles in Australia, Japan, Singapore, Canada, Brazil, USA and a number of European countries. And so it is that you can offshore your sysadmins and just keep a couple of people on site at each DC in case cables need reseating or old hardware needs deracking.

Except manually reboot in safe mode to stop the computer going through the start up process which triggers the installed defective software with the dodgy update to blue screen the computer.

Which is the solution to this particular problem and hence why it's going to be a complete nightmare to rectify for many companies.

Except manually reboot in safe mode to stop the computer going through the start up process which triggers the installed defective software with the dodgy update to blue screen the computer.

I don't know about Windows machines, but Linux servers I've been on, you can interrupt GRUB to do maintenance stuff before the PS gets booted. I suppose laptops etc though.

Well, what exactly do you want people with no issues to do? Why do they have to worry about it because it is affecting other people? If that was the way the world worked no-one would ever be free of worrying about something on behalf of other people.

I'm the same. I even did online banking last night (not in the UK) which went fine, and this morning I hear my bank was one of those affected. I went to the supermarket this morning, where it was business as usual. If I hadn't heard about it on the news I wouldn't have even known there was a problem.

Some people are being rather dramatic!

Except manually reboot in safe mode to stop the computer going through the start up process which triggers the installed defective software with the dodgy update to blue screen the computer.

Which is the solution to this particular problem and hence why it's going to be a complete nightmare to rectify for many companies.

If the remote consoles are set up properly, where the set up would be different depending upon if it is a blade server in a chassis, a stand alone server or a virtual machine, then they can be remotely fixed. If they are not set up properly, then yes, manual intervention will be needed. I suppose it depends on what sort of outsourcing happened, it could be the data centre and operations or server admin etc that was outsourced. Obviously the implications are different for each one. Outsourcing comes with many risks that people often ignore, depending on what industry the IT is for, different protections are put in place.

If you didn't get affected by this then great. You're lucky. But make no mistake - this was a bad one. It deserved the press coverage.

The problem became apparent in NZ and Australia first -they are 9/10 hours ahead of us -and MS then took steps to mitigate the problems -we got off lightly.

This kind of outage is precisely the reason we must fiercely resist the cashless society. “Sorry, you can’t get your shopping, the internet is down, we don’t know when if it will be back up.”

Ditto software updates: I loathe them with a passion. You pick up your laptop to use it, and “installing updates… please be patient…” and your laptop is never the same again,

I found it fascinating how Australia and NZ are going to find themselves on the front line of this as they are closest to the international dateline. This is something they are going to have to get used to.

Apparently the US has been less affected because when the issue started, a lot of PCs were switched off and didn't get the relevant update which would have been rolled out at a scheduled time (probably midnight) in addition to Microsoft having time to fix the issue.

This also would give the UK an extra hour or two compared to much of Europe as well. There's been talk of effectively unifying time across the EU and I think this gives a strong case for the UK and Ireland to resist that.

If the remote consoles are set up properly

The if in that sentence is doing a lot of heavy lifting.

It goes to show how dependent we are on technology and how screwed we are if something goes wrong. Forget a nuclear bomb, or a pandemic this is the real issue

Even if the remote consoles are set up properly, recovery is a very manual procedure. If you've got one or two servers to recover then it's not a huge deal. If you've got hundreds then it's going to take a lot of man-hours.

Same here. I use my card a lot of the time, but always have cash at home.

Yes - why can't more people see this? They sleepwalk into a cashless, digital society and don't realise the extent to which they are surrendering themselves to things outside their control. Not just the risk of global outages, but having your life messed up by 'computer says no' cock-ups, having your choice of entertainment and news dictated by whichever magnates control streaming services.

And the idea of the infallibility of a computer system where nothing can be changed if information is incorrect on it. The whole computer says no problem.

We are worried about AI but actually the underlying nuts and bolts of systems we already have are more problematic. Many are patched up and carry on. Or there's the risk of corruption or mistakes when you transfer data from an old system to a new one.

We are overestimating the ability of system a lot of the time.