Meet the Other Phone. Only the apps you allow.

Meet the Other Phone.
Only the apps you allow.

Buy now

Please or to access all these features

More about the Technical side of the attacks on Mumsnet

77 replies

JustineMumsnet · 19/08/2015 11:17

Hi all,
There are have been, understandably, a lot of questions about the tech side of the attack on Mumsnet, so here - courtesy of the tech team is some more detail. We obviously do have to be a bit careful with the details because we don't want to give away information that could help other hackers. Whilst it's true that "security through obscurity" isn't real security, we have no wish to make it easier for a future attacker.

We've spent a lot of time since the attacks began, proactively defending against them, minimising the impact of it and protecting against future attacks. With a busy site like Mumsnet there is a lot of information to go through. When we uncover a new snippet of information, perhaps a new suspicious user account, we have to go back to the start and reanalyze, so it can be slow going at times. We are working with our technology partners who have a lot of experience of these kind of attacks and we have used lots of resources available to us.

Some aspects of our technology stack have already been extensively tested by external specialists. Some of our software code is quite old - nearly as old as Mumsnet itself - and things have moved on a lot over that time. However, we have a program of code review whereby all new code is checked by someone other than the person who created it. It's not perfect and everyone makes mistakes, but we take the quality of our code very seriously.

The Denial Of Service (DOS) attack against Mumsnet was a heavy, sustained attack which initially overwhelmed our ability to respond to legitimate requests. Mumsnet might typically get something like 50-100 requests per second. During the attack we were getting around 17,000 requests per second. Each request carried more data than is normal as well.

The hacking attack on our website was separate from the DOS, though we believe perpetrated by the same person or people. We follow many of the industry's best practices, such as using HTTPS for our login pages, keeping our database separate from our cluster of web servers and not accessible from the internet, and so on. We don't necessarily use the same standards of security as say your online banking service might use, for example requiring multiple passwords or using two factor authentication. We try to balance security against usability and the sensitivity of the information we hold. After all, as pointed out by one of you in an earlier thread, the majority of information we have about a user is what that user publishes in Talk, which is there for all to see.

As has been mentioned several times, we keep our passwords encrypted and we use the recommended algorithms for this, with high "strength" settings. This means that if someone somehow obtained the password data from our database they wouldn't be able to make any use of them - they wouldn't work on our site or on any other site even if the user used the same password on that other site. This remains the case even for MNHQ staff; they cannot un-encrypt the passwords either.

We are now pretty confident it was a phishing attack. Phishing, where a hacker gets a user to enter their username and password into a form from which they can capture that information, fits all the data we have. The hacker doesn't need to decrypt anything, because they capture the password in the browser as it is entered (either by typing it, or if it was automatically remembered by the user's browser or password manager). The list of passwords that has been published includes some that users have identified as being ones that they've mistyped. Our database wouldn't have mistyped ones, only accurate ones, whereas those collected by recording what a user submits would and does contain errors.

It's not obvious how it has been conducted though. We have been able to create a proof of concept which shows that it could work, but that relies on some steps that would be difficult or virtually impossible for a hacker. Phishing attacks sometimes use social engineering to "trick" people into using the fake website rather than the real one, but again, for various reasons, we can rule some of these out. Other phishing attacks are more technical and use other means to get people to visit the fake page. One such example is Cross Site Scripting (XSS). XSS is ranked number three on Open Web Application Security Project top ten list of web site security problems. If the hacker can get the website to put his own code on pages which are to be viewed by other users, s/he can modify the page to either redirect the login process to their own site, to a page which looks just like our login page but is actually recording the details and sending them to the hacker. Also possible, but even less likely, is modifying our login page to submit the details to the hacker as well as to us. If the hacker had gained access to our Content Management System he could have done the former, though not the later. However, we record all changes that are made and there are no suspicious ones.

It's impossible for us to know how many users' passwords have been collected. It's a reasonable assumption, and our working one, that the passwords of everybody that has logged since 6th August 2015, and possibly some time before that, have been collected.

In light of the attacks, we've bolstered some aspects of our security, particularly around our administrative functions. We have further changes planned and will be working on these in the coming days.

Forcing everyone to reset their password, as we have done, would render the list useless provided that users don't choose the same new password and they've not used the same username and password elsewhere.

Some users have questioned why certain other changes aren't being made already, such as a move to enforcing stricter passwords, which makes sense. However, given how crucial the part of our system that deals with passwords is, we have to be really cautious when making changes to it so we don't want to rush and end up creating bigger holes but we will certainly take steps to encourage users to strengthen passwords as soon as practicable.

Any questions do post here - we'll answer as transparently as we can - bearing in mind the caveat about helping future hackers mentioned earlier.

DavidTech · 19/08/2015 13:31

@Fiderer

I'm on the list also and didn't have to type in my password in the last 10 days. The page said Log In but the info was already there.

I just got Fiderer / password ........ and then clicked log in.

Is that also phishing?

Not typing the password doesn't mean it wasn't phished. Your browser still processes the password, even if it was remembered by it or by a password manager plug-in, and thus it could still have been captured.

DavidTech · 19/08/2015 13:41

@PaulMoore

The tech team have made some curiously bizarre decisions/comments.

Passwords & encryption don't mix. They're either using the incorrect term or incorrect method, neither reflect well on their ability to protect users. There is no single "recommended algorithm"; each case is different. It doesn't mean a great deal unless you know where these recommendations came from. A Google search, for example, is festooned with recommendations like "use SHA256", "MD5" and "Base64"; none of which are appropriate.

Likewise your use of HTTPS/SSL/TLS. Sadly, the current implementation is virtually useless. It is not sufficient to "secure" login/registration pages, as the cookies which maintain the session are subsequently sent over an insecure protocol.

Had the team implemented CSP, it's highly likely that any XSS/rogue dependencies would have been identified before any breach had taken effect. As it happens, there's no CSP whatsoever.

Can they clarify exactly how passwords are stored? Revealing the storage algorithm won't adversely affect security at all... but it will help put minds at rest.

Thanks.

We use a bcrypt hash with a modular crypt, salt of course, and a high cost to minimize the likelihood of rainbow table attacks. If you're aware of a more suitable mechanism for doing this please let us know by emailing [email protected]

It's true that the session data is passed in HTTP, but nothing about this attack whatsoever suggests that it involved compromised session data. We are planning to move to all HTTPS, all the time, very soon.

We do not implement a CSP that's true, but we are expected to support a very wide range of browsers, including old ones, and they don't support CSP as far as I know.

DavidTech · 19/08/2015 14:00

@PaulMoore

bCrypt is more than adequate, assuming it's implemented properly.

CSP has pretty broad coverage now, but that doesn't preclude you from using it... as UAs which don't understand CSP headers will simply ignore them. Likewise HSTS.

We haven't implemented it ourselves, we've used one of the standard libraries.

IE9 doesn't support CSP as far as I know and we have a large number of users on that. However, it's something we'll be looking at implementing in the near future I am sure.

DavidTech · 19/08/2015 14:05

@RepeatAdNauseum

I don't believe this was a phishing attempt.

I stay logged in on my desktop, and the tab is open, at all times. The tab shows as secure. There are no alternate MN tabs showing in my browser history.

I work on a highly locked down system. I'm an Admin (hence being able to get onto MN!) but have corporate level cyber security. I work for a company that would lose billions if the systems were hacked. It alerts me to any possible phishing attacks, insecure updates, dodgy connections. Nothing.

I don't click links in emails. Infact, I haven't logged into my MN email in months and months. I did check that I still have access to it this morning. It's fine.

The username that they have is very old, and I wouldn't have logged in with it at all recently.

There are other possibilities for phishing. We know it wasn't an email based attack. If you are willing, would you please email to [email protected] or PM with the username that is on the list which you've not logged in with recently, together with your best recollection as to when you would have used it. No problem if you're not willing to do so, but it might help add to the overall picture.

JustineMumsnet · 19/08/2015 15:05

@RoosterCogburn

pirate I asked that question right at the start of the thread (post 5) and it hasn't been answered.

If many of MNHQ including tech's log ins are on the list were they phished ro hacked?

MNHQ passwords aren't stored in plain text but encrypted - no one, not even admins can see them - so a hack wouldn't reveal passwords.

JustineMumsnet · 19/08/2015 15:11

@RoosterCogburn

Thanks Justine. So in effect they also fell for the phishing page. LisaMumsnet really needs a lesson in password setting

Yes this is a good point - we do have regular security sessions but we will be enforcing stronger passwords both for users and mnhqers - that update coming soon.

JustineMumsnet · 19/08/2015 15:27

[quote ChristineDePisan]@Justine - is there anything really useful that MNers can do right now to help you and Tech?[/quote]

Thanks Christine for the question.
First, anyone who saw their password on the list and it was an old one that they hadn't used recently should get in touch with us via email please. The best email address for this is [email protected]

We're inundated with mails to [email protected] - please only send if you really need to and bear with us - we're not going to get back to you as quickly as usual but we will get to every query in time.

Don't panic! - we are doing a FAQs page now and will keep it updated and publicise it widely, so do check that for latest info.

JustineMumsnet · 19/08/2015 15:56

@Roussette

Some- where someone posted about good Password Keepers - I thought it was a techy bod from MN and on this thread but I've searched this and every other hacker type thread and can't find it... can anyone remember?

We list some recommended ones here

JustineMumsnet · 19/08/2015 16:02

@starsandunicorns

How do i go about deregging I read the thread yesterday afternoon and did password reset with different password via the email link the site sent me but not sure if this was enough i didnt get a another email from mn ref password change being dislexic lots of these posts with tech speak are going over my head ( i have tried honest)

i suffer from anixtey anyway and this is not doing me anygood i rather dereg than start again sometime next week or so with a differnet email etc

Go to the bottom of your account page - there's a dereg link there. I'm hugely sorry for for anxiety caused. Hopefully we'll see you back when things are a bit calmer.

JustineMumsnet · 20/08/2015 10:16

Morning all,

Here's an update of where we are at. The tech team found the hole which was accessed to capture user login data via phishing and patched it yesterday pm. Then, as you probably know, we forced another password update requiring higher-security passwords last night (once we'd rebutted a further DDoS attack).

We are undergoing full security testing by external experts over the next few days to determine if there are any other weaknesses which might be exploited. We'll update you when that process is completed.

Many thanks for your patience and understanding. The best advice remains to update your password here and any passwords used on other sites that are the same as ones you've used on Mumsnet before yesterday.

We're really sorry for the extra bother any anxiety caused.

JustineMumsnet · 20/08/2015 10:56

@wannaBe

justine is there any way of getting to reports of dubious posters quickly? There is currently a fairly dubious one on the 3rd hacker thread who has essentially already been called as a troll/jeffrey, although she has put her ahem unique typing down to the fact she is dialectic and has leaning difficulties... IMO posters are on to her, but the fact she is essentially claiming that no-one is safe from hacking of all their accounts isn't helping the current of unease atm.

thanks - will as community team to look urgently

JustineMumsnet · 20/08/2015 11:00

@SnakeyMcBadass

I haven't been forced to log out. I'm still here under the username I've had published and the password I changed yesterday. Any ideas?

From what we can see this is a bug effecting about 10% of users. If you haven't been forced to log out please do so yourself and update your password.

JustineMumsnet · 20/08/2015 11:02

@tribpot

I'm surprised to hear you're starting security testing now, I assume actually this has been ongoing for some time.

I asked the following earlier

Are the general login problems that have dogged the site for weeks completely separate from this attack? How could you be sure?

Jeffrey was able to modify posts, was this by phishing MNHQ account details?
In this case I think we have established that RebeccaMumsnet was targeted specifically to capture her password via a dodgy link?

Have you reported yourselves to the ICO?

Why is there a need for support for such a wide range of browsers?

Internal testing has been going on since we've known someone accessed the database - literally night and day - we've been reviewing code across the site. What we are doing now is stress testing via an external firm.

JustineMumsnet · 20/08/2015 11:17

@tigerscameatnight

Justine are you able to confirm if they have accessed addresses given through product testing or insight panel or local ed info (I am an old local Ed)

Insight data (including product testing) is kept in a different system and protected with a username/password combination which is changed regularly and set by our tech team. We have no reason to think this system was compromised.

Local data is only accessible by individual LEs or admin. Admin access was shut down last week - we don't know for sure what was accessed prior to that but we have no evidence to suggest it was.

JustineMumsnet · 20/08/2015 11:22

@JustineMumsnet

[quote tigerscameatnight] Justine are you able to confirm if they have accessed addresses given through product testing or insight panel or local ed info (I am an old local Ed)

Insight data (including product testing) is kept in a different system and protected with a username/password combination which is changed regularly and set by our tech team. We have no reason to think this system was compromised.

Local data is only accessible by individual LEs or admin. Admin access was shut down last week - we don't know for sure what was accessed prior to that but we have no evidence to suggest it was.[/quote]

Update - we don't store addresses relating to local in the part of admin that was breached - they are kept elsewhere and password protected.

RebeccaMumsnet · 20/08/2015 16:17

@wannaBe

justine is there any way of getting to reports of dubious posters quickly? There is currently a fairly dubious one on the 3rd hacker thread who has essentially already been called as a troll/jeffrey, although she has put her ahem unique typing down to the fact she is dialectic and has leaning difficulties... IMO posters are on to her, but the fact she is essentially claiming that no-one is safe from hacking of all their accounts isn't helping the current of unease atm.

Hi wannaBe,

Thanks for the heads up, we have had a look and have no reason to believe that this is a hacked account. Thanks for the heads up and apologies for the delay.

BeccaMumsnet · 21/08/2015 13:17

@RustyParker

I'm not on the list but just remembered that a couple of days ago I had a notification from Facebook saying someone had requested a reset of my FB password. Could this be linked?

Hi RustyParker - we can't say for sure if this is linked. The best thing to do is just ignore and delete these emails. If you are concerned, you can change your email and password on your Facebook account to be sure and to stop the emails coming through.

BeccaMumsnet · 21/08/2015 13:21

@SouthAmericanCuisine

Akkakk. My concern isn't the hacker, I couldn't give a toss what s/he reads.

My worry is the my anonymity of the posts I made under the username that has been publicly linked to my IP address. Can someone who knows my IP address (my boss, my DC, my spouse) work out which is my username by linking it to an IP address they already know from the list?

It seems that the answer is possibly. That doesn't sit well with me.

Are MNHQ going to make provision for the complete posting histories of those usernames to be deleted if they wish?

Hi SouthAmericanCuisine - please do email [email protected] and explain your situation and what you'd like done.

BeccaMumsnet · 21/08/2015 13:35

@LaurieJuspeczyk

I really don't think anyone should be worried about IP addresses - as previous posters have said the chances that most people will be identified from that alone are virtually zero. My only worry would be that if they have people's email addresses, it's not impossible that those might be published alongside usernames at a later date. Again, though, as BertieBotts says, someone would have to know about the hack, find the list, search for your email specifically, trawl through all your posts looking for anything interesting... It's way too much effort for the average nosy person, I'd only be worried if you already have a stalker/vindictive ex etc or if you've posted anything that might completely ruin your life it if came to light.

Question for MNHQ/tech - as you (presumably!) don't own the phishing site the answer is probably no, but having found the problem, do you have any idea how many users are likely to have been affected?

Hi LaurieJuspeczyk - no sorry there isn't. We know it was likely to be running for a while hence we have to assume it's everyone who logged in. Therefore everyone should change passwords anywhere they used them.

BeccaMumsnet · 21/08/2015 13:58

@boatsgoby

Please could MNHQ ask all users to delete PMs from their inbox? Many of us use the PM function to exchange addresses, tel. nos, more personal details, etc. We can delete our own PMs but the recipients need to delete theirs too.

Hi boatsgoby - there will be a response on your Site Stuff thread shortly.

BeccaMumsnet · 21/08/2015 14:00

@SmallLegsOrSmallEggs

Is therexany sign of the delete all for PMs?

I need to delete not only all my received, but sent also.

Hi SmallLegsOrSmallEggs - yes this function is now available.

BeccaMumsnet · 21/08/2015 14:19

@iamaboveandBeyond

I asked elsewhwre, but it got missed. If we were definitely logged out, went to log in page, filled in details and it did log us in, could that info have been phished? Or would it have to not work? Basically, could a phishing page actually log you in?

Hi iamaboveandBeyond - once 'log in' is clicked on the phishing page, you would've been sent back to the original page you were on, which will have seemed like you got logged in.

DavidTech · 22/08/2015 02:23

Hi wasonthelist.

I do realise that a lot of people think of phishing as (solely) "getting an email with a dodgy link which takes you to a fake login page" and indeed, that is one kind. But it's not the only kind, it wasn't the kind we suffered and as far as I know we didn't say that it was.

Keep in mind that at the point I was discussing it with you, or others like you if you're not the PMer I am guessing you are, we didn't necessarily want to give away too much information because there aspects of it we still needed to pin down. We knew the data was captured by phishing (in the sense of a fake login page) but there was more work to be done to ascertain precisely how people were arriving at that page. So it was deliberately a touch vague. Sorry if that was frustrating, but I do believe it was necessary at the time.

I can see the confusion quite clearly now - if you were saying "I am 100% sure it's not phishing [I never click links in emails]" and we were saying "it absolutely was phishing [because someone used a webpage that was masquerading as an MN page when in fact it wasn't]" then clearly there was, as you say, a mismatch. It was phishing, but not the kind you thought we meant. I don't know if it really is a case of [only] being defined this way in rarefied techy circles, but this was a technical thread. I agree though, maybe the low-tech message to the wider audience should use different words, though I am not sure how else one should describe it. Perhaps "a fake login page that you wouldn't have known was fake and didn't require you to do anything unwise" would be better; it is accurate but a bit of a mouthful!

I hope that adds a bit more clarity, though from the sounds of your post you'd pretty much figured all this out yourself.

DawnMumsnet · 22/08/2015 16:02

@Simurgh

DavidTech

If you're speaking for MNHQ, go and do so when you can have a blue post? What you said seemed unexceptionable but I think that blue is more reassuring at the moment given the possibility of...... interlopers.

Thanks for the information anyway. Smile

Hi all,

We've had a few queries about this. Don't worry, there's nothing sinister going on. We'll tweak things to make sure all of DavidTech's post show up in (reassuring) blue.

JustineMumsnet · 24/08/2015 14:55

Hi everyone. I'm pleased to say that all was reasonably quiet at the weekend so happily very little has changed since the end of last week. We've no evidence of further attacks on Mumsnet, or indeed further swatting attacks.

As you know we patched the hole used to access user login details in the middle of last week and forced a password update. The site is currently undergoing stress testing via an external security firm which should be completed this week. Meanwhile, the tech team are reviewing every bit of code to make sure it's as robust as can be.

Obviously we can't be complacent though - please do report anything that looks at all dodgy, and we'll investigate straight away.

I know some users are still awaiting for replies to mails you sent us last week. Thanks so much for your patience, if that applies to you - we're very much hoping to get through the backlog in the next couple of days. For those with new queries it's worth checking our FAQs page before mailing in case your question's answered there (we'll keep it updated).

The police taking these attacks seriously and are conducting a full investigation. We will, of course, update you with any news on that front as soon as we have any.

Once again, many thanks for you support and virtual (and in one case, actual Grin) provision of gin and many apologies, again, for any undue stress/ anxiety caused.

Watch this thread for updates

Tap "Watch" to get all the latest updates