Data, data, lots of data (about this board)

[SummerLightning]

I posted this on the Westministenders thread, but thought I would add a new thread for it.

There was some discussion about getting some stats about posting times and what/who people posted about at one point a while back.

So I ran some basic analysis on threads and posts on here. You can see a nice graph of it here

public.tableau.com/shared/B6QDZQKDH?:display_count=yes

It looks better on a desktop rather than a phone, though I think it will work on a phone.

Select a date or a group of dates at the top and you can see the top posters for that date or set of dates.

Select a poster and you can see the dates they posted.

I have made a basic stab at colouring some of the more prolific posters blue (remain) or red (leave) according to their persuasions.

Here are some facts I put together:

There have been over 127,000 posts on the board since it started

Over 5000 of these are from one user alone. I will leave you to look at the stats to see who that is

Of the top 10 posters only one is pro-leave

There have been 36 Westministenders threads. So that’s over 35000 posts.

There have been 11 Brexit arms threads.

There are several peaks of posting:
The referendum date (over 5000 posts on 26th June)
Around 3rd November (probably the Tory party conference?)
Around 17th Jan (when it became apparent that yes we were leaving the single market)
Around 17th Feb (not sure? Tony Blair throwing in his tuppence perhaps?)
29th March (Article 50)

Enjoy. If you want to, I think you can download the actual data I used from this link too.

(I did it to learn some new skills rather than to prove any political point, btw. Oh and Tableau is awesome)

PS There is nothing "dodgy" or "hacky" about this, it is very basic taking data from the internet where you all put it . I know nothing about you posters except your usernames (no Cambridge Analytica cleverness from me!).

And Tanith, I think that freedom of information requests only apply to public bodies. Which MN is not!

I find it fascinating as well but agree about the hmm aspect of it as well. But not sure why.

It doesn't take account of length of posts. So lots of s (or, more likely, faces!) could get you higher. And I'm pretty sure I can see a name-changer in there.

SummerLightning

I think I love you

See I don't think you can say with any certainty at all that '5000 of these were from one poster alone' nor 'of the top ten posters only one was a leaver'.

It would be more accurate to say that the data show posts for each username rather than for each poster. If a poster has namechanged, which some have (I would suggest more leavers have than remainers have but I have no evidence for that, it's just a hunch), then the data doesn't link up the two or more usernames for that poster. So it needs to be looked at with that in mind. Fascinating though.

I agree. Mumsnet locked me out and I had to name change in a freak incident.
Alas, it means all my neglecting of the children has been in vain- I didn't even make top twenty!

histiny yes you are correct it doesn't deal with name changes. You can always add up you and your previous posters counts and figure out if you would have made the top twenty .

rufus please don't worry about accidentally calling me boring. I am boring, I love data what can I say

wrong thankyou very much. to you too.

Yes I think FOI does not apply to mumsnet! - I think they would have been able to get this data much quicker than me, but would have told me to sod off

I got the data using a python script to trawl the webpages (geek alert). I did reuse this at work tho!

^Just wondering if you looked at present posters and their (our) frequency. Or if you looked at the numerous posters who seemed to overwhelm everything around the clock at one point.

What I mean did you look at a certain date and see who was most prolific or did you look at present posters and see when they were at their most active.

I suppose it won't capture a group of 50 posters who come and post with a particular purpose in mind and then disappear.^

Calleach I haven't really looked at anything, per se, just presented the data.

That is one of the things that you could look for in the data, very prolific posters who come and then disappear - obviously you won't be able to categorically tell why they did this though. Name change, quick bee in their bonnet, came in a link from elsewhere and got bored, paid Russian troll, etc, etc.

For those asking why, yes it is because it's pretty and fun to work with data, and more so with real data that you find interesting. I was very excited when I first imported my hard earned data, put it into the time series graph and saw the massive peak around June. As it meant I seemed to have done it correctly. And.....we're back to me being boring again .

And for those who find it creepy, sorry! BUT you would be much more creeped out by what facebook/google etc knows about you!

Can I just ask Summer did you include all the threads in Referendum 2016 but no threads in other sections?

Looking at the peaks and troughs is really interesting, especially how quite it was during August and September.

And for those who find it creepy, sorry! BUT you would be much more creeped out by what facebook/google etc knows about you

Facebook and Google analyse data for a purpose; I don't understand your motive for doing this in the first place but certainly not for posting it.

What do you think it achieves?

anlaf tableau public is a little annoying though as when you want to save what you have done you have to upload it. So you can't save a half screwed up version without making it public. (unless I'm doing it wrong).

Very clever on their part I guess as people can use most of the functionality for as long as they want but are likely to want the paid version if they want to use it professionally. Apparently some journalists do use Tableau public though.

rufus you are in my data, are you both the renegade and the spartacus reindeer? The name change will have dragged you down the ratings!

bear it's got you all talking, hasn't it? It's interesting? Others seem to find it interesting? I thought it might be interesting to see certain trends. there are probably interesting trends to find in there above the obvious ones.

wrong correct, just this board.

Is there anyone on there who could have been considered to be astroturfing? Or are all posters there genuine?

I'm betting we'll see a lot more instances of this kind of thing in the future ... as a sniff test for whether a site/forum is peddling sock puppet posts, or truly representing a cross section of real people.

Of much more interest would be the same exercise applied to the BBC comments section, plus the Daily TeleMail comments sections.

I think it would be much creepier if you organised the data and didn't post it, somehow.
August and September quiet period = holidays maybe?
There was one particular username on there who posted a vast amount in a short space of time before moving onto the politics board to post vast amounts about US politics disappearing from the referendum boards.

Surprisingly I'm on the leaderboard and thought I was only an occasional poster who prefixed most comments with "I'm confused about..." or "Do you mean...?"

lurking i have thought about doing that! There must be others doing it.

The thing about presenting data like this is it throws up so many questions - like the ones already asked on this thread - and addressing these requires sorting and presenting the data in a different way, which throws up more interesting questions, which require even more in depth analysis etc etc...

(I don't think you're creepy or boring Summer. Your work is both interesting and clever)

We're straying into Homeland territory here . Where's Max ?

Alternatively, we have a 2000s version of the infinite monkeys paradigm.

If you had an infinite number of monkeys, with an infinite number of keyboards and infinite bandwidth, they will eventually come up with a Leave post that makes sense

Interesting and rather creepy, glad I name change regularly

Tanith

Mumsnet is not a public authority.

Thank you Summer - that's very interesting!

Given that I try to tend to post mostly on the Westminstenders threads and avoid the Brexit Arms , I have posted an awful lot

Having said that, I'm glad to see the well-argued Leaver contributions from the likes of Semipermanent , corcory and Howabout (I don't think that I live that far from howabout and probably in real life, we could have some really interesting and friendly disagreements discussions )

If you had an infinite number of monkeys, with an infinite number of keyboards and infinite bandwidth, they will eventually come up with a Leave post that makes sense

What was the statistical probability that LurkingHusband would be the first poster to use this, up to now fairly good-natured, thread to make a derogatory comment about leaver posters? Further analysis needed perhaps?

Can you see times of day that people post? I do wonder if you could work out name changes relatively easily - or, at least, narrow them down - as you can see when a prolific poster stops and a new prolific poster starts.

Wish I could remember all the names of the paid posters.