Data, data, lots of data (about this board)

[SummerLightning]

I posted this on the Westministenders thread, but thought I would add a new thread for it.

There was some discussion about getting some stats about posting times and what/who people posted about at one point a while back.

So I ran some basic analysis on threads and posts on here. You can see a nice graph of it here

public.tableau.com/shared/B6QDZQKDH?:display_count=yes

It looks better on a desktop rather than a phone, though I think it will work on a phone.

Select a date or a group of dates at the top and you can see the top posters for that date or set of dates.

Select a poster and you can see the dates they posted.

I have made a basic stab at colouring some of the more prolific posters blue (remain) or red (leave) according to their persuasions.

Here are some facts I put together:

There have been over 127,000 posts on the board since it started

Over 5000 of these are from one user alone. I will leave you to look at the stats to see who that is

Of the top 10 posters only one is pro-leave

There have been 36 Westministenders threads. So that’s over 35000 posts.

There have been 11 Brexit arms threads.

There are several peaks of posting:
The referendum date (over 5000 posts on 26th June)
Around 3rd November (probably the Tory party conference?)
Around 17th Jan (when it became apparent that yes we were leaving the single market)
Around 17th Feb (not sure? Tony Blair throwing in his tuppence perhaps?)
29th March (Article 50)

Enjoy. If you want to, I think you can download the actual data I used from this link too.

(I did it to learn some new skills rather than to prove any political point, btw. Oh and Tableau is awesome)

PS There is nothing "dodgy" or "hacky" about this, it is very basic taking data from the internet where you all put it . I know nothing about you posters except your usernames (no Cambridge Analytica cleverness from me!).

I didnt say you were boring summer

I was implying that you were bored

Like i was at work today

Oh i am on the stats yay!!!!

Yep i am the renegade spartacus reindeer

I can yes Swedish. Will put similar graph together by time of day but not til later as i have to work now (different timezone)

You sound like me lala

I usually say 'its my understanding that...' or 'i am confused'

What was the statistical probability that LurkingHusband would be the first poster to use this, up to now fairly good-natured, thread to make a derogatory comment about leaver posters? Further analysis needed perhaps

Hopefully 100%, or I'd be slipping. .

Coming up next - cross tabulation of whether you think I am creepy/interesting/boring/clever by whether you voted leave or remain

wrong probability quite low I'm afraid, was always going to happen, but who said it - my data says many possible suspects.

lurking naughty naughty. My data has no prejudice between leave/remain. Until I upload the actual content of the posts and run some machine learning sentiment analysis across it that is...

Hopefully 100%, or I'd be slipping

It's not something to be pleased about . Most posters , particularly the regulars, on these threads seem to be moving towards a stage where we can have civil, respectful conversations. Which is how it should be. Posters chipping in with derogatory stereotyping remarks about leavers, or indeed remainers, is really not helping anyone. Well, perhaps it makes you feel better in some way, I can't understand how or why, but it's not helping anyone else or the state of political discussion in this country.

Most posters , particularly the regulars, on these threads seem to be moving towards a stage where we can have civil, respectful conversations

To clarify, many posters were doing this all along.

Summer

A huge bulk of the threads have been the Westministerenders threads, which have carried on, with peaks and troughs, but always going.

The Brexit Arms has been more intermittent, busy sometimes but with gaps.

Obvs WM is remainer focused, although we leavers have always been welcomed to contribute.

I would be interested to see the remainer/leaver split if the WM and Brexit Arms threads were excluded to see if the number of remainer posts are still much higher.

Just a thought if you feel the urge to do any more data analysis flim flam and are looking for questions to answer

rufus I do stuff like this even when I am not bored! It took a bit too much concentration to do it while boss is waffling on so I did it in my own time (though some of the extraction script was taken from work I did at work).

I agree squishy re the more questions, etc, etc. However, you have to be careful you don't ask very specific questions to confirm your own bias! Kind of if you want to find something in your data and you ask the question enough slightly different ways eventually you will find it. Lies damned lies and statistics and all that.

danny astroturfing is the same poster posting under different names right? It would be very hard to find unless it was really happening very regularly. Two posters always posting at the same time and confirming what each other says perhaps? But given I don't have the content of the posts, hard to do...?

I have to say I am not surprised but genuinely kind of want to comment on people thinking this is creepy. I take the point about google and facebook doing analysis with a goal in mind. You have to think what that goal is though. For anyone interested in this I would suggest reading Cathy O'Neil's Weapons of Math Destruction - very interesting on how data analysis, machine learning can get in a reinforcement loop that confirms its existing bias - resulting in people from certain demographics being targeted differently in quite alarming ways, or charged different amounts for insurance, etc.

Summer - I've opened it but I can only see the page with the graph. How do I see the other pages?

wrong easy to do, I will look into it.

So many posters I have unmarked as leave/remain at the moment, if you look at the posts around the referendum I saw that by far the majority are from posters that only posted a bit around that time.

dapple I think I had set it so you couldn't see the underlying data actually. I've changed it now.

Do you see two graphs that are interactive with one another?

Now if you click a user at the bottom a little popup comes up and one the right there is a little icon that looks kind of like lines in a book. Click on this and you can actually see the underlying data - when the post was made and (if you click show all columns) the names of the threads and urls)

OK I must do some real work...

summer, serious question; what is your motivation for doing this?

I've never seen you around this section of MN before- why the curtain twitching now?

What do you hope to prove by doing it?

bear I read this section all the time - it's the only thing I read on mumsnet. You'd be surprised how many lurkers there are I expect. By the time I read I have no time to post. I am very interested even though I don't post much.

I am not sure I want to prove anything. Just share something interesting and start a discussion. I am interested in suggestions that there are paid posters across the internet, but skeptical about the extent. I have no doubt there are twitter bots. I think I could write one myself if I wanted! I am not sure my data could show/prove it even if there was.

I'm interested in machine learning/artificial intelligence. I could try to do so much with the data including the content of the posts if I had time. But now I will creep you out more.

This article is what got me first so interested in this stuff

medium.com/i-data/israel-gaza-war-data-a54969aeb23e

I defy you to read that and not think it's fascinating. If you don't your mind doesn't work like mine.

I don't find it fascinating at all and I 'do' numbers for a living.

It shows nothing other than the number of people on here who feel the most strongly about a particular subject.

The shills are only here for a short time and goady fuckers that name change all the time wont feature in this analysis so it doesn't weed them out.

What do you think it actually proves?

I work with juggling numbers across a gazillion spreadsheets for a living (so also somewhat of a geek) but I'm still finding this a bit bizarre.

I'm interested in machine learning/artificial intelligence. I could try to do so much with the data including the content of the posts if I had time. But now I will creep you out more.

Yup you would!!

I agree the stuff I have done isn't that fascinating, I was referring to the article I posted.

As I said, I'm not sure it proves anything, however, it has provoked discussion.

sorry time

Now I'm starting to feel a weirdo.

I agree with Bearbehind.
What's the point?
All it shows is a group of posters who post a lot in a topic they're interested in.
You could do this with any board on MN.

I love it op! Id like to commission you to analyse some individual threads maybe?? (I'd pay you in gushing praise, and sincere thanks)
As a total, and utter failure at anything mathematical, seeing the numbers crunched into graphs helps those like me immensely.

My initial unscientific analysis just licking my finger, vaguely looking at the posts suggests that there is little correlation between liking my analysis and voting leave/remain.

surfer aye, I could indeed. tonight I shall run my analysis on Style and Beauty.

wendy if you ask me to do something that's easy to do, I'll do it! however I have to warn you that my DH gets back from a business trip tonight so I may spend my spare time watching TV, drinking wine and doing, erm, normal stuff with him, rather than this.

I agree with Bearbehind

Bloody hell surfer - I bet you never thought you'd use those words in the same sentence!!!!

I know!

United in distrust of me and also thinking my thread is pointless.

Thanks, I am flattered

It's not the first time I have been accused of posting boring pointless threads on mumsnet, but I think the last time was years ago when I posted about how to freeze carrots so somewhat less controversial.