Meet the Other Phone. Flexible and made to last.

Meet the Other Phone.
Flexible and made to last.

Buy now

Please or to access all these features

Site stuff

Join our Innovation Panel to try new features early and help make Mumsnet better.

See all MNHQ comments on this thread

Mumsnet Corpus

1000 replies

TokyoBouncyBall · 19/04/2024 11:36

Not a TAAT, but a bit of googling as a result of a now deleted thread has led me to this:

https://fold.aston.ac.uk/handle/123456789/18

I note it says that the License is uncertain. Can you confirm that you have given permission for posts to be used in this way, or is there something that Aston might like to look into?

I note it says Users who wish to access this dataset must make a detailed application to FoLD and the researcher, as well as potentially gain additional agreement from an external organisation before they can be approved for access.

Given one of the uses it is being put to, I think it is a bit dubious to say the least.

OP posts:
Thread gallery
82
AstonToTheNaughtyStep · 03/05/2024 08:47

"If you don’t consent to it at all you probably shouldn’t be posting on here because Mumsnet have been perfectly open about the fact that they do sometimes allow it and I don’t think they always make individual permission a condition?"

Thank you for bringing that to my attention. It is not something I was aware of until very recently, and I have significantly reduced my posting frequency in the wake of the Aston incident. At this point I'm only really hanging about to see how this plays out.

VitoCorleoneOfMNMafia · 03/05/2024 09:10

AmaryllisNightAndDay · 03/05/2024 08:37

Hm, The link to the Springer (Manchester) article isn't working this morning. Points to the journal but not the article.

Meanwhile the Newcastle article has references to other articles doing similar things: "Data collected from Mumsnet have previously been used to describe the views of parents (particularly mothers) and answer a wide range of research questions [17-20]."

which led me (among others) to this from UCL (2020) https://mental.jmir.org/2020/9/e18271
"The use of mumsnet by parents of young people with mental health needs: qualitative investigation. JMIR Ment Health."

MumsNet might possibly have agreed to this one but there's no indication that MNHQ were asked or permission was granted, and it looks as if the reverse is true:

"Parsehub was used to extract data from Mumsnet threads. Parsehub is a freely available web-based scraping tool designed to extract internet data. Any original posts or comments including information that could potentially identify the user, such as age, name, or location, were omitted manually by the researchers before the data were analyzed. Following this, the raw data were transferred into word documents for analysis."

So this team UCL have been scraping conversations between distressed mothers about their children with mental health needs. The paper includes brief anonymised quotes from individual posters.

And to start answering @MarkMenziesFakeMugger why do they think this is OK?

The ethics section says they had approval from the UCL ethics team, plus this:
"In line with the recommendations of the Association of Internet Researchers Ethics Working Committee [23], all data were extracted without the inclusion of usernames, and direct quotes were altered slightly (without changing meaning) to maintain the privacy of those posting on the forum during the initial data extraction stage of the analysis."

So here is a link to that paper about ethics: https://aoir.org/reports/ethics2.pdf

Markham A, Buchanan E. Ethical Decision-Making and Internet Research: Recommendations from the AoIR Ethics Working Committee (Version 2.0). Association of Internet Researchers. 2012.

Members of the AoIR Ethics Working Committee who contributed to the report are listed at the top, dunno what the mix is of Information Retrieval and ethicists. It's rather elderly (2012) My quick skim and a keyword search says that this document does mentions copyright as a potential problem, and terms and conditions. It mentions scraping right at the start but nowhere else. It says this is ethically complex and it doesn't mention the bleeding obvious - try reading the Ts&Cs and asking the site owners. Maybe they have a better idea of what's acceptable on their own site than you do!

Archive at https://archive.ph/QRfIY

Maybe we tired the Springer server out by all reading that paper at once and it's having a nap?

RethinkingLife · 03/05/2024 09:11

It would be helpful in MNHQ would put together a list of the research projects (and harvesting) to which they have consented. So that we know when checking publications rather than being left in a state of not knowing.

PerkingFaintly · 03/05/2024 09:39

RethinkingLife · 03/05/2024 09:11

It would be helpful in MNHQ would put together a list of the research projects (and harvesting) to which they have consented. So that we know when checking publications rather than being left in a state of not knowing.

Yes, that would be useful – and probably short enough not to be too burdensome for MNTowers to produce.

SqueakyDinosaur · 03/05/2024 09:56

Maybe we tired the Springer server out by all reading that paper at once and it's having a nap?

Don't be ridiculous! It's impossible to tire out a Springer. You can take it for a 2 hour walk and it will still demand to play ball or try to climb trees to catch squirrels. I speak from experience.

Dumbledoreslemonsherbets · 03/05/2024 09:56

AstonToTheNaughtyStep · 03/05/2024 08:47

"If you don’t consent to it at all you probably shouldn’t be posting on here because Mumsnet have been perfectly open about the fact that they do sometimes allow it and I don’t think they always make individual permission a condition?"

Thank you for bringing that to my attention. It is not something I was aware of until very recently, and I have significantly reduced my posting frequency in the wake of the Aston incident. At this point I'm only really hanging about to see how this plays out.

Yes, this (I'm heading over to Ovarit and won't post anything personal anymore) but actually I think MN need to consult the ICO about this.

If it's sensitive data (and possibly even if it's not) new consent is required for a new use and a catch all 'we might use this for any research in the future' just isn't really within the spirit of the law at the very least. I think if people posting on fertility boards knew there was going to be specific fertility research they may or may not be happy to sign up - but just mining their posts? I think that's a no under existing legislation. Where I work, we need to get a new consent for a slightly different use of someone's data and it's a lot less contentious than what's happened here.

Some women on this site are very vulnerable and very isolated and they post here in desperation - I've seen some abuse threads along these lines. Often these threads do eventually get deleted so would not be used for research which had MN permission, but for those that don't, I really think new consent is needed.

Dumbledoreslemonsherbets · 03/05/2024 10:01

ArabellaScott · 03/05/2024 07:18

Yes, there needs to be a site wide announcement and as many MN users and past users need to be informed.as.is possible. The ICO needs to be informed.

Women have shared the most intimate details of their life on here; and while most know its a public forum, that doesn't mean anyone anticipated the possibility of software potentially being used to identify people.

Aston may deny that that's been done so far, but this is one of the activities the Institute are involved in, and it seems these Unis feel entitled to scrape, store, use and sell data as they please. Which doesn't inspire confidence.

Any mass data held by universities - or commercial companies- should be deleted if it was scraped without permission.

This - why should we believe they're not trying to use the tools they're developing to identify individuals in real life from internet posts to do it to the MN data they illegally took?

They've lied and lied and lied so far.

I'd like the ICO involved at the very least to have some oversight about whether they actually delete our data.

Dumbledoreslemonsherbets · 03/05/2024 10:03

Oh and just to add - where we need to get new consent for use of data - this is anonymised data which does not have the content creators name on it nor linked to any other identifying information. Our DPO is clear - EVEN thought it's anonymous and no identifiable data on it / linked to it at all, because it's THEIR WORK and they did not consent to it being used in this slightly different way WE NEED NEW CONSENT.

I suspect our DPO is right about this.

Dumbledoreslemonsherbets · 03/05/2024 10:16

I think one of the things anyone who is worried about Aston / other university data scraping can do is ask MN to delete all their past posts. As I understand it, you have this right under GDPR / DPA law.

I'd also like to know that research papers from illegally scraped data are deleted.

ArabellaScott · 03/05/2024 10:24

I've had a chat with someone at the ICO.

Copied and pasted below (with their name redacted and shown as ICO):

[03/05/2024, 10:11:27] Arabella: Hi there. I have a query about data scraping and am hoping you can help.
[03/05/2024, 10:11:46] Arabella: I'm a registered user of Mumsnet, a large social media site that users use pseudonymously.
[03/05/2024, 10:11:59] Arabella: The Terms and Conditions explicitly forbid data scraping without permission.
[03/05/2024, 10:12:35] Arabella: It's recently come to light that several universities have been scraping enormous quantities of data from Mumsnet without asking permission from either the websirte or users.
[03/05/2024, 10:12:51] Arabella: This is then stored, sometimes 'made available' for research.
[03/05/2024, 10:13:10] Arabella: Many users are concerned that sometimes personal and intimate data has been shared without consent.
[03/05/2024, 10:13:16] [ICO]: It would very much depend on the lawful basis being relied upon to use the data. This can often be consensual during sign-up but we would need to investigate fully to be able to provide a definitive outcome decision.
[03/05/2024, 10:13:37] Arabella: Would this be when a university claims it's for 'research'?
[03/05/2024, 10:13:48] Arabella: How could we ask for an investigation?
[03/05/2024, 10:13:55] [ICO]:: Potentially. I would first
try raising your concerns with the organisation's Data Protection Officer (DPO)
in writing (details often found on their privacy policy at the bottom of their
website). If you fail to get a response or an inappropriate response please
feel free to raise a complaint with the ICO and we will investigate
accordingly.
https://ico.org.uk/make-a-complaint/data-protection-complaints/data-protection-complaints/
[03/05/2024, 10:14:06] [ICO]:: In order to investigate, and
as an evidence-based regulator, we require service users to raise their
concerns with the organisation's Data Protection Officer (DPO) first, and have
received a response to their correspondence. If you have already contacted the
DPO and they have provided a response (or lack of after one calendar month) you
would be in position to raise a complaint with the ICO here:
https://ico.org.uk/make-a-complaint/data-protection-complaints/data-protection-complaints/

If you haven't yet raised your concerns with the
organisation's Data Protection Officer we would kindly ask that you do that
first (details often found on their privacy policy at the bottom of their
website). If you need further guidance, and a number of useful templates to
assist with raising information rights complaints, please find additional
material here: https://ico.org.uk/for-the-public/

If you are unable to raise your concerns with the DPO please
feel free to raise your complaint via the link and we'll investigate
accordingly.
https://ico.org.uk/make-a-complaint/data-protection-complaints/data-protection-complaints/
[03/05/2024, 10:14:28] Arabella: Thanks. Do you mean the Mumsnet Data protection officer or that of the Universities involved?
[03/05/2024, 10:14:57] Arabella: Part of the problem is that women have shared information anonymously and often don't want to compromise that by contacting either organisation directly
[03/05/2024, 10:15:05] [ICO]:: Both parties. Whomever is processing or securing the data.
[03/05/2024, 10:15:12] Arabella: Thanks. That's really helpful.
[03/05/2024, 10:15:21] [ICO]:: If you feel unable to raise your concerns with the DPO please feel free to raise your complaint via the link and we'll investigate accordingly. https://ico.org.uk/make-a-complaint/data-protection-complaints/data-protection-complaints/

For the public

We live in a data-driven world. Almost every transaction and interaction you have with most organisations involves you sharing personal data, such as your name, address and birth date. You share data online too, every time you visit a website, search f...

https://ico.org.uk/for-the-public

TokyoBouncyBall · 03/05/2024 10:33

Dumbledoreslemonsherbets · 03/05/2024 10:16

I think one of the things anyone who is worried about Aston / other university data scraping can do is ask MN to delete all their past posts. As I understand it, you have this right under GDPR / DPA law.

I'd also like to know that research papers from illegally scraped data are deleted.

I'm not sure this is helpful as it will just make MN a mess and won't change the scraped data. And will damage MN, not Aston

If you really want to make their lives difficult, create a gmail address under one of your usernames, and mail Aston's DPO and ask for all your posts to be deleted from their records. Then report them to the ICO under the same gmail address.

OP posts:
TokyoBouncyBall · 03/05/2024 10:39

Some work on Manchester. Springer link is back up, so have run a search on ethics.

It says this: The study was approved by the University of Manchester computer science department panel of ethics, with reference 2020-8214-12903.

Now, this is not compliant with the university's own ethics guidance (link below) which says that School review is for low risk student projects only (waves to Eden Palmer). So it should have gone for a University level review.

And their policy says:

You are able to provide explicit consent from the data controller to access the data;
You are able to prove that the data will be used for a purpose which falls within the remit of the original consent provided by data subjects.

I see this as an epic fail from Manchester. Will be reporting this post so that MNHQ see it.

https://www.manchester.ac.uk/research/environment/governance/ethics/approval/

University ethical approval | The University of Manchester

Learn about the five possible routes of ethical review and approval at The University of Manchester.

https://www.manchester.ac.uk/research/environment/governance/ethics/approval/

OP posts:
MaiseeBee · 03/05/2024 10:45

That's really helpful @ArabellaScott. Justine mentioned they changed the T&C's to specifically ban scraping last year and it'd be good to see what they were before. I don't suppose anyone here has kept a copy of the old T&C's?

BIWI · 03/05/2024 10:48

Encyclopediaofnonsense · 03/05/2024 08:23

I think the longer MNHQ leave a lot of these things unanswered now the more reputationaly damaging it will be in the long run. Lots of posters will want to know why they weren't informed early on, lots of posters will want their data deleting and lots more will want assurances they won't be identified.

I think this stoic silence and non-answers from MNHQ is going to do them more harm in the long run.

I think this is rather unfair - there's obviously stuff going on in the background - probably involving lawyers - and Justine has been back to give her response as to what's happened so far. I'd suggest that there isn't any obvious course of action for MN or MNetters to take at the moment, until a plan has been agreed.

Ormally · 03/05/2024 10:49

Dumbledoreslemonsherbets · 03/05/2024 09:56

Yes, this (I'm heading over to Ovarit and won't post anything personal anymore) but actually I think MN need to consult the ICO about this.

If it's sensitive data (and possibly even if it's not) new consent is required for a new use and a catch all 'we might use this for any research in the future' just isn't really within the spirit of the law at the very least. I think if people posting on fertility boards knew there was going to be specific fertility research they may or may not be happy to sign up - but just mining their posts? I think that's a no under existing legislation. Where I work, we need to get a new consent for a slightly different use of someone's data and it's a lot less contentious than what's happened here.

Some women on this site are very vulnerable and very isolated and they post here in desperation - I've seen some abuse threads along these lines. Often these threads do eventually get deleted so would not be used for research which had MN permission, but for those that don't, I really think new consent is needed.

Brilliant set of recent posts, but this one particularly.

If a (mere) Ts and Cs update is a natural consequence of this point in time, I would also want to have all historical account data and messages deleted before that.

This would need to apply to the primary controllers of the online forum up to the moment of change of use terms, but also to any derivatives from the forum where legitimate interest was not made explicit for a funded research purpose, and where projects are technically subject to an external ethics assessment for its uses.

Ormally · 03/05/2024 10:58

MaiseeBee · 03/05/2024 10:45

That's really helpful @ArabellaScott. Justine mentioned they changed the T&C's to specifically ban scraping last year and it'd be good to see what they were before. I don't suppose anyone here has kept a copy of the old T&C's?

The Wayback Machine comes to mind.
There are snapshots from various years on it, and the link to the privacy policy and terms and conditions seems to be active.

This is what came up from a look at 17 Jan 2018 - privacy policy link. DPA is still the act that is relevant. The changes through GDPR coming into play should certainly be trackable.
The part with asterisks on it (added by me) is interesting.

Our privacy policy
This privacy policy sets out how Mumsnet uses and protects any information that you give us when you use this website.

We are committed to ensuring that your privacy is protected. Any identifying information we ask you to provide will only be used in accordance with this privacy statement.

Mumsnet may change this policy from time to time by updating this page, so please do check it from regularly to ensure that you are happy with any changes. This policy is effective from 1 March 2017.

Your consent
By using this site, you agree to the terms of this Privacy Policy and Mumsnet's <a class="break-all" href="https://web.archive.org/web/20180331222204/www.mumsnet.com/info/terms-of-use" rel="nofollow" target="_blank">Terms of Use. Whenever you submit information via this site, you consent to the collection, use, and disclosure of that information in accordance with those Terms of Use and this Privacy Policy.

What we collect
We may collect the following information:

  • your IP address and device IDs
  • your email address should you choose to provide it
  • your chosen Talk name and password, your postcode, your date of birth, whether you are pregnant or not (and, if you are, when your due date is) and/or whether or not you have children and other demographic information, should you choose to register
  • your blog url and Twitter handle, should you join the Mumsnet Bloggers Network.
What we do with the information we gather

We do not, as a clear and overarching rule, pass on your personal information, unless you specifically consent to us doing so or we are specifically required to by law. However, on rare occasions where there appears to be a clear need to safeguard the welfare of a Mumsnetter and/or his/her family, it may be necessary to contact relevant authorities about a clearly identifiable member and his/her posts on Mumsnet.

We only use the information you give us to understand your needs and provide you with a better service, and in particular:

  • For internal record keeping.
  • To improve our services, and email you responses to discussion-board posts you have reported.
  • To periodically send you emails about parenting news, Mumsnet news and courses, competitions, discounts or other information which we think you may find interesting, using the email address which you have provided. This includes updates on the Mumsnet Blogging Network, should you join.
  • To send you pregnancy-stage emails or child development emails, should you choose to receive them.
  • To contact you, but only with your prior permission, by email or personal message, for market-research purposes.
  • To offer you, on our website pages, information based on your location.
  • To tailor advertising to make it more relevant to your likely interests and needs.
We occasionally provide anonymised data to other companies for purposes of understanding users in aggregate or to personalise advertising on other websites based on your usage of Mumsnet.

*** Any personal data that you give us will be retained by us for as long as it's needed to perform its function. There's no one-size-fits-all time limit, but to give some examples: data provided by members of our Insight panel for that specific purpose is retained for as long as the user remains a member of the panel. Information provided in respect of competitions is held for three months. We regularly review the data held by different teams and delete information that is no longer needed. All data we hold is stored securely and only accessible by authorised members of MNHQ.

Security
We are committed to ensuring that your information is secure. In order to prevent unauthorised access or disclosure, we have put in place suitable physical and electronic procedures to safeguard and secure the information we collect online.

Passive Information Collection
As you navigate through this site, certain anonymous information can be passively collected (that is, gathered without your actively providing the information) using various technologies, such as cookies*, clicktags, beacons and other navigational data collection (such as server logs). Your internet browser automatically transmits to this site some of this anonymous information, such as the url of the website you just came from and the IP address and the browser version your computer is currently using. Via the IP address we may also determine your rough geographical location, such as town or city.
This helps us analyse data about our site traffic, so we can improve Mumsnet and tailor it more to our members' needs. All information is anonymous and only used for operational purposes.
*A cookie is a small file which "asks permission" to be placed on your computer's hard drive. Once you agree, the file is added and the cookie helps analyse web traffic or lets you know when you visit a particular site. Cookies allow web applications to respond to you as an individual. A cookie in no way gives us access to your computer or any information about you, other than the data you choose to share with us. You can choose to accept or decline cookies. Most web browsers automatically accept cookies but you can usually modify your browser setting to decline cookies if you prefer. This may, however, prevent you from taking full advantage of our website. <a class="break-all" href="https://web.archive.org/web/20180331222204/www.mumsnet.com/info/cookie-policy" rel="nofollow" target="_blank">Read more about our Cookie Policy.

Links to other websites
Our website may contain links or references to other websites to which this Privacy Policy does not apply. If you click on these these links to leave our site, you should note that we cannot be responsible for the protection and privacy of any information you provide whilst visiting those sites. We suggest you always look at the privacy statement of any other website you visit.

Controlling your personal information
You may choose to change or restrict the use of your personal information by going to <a class="break-all" href="https://web.archive.org/web/20180331222204/www.mumsnet.com/member/" rel="nofollow" target="_blank">your My Mumsnet area and altering your registration details and subscribing/unsubscribing to our email options.
You may ask us to provide details of the personal information we hold about you, under the Data Protection Act 1998. To request this information, please mail us at [email protected] or write to Mumsnet Limited, Unit 6, Deane House Studios, 27 Greenwood Place, Highgate Rd, London NW5 1LB.
If you believe that any information we are holding on you is incorrect or incomplete, please write to or email us as soon as possible. We will, of course, happily correct any information that's found to be incorrect.

Signing in via Facebook or Google, what information do we collect?

If you choose to sign in via Facebook or Google you grant us access to information held by those services. The information we have access to depends on the service and is explicitly stated when you first connect those services.

Facebook
You may choose to sign in via Facebook. By doing so you grant us access to your email address, your profile and your friends list. We respect our users privacy so we choose never to ask for your Friends List and we will never use your Friends List. You may revoke access at any time via your Facebook account settings. We follow the guidelines set out by Facebook regarding data collection and use as laid out <a class="break-all" href="https://web.archive.org/web/20180331222204/developers.facebook.com/policy/#data" rel="nofollow" target="_blank">here.

Google
You may choose to sign in via Google. By doing so you give us your email address and basic account information. More details on signing into apps with Google can be found <a class="break-all" href="https://web.archive.org/web/20180331222204/support.google.com/plus/answer/2980762?hl=en-GB&ref_topic=2980759" rel="nofollow" target="_blank">here.

If you have any concerns about this policy or would like any further info please get in touch by emailing [email protected]

Sign in to other sites using Google - Google+ Help

You can use your Google username and password to sign in to other apps and websites, and you can control what information from each app will be shared on Google+. Tip: If you're having trouble signin

https://web.archive.org/web/20180331222204/https://support.google.com/plus/answer/2980762?hl=en-GB&ref_topic=2980759

DrBlackbird · 03/05/2024 11:04

AstonVillains · 03/05/2024 04:50

If the scraping is more widespread than originally thought, I think it might be necessary to make the talk section only visible to logged in members of the site. It won't stop people taking screenshots and posting on twitter, but could stop the scraping I think, although it might require some clever coding on the servers.

This ^

Surely something technical can be done to block scrapers. I know this is a public forum. But my god the laziness and contempt and disregard by researchers for the largely women posting in this space is despicable. Knowing that exact phrases are being used in research papers now means I will cut back considerably on my posting and definitely nothing about personal concerns. Such a shame. I suppose we have Eden to thank though for drawing our attention to what’s been happening.

Ormally · 03/05/2024 11:07

Comparing with a 3rd Feb 2020 policy on the Wayback Machine - not all copied -

How long is your data held

−Any personal data you give us, or that we collect when you use Mumsnet, will be retained by us for as long as it’s needed to perform its function. Registration data is held permanently unless you deregister. There’s no one-size-fits-all time limit, but we regularly review the data we hold to ensure it is still necessary.
Any personal data that you give us will be retained by us for as long as it's needed to perform its function.

There's no one-size-fits-all time limit, but to give some examples: data provided by members of our Insight panel for that specific purpose is retained for as long as the user remains a member of the panel. Information provided when entering competitions is held for a maximum of three months. Information you give us when you sign up is stored permanently unless you choose to delete it or unless you delete your Mumsnet account.

Private Messages are stored until you delete them. If you delete your account, your messages will also be deleted.

We regularly review the data held by Mumsnet’s different teams and delete information that is no longer needed. All the data we hold is stored securely and only accessible to authorised staff members, who have regular data training.

...And a new section on Updates (to the privacy policy, presumably):

Updates−Further updates will be noted here as and when they are made.

22 May 2018: deletion policy on Private Messages updated
8 May 2018: Privacy Policy updated
17 September 2018: cookie explanations updated to reflect new consent process
29 August 2019: updates to privacy policy – what data is stored and why
12 September 2019: Alexa Skills privacy policy included

DrSoupDragonsFriend · 03/05/2024 11:13

WookeyHole · 03/05/2024 06:56

I am no expert in this area and I am as apalled as you all; it was a pretty bleak personal circumstance which first brought me to mn (under a different username, of course) and the support I received was invaluable.

However, I don't think it's right to call this a data breach, and whilst MNHQ absolutely need to put a stop to this, the precautionary tale for all of us is that anything on the internet is accessible to anyone. It doesn't make it ok to misuse, but we can't be too trusting.

I think I should have been more careful about saying it was a breach. What's going on is more nuanced and multi-layered than that. I was posting way past my bedtime and was feeling very fed up about what I'd been finding and the scale of what must be out there. Subtlety wasn't in my brain!

Encyclopediaofnonsense · 03/05/2024 11:27

Dumbledoreslemonsherbets · 03/05/2024 10:16

I think one of the things anyone who is worried about Aston / other university data scraping can do is ask MN to delete all their past posts. As I understand it, you have this right under GDPR / DPA law.

I'd also like to know that research papers from illegally scraped data are deleted.

I did this a while ago and they haven't confirmed they can/will.

Codlingmoths · 03/05/2024 11:31

DrBlackbird · 03/05/2024 11:04

This ^

Surely something technical can be done to block scrapers. I know this is a public forum. But my god the laziness and contempt and disregard by researchers for the largely women posting in this space is despicable. Knowing that exact phrases are being used in research papers now means I will cut back considerably on my posting and definitely nothing about personal concerns. Such a shame. I suppose we have Eden to thank though for drawing our attention to what’s been happening.

There is smarter antiscraping stuff than that. People can hide huge files that your ordinary website user doesn’t see but the scraper collects it and it blows up. Seems legal to me- it’s only going to blow up if you’ve been scraping our website.

DrBlackbird · 03/05/2024 13:06

Codlingmoths · 03/05/2024 11:31

There is smarter antiscraping stuff than that. People can hide huge files that your ordinary website user doesn’t see but the scraper collects it and it blows up. Seems legal to me- it’s only going to blow up if you’ve been scraping our website.

So could MN implement something like this? Relatively easily and inexpensive? If so, I for one would be reassured to hear of this being done.

Codlingmoths · 03/05/2024 14:10

That’s on the extreme end probably to be fair. There would be a suite of common tools which primarily detect scraping. I don’t know how effective these are considered.

ifIwerenotanandroid · 03/05/2024 15:03

Codlingmoths · 03/05/2024 14:10

That’s on the extreme end probably to be fair. There would be a suite of common tools which primarily detect scraping. I don’t know how effective these are considered.

'There is smarter antiscraping stuff than that. People can hide huge files that your ordinary website user doesn’t see but the scraper collects it and it blows up. Seems legal to me- it’s only going to blow up if you’ve been scraping our website.'

Sometimes you've just go to take off & nuke it from orbit. It's the only way to be sure.🚀💥

Talulahalula · 06/05/2024 07:48

Just bumping.

Please create an account

To comment on this thread you need to create a Mumsnet account.

This thread is not accepting new messages.