SS and AI. What do we think of this then?

[MistressoftheDarkSide]

https://www.theguardian.com/society/2024/sep/28/social-workers-england-ai-system-magic-notes

Personally I'm a self confessed dinosaur so have huge reservations about the speedy proliferation of AI in every aspect if our lives, not so much due to the technology itself but due to human nature and a growing lack of faith in it as it has been shaped by technology in this brave new world.

So I'd be interested to hear what people think about this innovation.

The problem with the use of AI-like tools (LLMs like ChatGPT aren't really AI, they're an AI simulation - more like "predictive text on steroids", really) isn't their existence, but rather being put to inappropriate uses.

When they're used to save time by people who are already skilled enough to do the job in the first place, or supervised heavily by humans, they're brilliant.

The problem comes when they're used to replace humans in an interaction chain...and that seems to be their purpose as interpreted by management just about everywhere.

In the context of the article, it strikes me that the main problem is when it misses critical information - which it absolutely will do. Unlike a deterministic system, where there would be an audit trail allowing somebody to fix the bug that led to the omission, the "AI" in question will simply be a black box. They can try to influence its behaviour, but there are absolutely no guarantees about how it will behave in future.

It's a terrible idea, but one which will be forced into existence everywhere by people who have no business making those decisions due to a comprehensive lack of knowledge and wilful ignorance of facts in favour of marketing potential.

Agreed.

The article ended with that.
"Maris Stratulis, the national director of the British Association of Social Workers England, said: “We call for the regulation of AI, a national framework of ethical principles for its use and transparency about when AI applications are being used and by what industries to ensure accountability to citizens and uphold human rights"

I think run the pilots and implement the above. Labour is already in the process of drawing up legislation, it should have been done a year ago, we knew this was coming.

However, we should only be investing in this if we have the information sharing technology that we should have got in 2012. As long as a seperate budget is created, then it could a good thing. I'd like to move towards personal responsibility in SW/management and this tool would be needed for that. Those that ignored Star Hobson and Arthur should be held accountable, as well as other cases. We've just had numerous police officers charged and sacked for not following procedures and/or doing their job. I think it needs extending to other professions.

Right so I am in tech. AI used in this capacity saves so much time and accurate notes on things like Teams or Zoom calls. It literally takes notes and can draft emails etc

AI has been used for years already, detecting bank fraud etc I mean, predictive text on your phone is.

https://www.theguardian.com/australia-news/2024/sep/26/victoria-child-protection-chat-gpt-ban-ovic-report-ntwnfb

And meanwhile in Australia.

Rabbit holes.

Isn’t the AI irrelevant there? Anything generated should have been checked and edited. That’s more on a par with crap notes or a report that misses the point. These do happen and obviously shouldn’t

I use AI to write clinical letters, it's fabulous. All I need to do is to edit in any identifiable information offline. It saves me masses of time and I can see more patients instead of drowning in paperwork.
I also use it to write clinical meeting summaries.
Scanning for missing information and mistakes is a lot faster than writing everything from scratch.
If they (the trust) do adopt something which is secure and accurate, I am comfortable outsourcing admin tasks to AI

You're describing machine learning, not AI. They're not the same thing.

The issue with using it in this context is that there are legal and real-world consequences for inaccuracy; who's going to be responsible, the first time a child dies from abuse that wasn't flagged because the "AI" tools missed some key details in an interview?

And then the next step is inevitably taken - "Hey, we need more social workers but there just aren't enough candidates...but also hey, we can hire less-experienced workers and they can just lean on the AI tools a bit more, right?".

This is what I mean by inappropriate use of such tools. They're not a magic wand, but they will absolutely be treated as such because - as a wise man once said - any sufficiently advanced technology will appear as magic to the uneducated. That's why we have this belief in the population that AI can do anything, when anybody who's worked with it (as I have, extensively) knows that it absolutely can't.

Yeah I came back to say that used as described above it saves so much time

"Magic Notes tool records and analyses face-to-face meetings and suggests follow-up actions"

I think this is the bit that needs examination. I had a thread about proposals to record SS meetings with clients and SWs absolutely balked at the idea. Having been involved with SS in the past, I would have welcomed real time recording similar to the police as often a report would come back that bore little resemblance to the actual meeting. I thought that official recording would minimise the he said / she said issues on both sides.

So recording - yes, we have the technology. Analysing with AI? Not at all keen.

It's worrying that the social services are so underfunded they need to resort to this, although I think there can be positives.

As an extra on top it can be useful. For example, DH works for a computer game company and their community managers have been using AI to analyse threads on the fan forums, subreddits etc to find out what kinds of problems, suggestions, complaints, confusion etc people have about the game and they can then use this to inform what changes they make. Previously the community managers would read and get involved in some fan forums but it would be more than a full time job to read every single post ever made about the game. Whereas the AI can do that 24/7. So as well as the actual human involvement in the community, the company has the info of what discussions are going on as well.

So for example, for social work, if a human is minuting the meeting but then the AI looks at the transcript and makes its own summary and this is compared to the human-made minutes, that makes sense because it's extra on top, and the AI might pick up on things that the human missed.

Whereas if the AI is replacing the human minuting then you don't know if it's the one missing things. Also, I would be concerned in sensitive areas like social work that AI tends to perpetuate human bias - whereas (you would hope) actual social workers are more trained and experienced to see the nuances of a particular situation. Social work has an overrepresentation of families who are low income, BAME, single mothers, mental health difficulties, low education etc. It is crucial that they are treated fairly and not held up to classist, racist, sexist, ableist norms. (Of course, I know this doesn't always happen anyway. But good social workers ARE aware of these biases and work hard to counteract them - AI would not necessarily have that lean.)

I don't see a problem with AI drafting letters - this tends to be good - as long as a human checks it for accuracy and mistakes particularly with the most important points. Different LLMs do better and worse at this. GMail has just added one and the summaries it gives me of email chains are usually inaccurate - it's not useful at all.

To me it's more of an indication that social services are underfunded as said - it doesn't matter in the slightest if a video game makes improvements which don't please all of the players. Whereas mistakes in social work affect people's real lives deeply and can cause or prevent trauma or even death. We should be funding them adequately so that humans can do the job well, rather than needing AI to do it (though maybe - just maybe - it would help in the sense that one big complaint of jobs like social workers, teachers, police over recent years is that they have to spend more time on paperwork than actually working with the people they want to make a difference with?)

There is already a lack of accountability (and dare I say arse covering corruption in some cases) in these underfunded services so I am uneasy about the "just following computer orders" that may also creep in.

Sure, fully evaluate and carefully delimit the use of AI tools in such a sensitive and high-impact area, with ongoing checks.

But do similar evaluations on real-world human efforts, too (i.e. what they produce day to day, under all the pressures they're usually subject to, and preferably without the knowledge that they're being assessed, if possible). The absolute shite I've seen people in SW, MH and similar sectors write up on occasion, it's clear they barely listened, let alone understood to the extent they could write an accurate and useful summary. And that's on the occasions they bothered to write anything.

Of course, that's the opposite problem - I've always said that if your job can be replaced by an AI, then you can't have been that good at the job in the first place.

If any AI can do the job better than any given human, then the recruitment practices really need to be revised.

If it's used with guidelines and boundaries and stops social workers (and other professionals) having to spend so much time sitting at a desk often duplicating the same information across multiple platforms (you wouldn't believe how awful a lot of the systems are and how many forms you have to fill for various things) and frees up time for professionals to spend more time with families, or with vulnerable or dangerous people where they will gather more information than they can be, pick up on things because they've got an hour to spend not twenty minutes, build more effective relationships to keep people engaged with services and interventions I think it can be a good thing.

AI has its uses, for sure, but it comes with potential problems. My employer uses expert IT contractors to write original material for very sensitive projects precisely because their expertise is necessary. Their contract specifies no AI. Recently, one contractor used AI to produce stuff, and as a result, the whole project had to be paused, the contractor was dismissed (and will never work for my employer again), and because some of the sensitive material has now been fed into the AI corpus, the whole project may have to be abandoned, at enormous cost in time and money.

As IT specialists, we can identify AI produced material very easily, so the contractor was monumentally stupid as well as lazy and dishonest.

Are there any privacy concerns with this? I'd assume the AI is collecting data. I wouldn't assume all the data it collects can be anonymised.

It's unlikely the data collection is an intrinsic part of the AI - there may be additional functionality to do that (for quality purposes), but that's entirely controllable by the provider, and thus can be prohibited or controlled in the contract as it would with any third-party system provider.

It's a trial. You need to see if the system works or not. AI tools exist, you can't ignore them. You can test them and see if they are suitable. Which is what's happening here. Isn't it better than brining them in without testing? Or missing out on a potentially better system?

Agreed.

Re: When they're used to save time by people who are already skilled enough to do the job in the first place

That is one of the main problems currently with SS. Underfunding and insufficient lack of accountability means some are poorly run and with badly trained and/or staff simply not up to, or inappropriate for, the job.

Also, I would be concerned in sensitive areas like social work that AI tends to perpetuate human bias - whereas (you would hope) actual social workers are more trained and experienced to see the nuances of a particular situation. Social work has an overrepresentation of families who are low income, BAME, single mothers, mental health difficulties, low education

Ironically this is a both-ways issue - the lack of nuances, and reliance on stereotypes. It leads to those not considered "the usual" type in need of help often left without support. Happened to someone I know. On the surface middle class etc but because of DV no money of her own. Many families (and individuals) like her are often in hidden or ignored need - and sometimes not helped appropriately or at all by SS due to bias (conscious or subconscious).

Agree the work needs a human - with the ability to understand nuance and complex situations. That said, currently, that's often not the case. My friend described the people she encountered at SS as talking and behaving like robots - with rigid and narrow mindsets and a lack of ability to understand nuance, or complex situations or needs. Heavily reliant on stereotypes, and lacking essential qualities for SS - empathy and compassion.

There's a need for better - and better trained - human staff.

Separately:
The problem comes when they're used to replace humans in an interaction chain
At a time when there's complaints about "worklessness" and the "need to get people off benefits", how will people find jobs if they're being replaced by AI? This issue actually is related in some ways to the work of SS. Because poverty and unemployment increases the risks of someone falling into struggles that eventually need support from social services.

AI is a game changer in my profession. It takes the heavy work away and allows more time for the human touch where it is needed - at the client (or customer/ patient/ consumer) level. It speeds up transactions and increases profits.

But rely on it, without the human touch and it is a business disaster.

I think an important part of the note taking and reviewing is just that - an opportunity to review the information and notes. Although the actual meetings with clients is important, so is spending time reviewing and writing notes. It helps the social worker think through the situation and focus on something they might otherwise miss or overlook or misunderstand.

Already a problem with some SS departments is outsourcing initial contact to call centres with untrained customer service staff doing the vital initial contact.

The job needs to be holistic and in-house.

@ntmdino ML is AI.

As long as the data used is protected within the enterprise/organisation boundaries database or CMDB and fact checking is done I don't see the issue.

It's saves me a lot of time. Efficiency gains are really important in specific jobs with high workloads.

Yes, absolutely - these things should always be run in limited trials. However, my issue is with:

a) the criteria used to evaluate it
b) who set those criteria
c) who will be measuring the system against those criteria, and
d) who will be making the final go/no-go decision

Remember, this is civil service. They've outsourced the system itself, meaning that they don't have the requisite skills to build one or manage the build of one, so it's a reasonable conclusion that they probably also lack the skills with which to evaluate one (AI evaluation is a really specialised area, which requires deep knowledge of the subject matter).

I can also almost guarantee that (b) will be a contractor, (c) will be somebody in-house who's done some form of 2-3 day course related to AI, and (d) will be a committee of upper-tier management whose only expertise in the area will be what the salesdroids for the system have told them, and will probably inappropriately factor in cost savings at some point along the line.

I can't see this going well at all.