IMO this intuitively feels very wrong, wondering what other people here think? I'm not sure how much of my reasoning about this is just changing norms or something, but LLMs pretending to be human...
IMO this intuitively feels very wrong, wondering what other people here think? I'm not sure how much of my reasoning about this is just changing norms or something, but LLMs pretending to be human always feels like it's crossing a boundary, and the designers are clearly trying to make it as indistinguishable as possible, not just ambiguous like the chatbots are (where the tells are still fairly obvious). Also, the implied privacy of a texting conversation which is actually being monitored, while obviously useful for the intended purpose, seems not great to me in terms of violating the implicit trust in normal conversations.
I do still think that on balance LLMs (not necessarily this kind of implementation, something running on-device would be ideal) are probably better than not having a counsellor, but obviously if this became popular it would encourage lowering funding to actual human counsellors, further exacerbating the issue.
I can't read the article, but I don't think it's a good replacement, or even a passable one. I've seen too many chat bots gone bad. A contracted virtual service would be the "last resort" sort of...
I can't read the article, but I don't think it's a good replacement, or even a passable one. I've seen too many chat bots gone bad. A contracted virtual service would be the "last resort" sort of option.
I don't do clinical work, and it's not that a machine or even a website can't share the resources I give to students. It's that I'm good at getting them to use the resources, at recognizing they're not ok when they say they're fine, etc. I've tested out mental health bots, and they're ultimately unsatisfying Imo
Having read it, my feelings don't really change. They're not counselors monitoring those chats, they're "people with backgrounds in" those fields. And if there isn't a school counselor what are...
Having read it, my feelings don't really change. They're not counselors monitoring those chats, they're "people with backgrounds in" those fields. And if there isn't a school counselor what are the odds external counseling is actually available/achievable for these students. The areas that struggle with one tend to struggle with the other in my experience.
And, they figured out kids need more emotional support before tests? Who could have guessed. ಠ_ಠ. At the university level we explicitly do destress programs at midterms and finals, and have been for a while.
Gaps this large just shouldn't exist and shouldn't need to be filled in. I suppose if you have no other option it's better than nothing. But that's not the same as good. Actual contracted virtual "talk to a human" services would be better. More expensive, sure, but better.
Exactly. I'm inclined to go a step further, but first let me back up and say this frightened and angered me. I've cooled down and can say at least two reasons why. It tells our children by example...
Gaps this large just shouldn't exist and shouldn't need to be filled in.
Exactly. I'm inclined to go a step further, but first let me back up and say this frightened and angered me. I've cooled down and can say at least two reasons why.
It tells our children by example that they don't matter. They don't get the attention/love/connection/help from another human. They plug text in and literally "the machine" hands them an answer. The problem (IMO) most likely stems from some lack of connection or communication and instead of providing what they need, the mechanisms of the modern era try to profit on it while flagrantly sidestepping it with unknown consequences. It's literally, "if there's no way to profit on your problem, there's no way to fix it" It's the height of cynicism.
It gives the appearance of a solution. It's a red herring. It's another attempt at systemic parenting for commerce's benefit. And I think it might make it worse. I hope I'm wrong though because there's already VCs and deployment, so the cat's out of the bag and this is going to have a go at our kids now.
This is a conversation that should have happened publicly years ago - if we lived in a functional democracy.
This completely removes the emotional aspect of psychotherapy. Psychotherapy is not just someone trying to come up with the best answers. It is a human being showing an interest in the well being...
This completely removes the emotional aspect of psychotherapy. Psychotherapy is not just someone trying to come up with the best answers. It is a human being showing an interest in the well being of another. I am not sure how important that is in other countries, but that is an important aspect of psychotherapy in Brazil.
There's a reason why reading books about psychology can't replace psychotherapy.
It's very important but I do want to highlight that school counselors aren't even always doing psychotherapy. There are other levels of mental health and basic needs support and services - and I'd...
It's very important but I do want to highlight that school counselors aren't even always doing psychotherapy. There are other levels of mental health and basic needs support and services - and I'd indeed suggest the vast majority need a human connection. But I don't do therapy, despite being trained to do so, and still work with students in crisis and in need of support.
I understand. I dint think we have many other than "psychotherapy" as a designation. With the exception of psychoanalysis that is it's own thing, and "coaching" that is a more recent thing and the...
I understand. I dint think we have many other than "psychotherapy" as a designation. With the exception of psychoanalysis that is it's own thing, and "coaching" that is a more recent thing and the target of lots of memes.
Yeah and that's different too, but you can do non-clinical work (advocacy and support, connecting with resources and helping people navigate it for example) as well Coaching gets questionable due...
Yeah and that's different too, but you can do non-clinical work (advocacy and support, connecting with resources and helping people navigate it for example) as well
Coaching gets questionable due to a lack of standards or acting outside their scope. But the concept of a support person is the same
I don’t believe this. There is no way this data won’t be mishandled and abused as soon as they think they can get away with it. It’s not “a judgment-free zone” — it’s a private data collection...
Outside of self-harm or violence, Barvir says staffers don’t disclose the content of the exchanges students have with Sonny. If students close their account with Sonny, the company typically retains their data for 60 days. Barvir says students or families can request to delete any chats at any time.
I don’t believe this. There is no way this data won’t be mishandled and abused as soon as they think they can get away with it.
Sharp says the school has seen a 26% drop in student behavior infractions since students began using Sonny. Many students have told her they appreciate having a companion. “They feel it’s a judgment-free zone,” she says.
It’s not “a judgment-free zone” — it’s a private data collection zone, and it’s a goldmine.
I am viscerally disturbed at all levels by this. This ‘benign’ tech is anything but; it is a wolf in sheep’s clothing, and it is lulling children and parents into a false sense of security.
Also: Were I a newly minted liaison of the WHITE HOUSE FAITH OFFICE looking to justify my existence as an indispensable member of the government, surely I wouldn’t be tempted to float the idea of exploiting this system as a means of quietly monitoring children for early signs of deviation from what I might deem the Christian nationalist norm? Because that would be invasive, lol.
New troubled teen/conversion camp industry boom, anyone?
Even if the company were completely honest about their data collection policies, eventually they'd be forced by legislators or replaced by a company willing to do it, since since all that data...
Even if the company were completely honest about their data collection policies, eventually they'd be forced by legislators or replaced by a company willing to do it, since since all that data would be invaluable for "improving educational outcomes". From there, they'd start selling it, and that's okay because all the money is going to the severely underfunded school systems. And once we're doing that, is it really that much difference to let the advertisers use it? It's another valuable income stream for the school.
We end up at a place were sad kids are told by the friendly chatbot that a can of Red Bull is what they need so they'll have the energy to crush those negative feelings, where every step on the way was ostensibly for the children.
New troubled teen/conversion camp industry boom, anyone?
Given the way things are going, probably, but as work camps with all the same old abuse added on top.
I think the fact that so many young people already use LLMs through services like character.ai or just "vanilla" chatGPT to already do something like this devalues the service (though the issues...
I think the fact that so many young people already use LLMs through services like character.ai or just "vanilla" chatGPT to already do something like this devalues the service (though the issues are just as bad, maybe worse in some ways for going through 3rd party websites due to not being tied into safety systems, lack of oversight)
This is better than those because the human would (I assume) stop the bot from encouraging self harm or suicide. But the lack of value comes from not having it all be people. It would genuinely be...
This is better than those because the human would (I assume) stop the bot from encouraging self harm or suicide. But the lack of value comes from not having it all be people.
It would genuinely be better to have your 2 overworked school counselors (and not have scheduling tests be part of their job) and a handful of bachelors level social work/psych degrees (who are probably about what you're getting here, if that) physically be in person at the schools. Bachelor levels can't do a ton in those fields because the advanced degrees make a difference, but I can train a BA to do the basics and refer to the professionals when needed. I wouldn't want them monitoring a dozen chats instead of focusing on one student at a time
IMO this intuitively feels very wrong, wondering what other people here think? I'm not sure how much of my reasoning about this is just changing norms or something, but LLMs pretending to be human always feels like it's crossing a boundary, and the designers are clearly trying to make it as indistinguishable as possible, not just ambiguous like the chatbots are (where the tells are still fairly obvious). Also, the implied privacy of a texting conversation which is actually being monitored, while obviously useful for the intended purpose, seems not great to me in terms of violating the implicit trust in normal conversations.
I do still think that on balance LLMs (not necessarily this kind of implementation, something running on-device would be ideal) are probably better than not having a counsellor, but obviously if this became popular it would encourage lowering funding to actual human counsellors, further exacerbating the issue.
I can't read the article, but I don't think it's a good replacement, or even a passable one. I've seen too many chat bots gone bad. A contracted virtual service would be the "last resort" sort of option.
I don't do clinical work, and it's not that a machine or even a website can't share the resources I give to students. It's that I'm good at getting them to use the resources, at recognizing they're not ok when they say they're fine, etc. I've tested out mental health bots, and they're ultimately unsatisfying Imo
This is an archive link.
Having read it, my feelings don't really change. They're not counselors monitoring those chats, they're "people with backgrounds in" those fields. And if there isn't a school counselor what are the odds external counseling is actually available/achievable for these students. The areas that struggle with one tend to struggle with the other in my experience.
And, they figured out kids need more emotional support before tests? Who could have guessed. ಠ_ಠ. At the university level we explicitly do destress programs at midterms and finals, and have been for a while.
Gaps this large just shouldn't exist and shouldn't need to be filled in. I suppose if you have no other option it's better than nothing. But that's not the same as good. Actual contracted virtual "talk to a human" services would be better. More expensive, sure, but better.
Exactly. I'm inclined to go a step further, but first let me back up and say this frightened and angered me. I've cooled down and can say at least two reasons why.
It tells our children by example that they don't matter. They don't get the attention/love/connection/help from another human. They plug text in and literally "the machine" hands them an answer. The problem (IMO) most likely stems from some lack of connection or communication and instead of providing what they need, the mechanisms of the modern era try to profit on it while flagrantly sidestepping it with unknown consequences. It's literally, "if there's no way to profit on your problem, there's no way to fix it" It's the height of cynicism.
It gives the appearance of a solution. It's a red herring. It's another attempt at systemic parenting for commerce's benefit. And I think it might make it worse. I hope I'm wrong though because there's already VCs and deployment, so the cat's out of the bag and this is going to have a go at our kids now.
This is a conversation that should have happened publicly years ago - if we lived in a functional democracy.
Okay. I'm taking a time out.
Oh, nice, they're getting prepped for what will amount to healthcare for the rest of their life.
This completely removes the emotional aspect of psychotherapy. Psychotherapy is not just someone trying to come up with the best answers. It is a human being showing an interest in the well being of another. I am not sure how important that is in other countries, but that is an important aspect of psychotherapy in Brazil.
There's a reason why reading books about psychology can't replace psychotherapy.
It's very important but I do want to highlight that school counselors aren't even always doing psychotherapy. There are other levels of mental health and basic needs support and services - and I'd indeed suggest the vast majority need a human connection. But I don't do therapy, despite being trained to do so, and still work with students in crisis and in need of support.
I understand. I dint think we have many other than "psychotherapy" as a designation. With the exception of psychoanalysis that is it's own thing, and "coaching" that is a more recent thing and the target of lots of memes.
Yeah and that's different too, but you can do non-clinical work (advocacy and support, connecting with resources and helping people navigate it for example) as well
Coaching gets questionable due to a lack of standards or acting outside their scope. But the concept of a support person is the same
I don’t believe this. There is no way this data won’t be mishandled and abused as soon as they think they can get away with it.
It’s not “a judgment-free zone” — it’s a private data collection zone, and it’s a goldmine.
I am viscerally disturbed at all levels by this. This ‘benign’ tech is anything but; it is a wolf in sheep’s clothing, and it is lulling children and parents into a false sense of security.
Also: Were I a newly minted liaison of the WHITE HOUSE FAITH OFFICE looking to justify my existence as an indispensable member of the government, surely I wouldn’t be tempted to float the idea of exploiting this system as a means of quietly monitoring children for early signs of deviation from what I might deem the Christian nationalist norm? Because that would be invasive, lol.
New troubled teen/conversion camp industry boom, anyone?
Even if the company were completely honest about their data collection policies, eventually they'd be forced by legislators or replaced by a company willing to do it, since since all that data would be invaluable for "improving educational outcomes". From there, they'd start selling it, and that's okay because all the money is going to the severely underfunded school systems. And once we're doing that, is it really that much difference to let the advertisers use it? It's another valuable income stream for the school.
We end up at a place were sad kids are told by the friendly chatbot that a can of Red Bull is what they need so they'll have the energy to crush those negative feelings, where every step on the way was ostensibly for the children.
Given the way things are going, probably, but as work camps with all the same old abuse added on top.
Mirror: https://archive.is/0WrBg
I think the fact that so many young people already use LLMs through services like character.ai or just "vanilla" chatGPT to already do something like this devalues the service (though the issues are just as bad, maybe worse in some ways for going through 3rd party websites due to not being tied into safety systems, lack of oversight)
This is better than those because the human would (I assume) stop the bot from encouraging self harm or suicide. But the lack of value comes from not having it all be people.
It would genuinely be better to have your 2 overworked school counselors (and not have scheduling tests be part of their job) and a handful of bachelors level social work/psych degrees (who are probably about what you're getting here, if that) physically be in person at the schools. Bachelor levels can't do a ton in those fields because the advanced degrees make a difference, but I can train a BA to do the basics and refer to the professionals when needed. I wouldn't want them monitoring a dozen chats instead of focusing on one student at a time