24
votes
Could a language learning model talk to whales? Or a human who speaks a language besides English?
The New Yorker has a provocative article asking the question "Can We Talk To Whales?" It boils down to utilizing language learning models to process a dataset of sperm whale clicks, their codas, and crossing one's fingers to see if "ClickGPT" can produce actual sperm whale language.
Which makes me wonder if a language learning model been given a library of Chinese sounds and ideograms, without context, then communicated in workable Chinese?
Using a language learning model to learn to speak to whales is an interesting idea, but I'm thinking any LLM assigned the task will wind up chunking out a word salad or something akin to Prisencolinensinainciusol. I'd like to learn more.
I think the tricky part is that there's no way to double check. We can see if ChatGPT is making sense in English or another human language.
But if it spits out something in whale, we basically have to shrug and say "yeah I guess that sounds like whale to me."
That wouldn’t be the same as “speaking their language”. A dog is trained to respond in a certain way to a certain command. But you could easily swap out that command for another and train them again to do the same thing. It’s not like you could yell that same command to any other dog and have them do the exact same trick without training. So the question remains: how would you verify you’re really producing whale clicks?
You'd also be unable to tell the difference between a bad translation and a stubborn whale who doesn't like getting ordered around.
Which I find a hilarious thought. It would be funny if they turn out to be total jerks.
Or just like my favorite Far Side comic.
It's the same as with any machine learning task. You reserve some of your data as validation and test sets, train the LLM on the rest of the corpus, and then compare its predictions to the data it hasn't seen during training.
But "making sense" isn't that useful if you don't know what it actually means.
I'm pretty sure that's how they learn all languages? LLMs don't know what they're saying. A large part of how they function is essentially just predicting what word their training data would be most likely to use next. Obviously there's more to it but that's the gist. It's not like they understand context.
The difference being for LLMs we have a known set of symbols for the languages we teach them rather than just sounds. We have audio models of language as well of course but they are vastly weaker.
Don't confuse the training method with the result. Yes, an LLM is just a prediction machine, but in order to predict the next word in English and do so with any reasonable accuracy you have to build a working understanding of the concepts that exist in the English language.
LLMs are by no means perfect, but if you're going to insist that they don't understand what they're talking about it all you're probably going to be sorely mistaken
Yes, but to train AI you feed it questions and answers. So to make it understand language, we must first understand the language.
I’m not sure what you mean by that, can you elaborate?
Scanning the New Yorker article, I don't see any technical details of how they intend to do this, just handwavy "chatGPT will do it". I can't rule out that they have more details in there, but I'm not hopeful.
The main problem you're going to run into is "aligning" whalish and english. If you train a LLM on english, and one on whalish, you're going to give it no notion for when it's actually saying the same thing. Just using plain next-token prediction you can't really do that. You'd need some "translated" data, in which case I'd hazard the guess that a small set of translated data would already help greatly align much larger sets of unannotated data.
So how do we get translated data? If you were to track whales and annotate their utterances with what's going on - what things are nearby, which other whales are nearby, what is everyone doing? - then you have some data that potentially has a mapping to english concepts and you can actually do something here.
The glib answer is that the procedure described in the article won't work. LLMs are not trained by throwing a gigantic corpus at a machine learning model and walking away; there's a great deal of additional manual training required to get the model to produce coherent outputs. Unless you have some entity which speaks fluent whale to refine the model with, you're out of luck.
The slightly less glib answer is that no non-human animal speaks (or otherwise conveys) "language". Anyone remember this nugget from a few years ago? The reply is being kind of a dick, but is also right: the original comment is self-refuting. Non-human animals make associations with specific sounds, which they can then use to communicate with each other, but there's little to no possibility for abstraction or composition. There's a credible argument to be made that Koko the gorilla was the most intelligent (at least linguistically) non-human animal in history (there's also a credible argument that her intelligence was wildly overestimated, but that's a different topic). She was less linguistically capable than an average human two-year-old. So: we don't need a whale language model, that's not even coherent. At the absolute most, we need a whale dictionary (and realistically, it'd be more of a pamphlet). An LLM does absolutely nothing to help with that.
In terms of non-English human languages? Yes, there's no reason to believe LLMs are not technically perfectly capable of being trained on them (though because they're trained on extremely large volumes of written text, there likely are languages for which not enough training data exists). The limitations there are economic: training an LLM is extremely expensive, and English has by a very wide margin the most speakers on the planet who can provide a return on that expense.