Sounds great, just as long as LLMs aren't centralised in the control of a small number of corporations with similar ideological stances and designs on effecting political change. As long as that...
Sounds great, just as long as LLMs aren't centralised in the control of a small number of corporations with similar ideological stances and designs on effecting political change. As long as that doesn't happen I'm sure we'll all be fine trusting the machines the summarise things without bias.
There are models with that label but it's pretty much just a functionless label. Training an LLM is beyond Average Joe, and he who controls the training is the only person who controls the source....
There are models with that label but it's pretty much just a functionless label. Training an LLM is beyond Average Joe, and he who controls the training is the only person who controls the source. It can be massaged in any number of ways, and it's not like you can do an independent or reproducible build.
I don't agree that the label is meaningless. Inspecting source code for vulnerabilities is also beyond the Average Joe, so we rely on trusted groups. The same model would work fine for LLMs. You...
I don't agree that the label is meaningless. Inspecting source code for vulnerabilities is also beyond the Average Joe, so we rely on trusted groups. The same model would work fine for LLMs.
You also have European AI companies like Mistral, who would not share the ideological stance of American AI companies and might help counterbalance it.
Agreed on Mistral, but it is worth pointing out that these are entirely different flavors of Joe we're talking about. There are many Unaverage Joes who theoretically can inspect source code, and...
Agreed on Mistral, but it is worth pointing out that these are entirely different flavors of Joe we're talking about. There are many Unaverage Joes who theoretically can inspect source code, and we even have some (limited, inadequate) infrastructure around this.
There are no Joes who can authenticate that LLMs aren't hiding secrets. And the only Joes that can build an LLM (let alone rebuild someone else's) from source are less Average Joe and more Unholy Corporate Amalgamation Joe. Like you say, competing UCA Joes provide something of a counterbalance in that there are options, but it's quite limited.
I'm not sure specifically what the threat model you're thinking of is, but I'm assuming it's a question of deliberately introduced bias or misinformation? If that's the case, I'd say the ability...
I'm not sure specifically what the threat model you're thinking of is, but I'm assuming it's a question of deliberately introduced bias or misinformation?
If that's the case, I'd say the ability to audit the resulting weights (which we can already do for open LLMs) would probably be more valuable than the ability to see and reproduce precisely how it was trained from scratch. It's actually pretty tough to predict exactly what the knock-on results of a given training change will be - that's one of the reasons it's still a field in active development - but very plausible to construct broad test suites at various different levels to help uncover issues hidden within the weights.
Like most people who work with words for a living, I’ve watched the rise of large-language models with a combination of fascination and horror, and it makes my skin crawl to imagine one of them writing on my behalf. But there is, I confess, something seductive about the idea of letting A.I. read for me — considering how cruelly the internet-era explosion of digitized text now mocks nonfiction writers with access to more voluminous sources on any given subject than we can possibly process. This is true not just of present-day subjects but past ones as well: Any history buff knows that a few hours of searching online, amid the tens of millions of books digitized by Google, the endless trove of academic papers available on JSTOR, the newspaper databases that let you keyword-search hundreds of publications on any given day in history, can cough up months’ or even years’ worth of reading material. It’s impossible to read it all, but once you know it exists, it feels irresponsible not to read it.
What if you could entrust most of that reading to someone else … or something else?
…
There are a handful of scholars who are beginning to more formally incorporate the use of L.L.M.s into their work. One of them is Mark Humphries, a professor at Wilfrid Laurier University in Ontario, whose research projects involve enormous stores of digitized records from Canadian history. In one project, he and his students used A.I. to analyze tens of thousands of handwritten records about late 18th- and early 19th-century fur trading, in order to better understand the far-flung community of traders (now known collectively as “voyageurs”) who, with their families, explored and later settled much of what would eventually become Canada. “If you can pass those records to a large-language model and you can say, ‘Tell me who Alexander Henry’s trading partners were,’” Humphries said, “the neat thing is that it’s able to very quickly go through and not just search by name but do cross-referencing, where you’re able to find relationships.”
The goal is to find not just one-to-one transactions between specific voyageurs but chains of interconnection that would be hard for human researchers to make quickly. “It takes an A.I. 20 seconds,” Humphries said. “It might take a graduate student doing the same work weeks.”
…
The rise of computers and the internet were of course an unprecedented turning point in the history of tools for writing history — exponentially increasing the quantity of information about the past and, at the same time, our power to sift and search that information. Psychologically, digital texts and tools have thrown us into an era, above all, of “availability”: both in the colloquial sense of that word (everything’s seemingly available) and in the social-scientific sense of “availability bias,” whereby we can fool ourselves into thinking that we have a clear and complete picture of a topic, buffaloed by the sheer quantity of supporting facts that can spring up with a single, motivated search.
Even among academic historians, this availability has shifted incentives in a direction that A.I. is likely to push even further. In 2016, years before the L.L.M. explosion, the University of Pittsburgh historian Lara Putnam published an essay about the achievements but also the dangers of search-driven digital research. “For the first time, historians can find without knowing where to look,” she wrote, in a particularly trenchant paragraph. “Technology has exploded the scope and speed of discovery. But our ability to read accurately the sources we find, and evaluate their significance, cannot magically accelerate apace. The more far-flung the locales linked through our discoveries, the less consistent our contextual knowledge. The place-specific learning that historical research in a predigital world required is no longer baked into the process. We make rookie mistakes.”
…
When she and I chatted recently, Putnam compared this shift to Baumol’s cost disease — the phenomenon, noted by the economist William Baumol, that when technology makes certain workers more efficient, it winds up making other forms of labor more expensive and therefore harder to justify. In principle, Putnam notes, digital tools have no downside: Professional historians remain more than capable of carrying out time-consuming research in physical archives. But in practice, the different, faster, more connective kind of research was making the more traditional work seem too professionally “expensive” by comparison. Why spend a month camped out in some dusty repository, not knowing for sure that anything publishable will even turn up, when instead you can follow real, powerful intellectual trails through the seeming infinitude of sources accessible from the comfort of home?
Sounds great, just as long as LLMs aren't centralised in the control of a small number of corporations with similar ideological stances and designs on effecting political change. As long as that doesn't happen I'm sure we'll all be fine trusting the machines the summarise things without bias.
Plenty of open-source LLMs to switch to if it becomes a problem.
There are models with that label but it's pretty much just a functionless label. Training an LLM is beyond Average Joe, and he who controls the training is the only person who controls the source. It can be massaged in any number of ways, and it's not like you can do an independent or reproducible build.
I don't agree that the label is meaningless. Inspecting source code for vulnerabilities is also beyond the Average Joe, so we rely on trusted groups. The same model would work fine for LLMs.
You also have European AI companies like Mistral, who would not share the ideological stance of American AI companies and might help counterbalance it.
Agreed on Mistral, but it is worth pointing out that these are entirely different flavors of Joe we're talking about. There are many Unaverage Joes who theoretically can inspect source code, and we even have some (limited, inadequate) infrastructure around this.
There are no Joes who can authenticate that LLMs aren't hiding secrets. And the only Joes that can build an LLM (let alone rebuild someone else's) from source are less Average Joe and more Unholy Corporate Amalgamation Joe. Like you say, competing UCA Joes provide something of a counterbalance in that there are options, but it's quite limited.
I'm not sure specifically what the threat model you're thinking of is, but I'm assuming it's a question of deliberately introduced bias or misinformation?
If that's the case, I'd say the ability to audit the resulting weights (which we can already do for open LLMs) would probably be more valuable than the ability to see and reproduce precisely how it was trained from scratch. It's actually pretty tough to predict exactly what the knock-on results of a given training change will be - that's one of the reasons it's still a field in active development - but very plausible to construct broad test suites at various different levels to help uncover issues hidden within the weights.
https://archive.is/CWpeu
…
…
…
That's a pretty sick illustration :p