vektor's recent activity
-
Comment on Cory Doctorow | AI companies will fail. We can salvage something from the wreckage. in ~tech
-
Comment on Cory Doctorow | AI companies will fail. We can salvage something from the wreckage. in ~tech
vektor (edited )Link ParentOh, I don't mean to get into the "the LLM said whatever is most likely to result in a thumbs up reaction, because RL finetuning" part. That's not where I tend to see the denial of knowledge, but...Oh, I don't mean to get into the "the LLM said whatever is most likely to result in a thumbs up reaction, because RL finetuning" part. That's not where I tend to see the denial of knowledge, but it is IMO an absolutely crucial caveat to always keep in mind when working with LLMs. If the honest answer is unsatisfactory, e.g. "you're wrong" or "I don't know", LLMs tend to lie with conviction. Though I believe that to be a transient artifact of current training methods. Yes, it's a useful thing for keeping customers engaged, but customer engagement isn't the promise that drives the bubble, it's customer productivity. They sell these services not to people who have time to kill and want to be entertained, at least not mostly. They sell them to people who want to get a job done. This yes-man lying is a problem to them.
The way I most often see this denial of knowledge is when people basically say "it's just a semantic parrot, and just selecting the maximum likelihood token according to its training data. It doesn't know anything". And, from an external perspective, I think that's a unhelpful definition of 'to know'. You can make the same argument about human cognition. We're also next token predictors, slightly biased towards predicting lower-utility token more often (a phenomenon called negativity bias). LLMs, at least without RL finetuning, are at least unbiased estimators, so mechanistically, they have a leg up on us. Knowledge in both cases is an emergent behavior, because at some point you see more complex and complex patterns in the training data, and exploiting those is more efficient. You don't have to build an explicit knowledge engine into the LLM, because us humans didn't get one either. If you have a chat with an LLM and that chat would convince you that a human knows his stuff, then IMO you must be convinced that the LLM knows its stuff as well. Knowledge is, IMO, a functional property. The process doesn't matter, unless it matters in a way that colors the conversation so you are not convinced.
Apologies everyone for the technical terminology, but if I recall correctly, sparksbet should understand.
-
Comment on Cory Doctorow | AI companies will fail. We can salvage something from the wreckage. in ~tech
vektor Link ParentTo be clear, my use of the word nitpicky was more poking at my own argument: "I know Ohm's law because I know V = IR." is using my -very nitpicky- definition of "to know". I know it, and only it,...To be clear, my use of the word nitpicky was more poking at my own argument:
"I know Ohm's law because I know V = IR." is using my -very nitpicky- definition of "to know". I know it, and only it, but I know nothing about what to do with that knowledge. I wouldn't describe that state of mind as "not knowing", but there's a lot more to know about Ohm's law, and you would easily find out in conversation. Hence me splitting up knowledge about a thing into (basically) being able to quote something back, and then transforming that quote into useful new material. I'm not saying "I know electical engineering because I know V=IR".
But to pull that back into something resembling a point about the original topic: I don't need the nitpick, when LLMs can at least sometimes cover both definitions of knowledge. An LLM can clearly use the things it can regurgitate to form useful new material. I can, to condense a complex topic, ask it for Ohm's law, but I can also apply it in a way that I am damn sure no part of its training data covers it. It isn't quoting back someone's homework, it's actually applying that stuff to a new situation. And at that point I am hard pressed to deny that the LLM knows Ohm's Law, insofar as I'd need to resort to an argument of -basically- "it can't know because it is a thinking rock. Only thinking meat can know."
I think part of the issue is our tendency to accept that other humans are like us and therefore they can know things. Then we quiz them about electrical engineering and conclude that they know Ohm's law. Why isn't the same process applicable to a machine? Because they aren't like us, and by the types of mistakes they make that is readily apparent. We excuse human errors as "just human", but the same is not true for machines, because machines don't make errors. Therefore, any error is not a failure to act on knowledge, but a failure to know. And because we can't empathize with a machine and mirror its thought process, we are less convinced by the same amount of evidence.
-
Comment on Cory Doctorow | AI companies will fail. We can salvage something from the wreckage. in ~tech
vektor Link ParentDepends on what exactly this person knows or doesn't know, you can split hairs there all day long. Your person knows the quotes. Easy one. Do they know anything about those topics beyond that?...Depends on what exactly this person knows or doesn't know, you can split hairs there all day long.
Your person knows the quotes. Easy one. Do they know anything about those topics beyond that? Different question. But they definitely know something. They could, for example, know verbatim some simple physics formula, but not know how to apply it. Those are different bits of knowledge. And a person who knows both can have more in-depth conversations that a person who only knows one. I can tell these knowledge states apart by duck testing.
But we don't have to go there and make nitpicky distinctions about knowing a fact and knowing how to apply it: LLMs can usually make some inferences that go beyond the retrieved material. They'll easily regurgitate Ohm's Law, but also know how to apply it. They thus know Ohm's Law. Do they know all the edge cases that a EE prof might know? Dunno. But they might compare favorably to undergrad students.
-
Comment on Cory Doctorow | AI companies will fail. We can salvage something from the wreckage. in ~tech
vektor Link ParentHonestly, the same argument can be applied to the argument of "LLMs can't have knowledge". I'd also apply duck testing here: If it knowledges like a knowledgeable person, it knowledges. If an LLM...Honestly, the same argument can be applied to the argument of "LLMs can't have knowledge". I'd also apply duck testing here: If it knowledges like a knowledgeable person, it knowledges. If an LLM writes as if it knew about a topic, then it knows about that topic. If you can instead lead it to produce inconsistent results (which can be trivial or impossible, depending on the topic), then evidently it doesn't know about that topic well enough - or it doesn't logic very well, leading to false inferences from true knowledge. Which honestly is the more likely culprit. In either case, I'd be on board with denying knowledge.
But to outright say "LLMs can't know things", with an argument that basically boils down to "because it is thinking rocks and not thinking meat" is asinine. Yes, it's just an engine to predict the next token with some RL finetuning on top. Guess what your brain is? It's an engine to predict its next sensory input, and then produce outputs that best shape the next sensory input. There's no reason to assume, judging from the way our brains are created, that we'd know anything. So arguing from the way LLMs are created seems a moot point to me.
</rant> -
Comment on Cory Doctorow | AI companies will fail. We can salvage something from the wreckage. in ~tech
vektor Link ParentHell, I'd imagine if we gave copyright more teeth, not only would media corps get massively more powerful. But we'd not see any functional impact on AI. They'll obfuscate the derivativeness of any...Hell, I'd imagine if we gave copyright more teeth, not only would media corps get massively more powerful. But we'd not see any functional impact on AI. They'll obfuscate the derivativeness of any derivative outputs slightly better, and that'll be that. I don't think openAI will actually throw out all their ill-gotten training data or delete their models.
And if we actually force them to by means of massive and unthinkable enforcement action, i.e. raiding their data centers and wiping all their hard drives, we'd just give the game over to the Chinese or any other nation willing enough to take the lead in the AI game.
My approach would be to go the opposite direction: restrict copyright protection. Your model parameters? Definitely not copyrighted. Model outputs? Yeah, nah. The datasets you're using to train? Fine to use as long as it's just for training. (Personally I'd go complete abolitionist, but that's an admittedly fringe take. ) That levels the playing field: 3 (british) guys in a shed have the same access to data as OpenAI, and we'll make sure that this technology isn't oligopolized by roughly 4 companies. Yes, compute is still a limiting factor, but it's much easier to distribute compute fairly by virtue of free markets. Ask Amazon or one of a bajillion other companies if they'll rent you the compute to train a model - this part of the equation isn't what keeps the big few in power.
-
Comment on US President Donald Trump isn't building a ballroom in ~society
vektor Link ParentA detail that the author doesn't cite, or if they do it's buried in the sources but not referenced: That any of this has anything at all to do with AI. Or autonomous decision making. Or anything...however the author has absolutely done their homework.
A detail that the author doesn't cite, or if they do it's buried in the sources but not referenced: That any of this has anything at all to do with AI. Or autonomous decision making. Or anything like that. Data center? Sure. Quite possible. But AI? I mean, I won't categorically say no, but I see no indication of this being built with AI in mind.
I'm also unconvinced about the insinuation that the Jerusalem data center is doing any kind of AI decision making or targeting in Gaza. For all I know, the data center is just a normal government data center. Holding your tax records, allowing you to file for unemployment benefits, banale things like that. The intelligence branches are involved? Ok, sure, they probably need somewhere to store their satellite imagery.
Not that any of this dimishes the fucked-up-ness of the project by much, even if I feel AI is being invoked as the boogeyman here. Even if Trump just centralized tax records in the White House, that's already a bad sign for the US, but "the military is very much involved" just makes this even worse. I'm thinking this could be the DOGE data grab, except this time under a mantle of a bit more legitimacy, and even less actual accountability, and bigger.
-
Comment on Without looking, do you have a vague idea of your coordinates? in ~talk
vektor Link ParentYou really sure you need elevation? I mean, these things come down nearly vertical, and the margin of error for considering a nuke "properly delivered" is probably large enough. Like, if you're...You really sure you need elevation? I mean, these things come down nearly vertical, and the margin of error for considering a nuke "properly delivered" is probably large enough. Like, if you're unsure if they're at 9km or 0km, you'll probably only miss by 4km at most, which is still good enough.
Beyond that, a 18MB .png in the guidance calculations will probably sort you out a lot better, bringing the elevation uncertainty down to a few hundred meters at most.
(Is this comment unvoluntarily ITAR-restricted now? Who knows!)
-
Comment on GPT-5 has come a long way in mathematics in ~tech
vektor (edited )Link ParentMy understanding is that this tool-calling is usually explicit in the output, though perhaps collapsed-by-default. With the providers I'm using anyway, that was my impression, though of course it...My understanding is that this tool-calling is usually explicit in the output, though perhaps collapsed-by-default. With the providers I'm using anyway, that was my impression, though of course it could be done on the down-low too.
But I'd be almost certain that they're being trained using such tools. Like, simply generating symbolic algebra problems, throwing them into a computer algebra system, and then training a LLM to do the same thing is very low-hanging fruit, but could pay dividends on all kinds of other problems of interest.
-
Comment on GPT-5 has come a long way in mathematics in ~tech
vektor Link ParentI mean, I use these things to shit-test my math ideas before they make it into papers. "Hey, this theorem seems interesting. Can you prove or disprove it?" [A few attempts, all of which the model...I mean, I use these things to shit-test my math ideas before they make it into papers.
"Hey, this theorem seems interesting. Can you prove or disprove it?"
[A few attempts, all of which the model discards itself.]
"Ok, well, here's a line of thinking that could lead somewhere:"
[actually reasonably fleshed out theory].
Seriously. These things are that good.
To be fair, I'm doing the math as part of other studies, so this is applied math territory I suppose.
As always, double check LLM results before you embarrass yourself in peer review. But honestly? The fact that I can (ab)use these models like a research assistant and just throw half-cooked ideas at it and expect a halfway cromulent proof back? Baffling 2 years ago, but we're there.
-
Comment on Donald Trump says nuclear weapons testing to resume in US after more than thirty years in ~society
vektor Link ParentHow reasonable is this step? Is there, like, a genuine, previously suppressed need to do some of this testing for validation purposes, i.e. to validate computer models still fitted with data from...How reasonable is this step? Is there, like, a genuine, previously suppressed need to do some of this testing for validation purposes, i.e. to validate computer models still fitted with data from checks notes the early 90s?
Like, sometimes I have the feeling that Trump has (almost) no original deranged ideas, but sometimes his cronies come with deranged ideas that he feeds into. And sometimes there is some genuine need for a thing, but there's good reasons why we never follow through on that, and Trump just ignores those good reasons.
Basically, I'm imagining some scrawny nerds in the Pentagon that have been like "we'd really like to test this nuke design, but we understand it can't happen, and that's ok. Would be really nice if we could test it tho." for the last 30 years. And now the "I understand it can't happen" just disappeared.
-
Comment on Grieving family uses AI chatbot to cut hospital bill from $195,000 to $33,000 — US family says Claude highlighted duplicative charges, improper coding, and other violations in ~tech
vektor Link ParentAnd taint our training data? No sir. No thank you. We'll keep that exactly as is, for internal purposes. If we ever sell it, we will anonymize it. Obviously I don't work there, but that is the...The developers should sanitize input so that private information is hashed, not stored, or just completely ruled out from being stored at all.
And taint our training data? No sir. No thank you. We'll keep that exactly as is, for internal purposes. If we ever sell it, we will anonymize it.
Obviously I don't work there, but that is the thinking I expect. And honestly? I kinda get it. I've had to work with one too many datasets that were mangled to the point of uselessness by compliance people. And not even for good reasons like patient privacy.
Though to clarify, even if I am a data monkey, I do have a conscience. I would hope the big players purge the worst PII from the data before it moves anywhere, and I think it wouldn't hurt them much to do so. I would be uncomfortable holding that kind of data.
-
Comment on Grieving family uses AI chatbot to cut hospital bill from $195,000 to $33,000 — US family says Claude highlighted duplicative charges, improper coding, and other violations in ~tech
vektor Link ParentDo you think an executive is going to proclaim "we're going to develop an AI tool to stop double billing"? We'll, there's your answer. It's a lot more plausible that they'll loudly proclaim AI for...I wonder why AI is becoming such a big thing in healthcare now, yet despite that, issues like this continue to happen. I don't think double billing is patient-centered care, but maybe I don't have that healthcare=profit mindset.
Do you think an executive is going to proclaim "we're going to develop an AI tool to stop double billing"? We'll, there's your answer.
It's a lot more plausible that they'll loudly proclaim AI for better care. It's also, with a tinfoil hat, plausible that they're quietly using AI for billing, but not with the goal of reducing double billing.
-
Comment on What is your 'Subway Take'? in ~talk
vektor Link ParentMy guess is: The line for "this is too much HOA" is when a house comes pre-packaged with a mandatory, paid membership in the HOA, including quasi-legal rules. Want to have a voluntary paid...My guess is: The line for "this is too much HOA" is when a house comes pre-packaged with a mandatory, paid membership in the HOA, including quasi-legal rules.
Want to have a voluntary paid membership attached to the house to access additional services like the community pool? Go ahead, but keep it voluntary. Want to ensure homeowners keep their homes to a certain standard? That's what local laws are for, and being actual laws the process and enforcement comes with the expected degree of oversight. Want to organize a thing like redoing the greenery around 'ere, or organizing a block party? No need for a HOA there. Want to organize and fund that the trash gets picked up? That's what the municipality is for.
There's no place for this private-public abomination. Parts of its functions are mandatory for the homeowner, but sit squarely in government/public territory. Some are voluntary, and thus private. Is there perhaps place for a level of organization or two here? Like, a subdivision level of local government, with a small amount of bureaucracy to organize the extremely-local issues like trash pickup, code enforcement of petty (aesthetic) issues? Maybe some structure for local private organizing, that has the infrastructure to stand up a block party, or organize a community-driven landscaping makeover? Absolutely. You can even have both those if you insist.
But the moment you commingle public and private issues and either (A) make a public issue a private problem or (B) give a private organization governmental powers, you start to tick off some principled sensibilities in me. I suspect there's a very good political-theory reasons we tend to avoid mixing private and public things.
-
Comment on What ridiculous thing would you spend billions on? in ~talk
vektor LinkR&D budget for procedural generation in video games, as well as development of a AAA game that is ProcGen all the way, in a genre that is generally viewed to mesh poorly with ProcGen Gamers hate...R&D budget for procedural generation in video games, as well as development of a AAA game that is ProcGen all the way, in a genre that is generally viewed to mesh poorly with ProcGen
Gamers hate ProcGen with a passion, so this is almost a guaranteed flop. To ensure I'm not philanthropic, the R&D results are proprietary to this sinking-ship r&d company. The marketing budget for the game is donation funded.
This is 99.999% a complete flop, if you believe that gamers know what they want. Needless to say, I believe they don't, and I'm betting all my monopoly money on it.
If the game sells a million copies at full price, I get any marketing donations back, plus a symbolic two cents because my controversial opinion was right after all. This last clause probably needs approval by the curse. I promise I'm not loaded enough to achieve a reasonable marketing budget by myself. The curse money is all for development.
-
Comment on Forgot Chrome's unusable, any recommendations? in ~tech
vektor Link ParentThe comment isalso about brave, not Firefox. Firefox is a FOSS nonprofit, which sets it apart from most tech companies imo. they're not perfect, but a country mile better than the rest, and most...The comment isalso about brave, not Firefox. Firefox is a FOSS nonprofit, which sets it apart from most tech companies imo. they're not perfect, but a country mile better than the rest, and most of their failures are down to lacking resources.
-
Comment on How many valid JSON strings are there? in ~comp
vektor Link ParentTesting how close to ideal your domain-specific compression algorithm is?Testing how close to ideal your domain-specific compression algorithm is?
-
Comment on What is a business/org that's so terrible no one should use if possible? in ~life
vektor Link ParentMaybe self-payer prices would even be more reasonable if there was no insurance for anyone...Better than nothing at all, but not by much.
Maybe self-payer prices would even be more reasonable if there was no insurance for anyone...
-
Comment on AI industry horrified to face largest copyright class action ever certified in ~tech
vektor (edited )Link ParentIf you ask me, take the price of the item in question. Then multiply with a fudge factor to ensure it discourages piracy. In the case of public transport here, that factor is x3 or x10, roughly,...What would you really consider fair for torrenting a movie or downloading a song? $100 maybe? $500? I think stretching above $1,000 is already getting pretty out of hand. We should be in speeding ticket territory, not losing your home territory. It certainly shouldn’t be more than the fine for shoplifting the same creative work, that’s for sure!
If you ask me, take the price of the item in question. Then multiply with a fudge factor to ensure it discourages piracy. In the case of public transport here, that factor is x3 or x10, roughly, just to give a real world example. That sounds a lot more reasonable than the x10000 implied by the 150k fine.
I also think we might not even have this problem if relevant quantities of these works could be licensed in bulk at reasonable rates.
-
Comment on What useful licenses or certifications are surprisingly cheap and easy to get? in ~talk
vektor Link ParentAlso, bad CPR is more valuable than no CPR. Even if you haven't taken a course, pumping someone's chest until you're sketched out about the integrity of their ribs to the tune of "staying alive"...Also, bad CPR is more valuable than no CPR. Even if you haven't taken a course, pumping someone's chest until you're sketched out about the integrity of their ribs to the tune of "staying alive" is a really simple concept, and it is miles above "no CPR". Everything else is a bonus - proper compression depth, ventilating lungs, those are nice. Just check first whether there's a pulse I guess.
Also, depending on your location, dispatch will stay in the line and coach you through it. Apparently, they even have a metronome to keep you in time and remind you to ventilate. So in case of doubt, just call them, put them on speaker and get pumping.
To clarify once more, duck testing like this isn't necessarily a short and simple thing. Let's stick with Ohm's law for an example. I wouldn't be convinced by simply regurgitating a formula. I wouldn't be convinced by an LLM solving a college level homework exercise. Those are easily in the training data, and I wouldn't be convinced if the answer is plausibly just regurgitated from somewhere. [If the training data were infinite, it might satisfy me because then I don't run the risk of going beyond the training data.] At least if I'm testing for knowledge beyond those particular quotes. Then I'd want to see some understanding that goes beyond the source material.
But if you have an arbitrarily long conversation with an LLM about Ohm's Law, and you can't find any flaws with its conception of Ohm's Law, IMO at some point you must concede that it knows Ohm's law. Knowledge is IMO a functional property, the process ultimately does not matter if the results are there. Yes, I can easily acknowledge that one might be easier convinced of someone's knowledge if the process is known. I (presume to) know you are human, therefore I get certain axioms for free about how you work and think. I don't get that for an LLM. But to say the process does matter in a way that no evidence that does not include internal insights will always be insufficient is basically saying that no non-human entity of any kind can ever know anything. Do ravens know how to use tools? Do elephants know what death is? Does your Roomba know the layout of your home? Does chatGPT know Ohm's Law? These are all categorically "no", if you put too much weight on processes rather than results. And I'm not saying the answer is necessarily Yes, but the method of determining that answer must at least in principle permit either answer.
As an olive branch of sorts, a middle ground if you will, I will easily concede that any artifacts of process that are observed in results are also fair game. If I explain to you my reasoning, or you see a raven experimenting with tools, or an LLM puts it's thinking
capmode on, and you read those thinking tokens, all of those are useful in getting a glimpse of the process.But the process line of thinking is often restricted to -more or less- something subjective or human-centric: They're human, therefore I know the process, therefore knowledge is possible. And that implicitly and often categorically excludes non-humans.
Put another way: Say we had the AIs of science fiction. The good ones that is, not the faulty ones. Think, perhaps, more Data and less HAL. How would you make the judgement that they know things? If your answer is "I wouldn't", pick a better AI. I hope you're convinced that for some AIs, the answer must invariably be "it knows things", and that's where we need to think about a judgement that doesn't rely on process.