skybrian's recent activity

  1. Comment on The fall of the theorem economy in ~science

    skybrian
    Link Parent
    This seems sort of like relying on an axiom or conjecture, except you're not free to assume the opposite.

    This seems sort of like relying on an axiom or conjecture, except you're not free to assume the opposite.

  2. Comment on Bernie Sanders: The public should own half of the big AI companies in ~society

    skybrian
    Link Parent
    Thanks, I wasn't aware of it. Do startups commonly use that?

    Thanks, I wasn't aware of it. Do startups commonly use that?

  3. Comment on Bernie Sanders: The public should own half of the big AI companies in ~society

    skybrian
    Link
    Of course this legislation has no chance, but maybe it will get people talking? I wonder if a variation on it might work, though. The trick would be to get in early, before a startup's valuation...

    Of course this legislation has no chance, but maybe it will get people talking?

    I wonder if a variation on it might work, though. The trick would be to get in early, before a startup's valuation has gone up. Suppose that, when a startup sells stock to an investor or grants it to an employee, they could give 20% to a sovereign wealth fund, and in return, the investor or employee pays no capital gains? It would probably seem like a good deal at the time (increasing the value of the shares), and the government would hang onto the shares, perhaps selling some if there's a stock buyback.

    A lot of startups fail, but some of them grow to be giant companies.

  4. Comment on Which Substacks do you subscribe to/follow? in ~tech

    skybrian
    Link
    You can probably tell from the links I share. Construction Physics is pretty good.

    You can probably tell from the links I share. Construction Physics is pretty good.

  5. Comment on The dead economy theory in ~society

    skybrian
    Link
    Maybe, but I’m wondering how many jobs are purely cognitive in each industry. Much like happened in some US manufacturing centers due to foreign competition, it could be devastating while also...

    General-purpose AI threatens cognitive labor comprehensively, across every industry, simultaneously.

    Maybe, but I’m wondering how many jobs are purely cognitive in each industry. Much like happened in some US manufacturing centers due to foreign competition, it could be devastating while also being far from “the end of work.”

  6. Comment on It's not just X. It's Y. in ~humanities

    skybrian
    Link Parent
    If the LLM’s remain the same, their tells will become more and more recognizable. But I think this is more likely to be a phase? I don’t think Claude says “you’re absolutely right” anymore?

    If the LLM’s remain the same, their tells will become more and more recognizable. But I think this is more likely to be a phase? I don’t think Claude says “you’re absolutely right” anymore?

  7. Comment on Who’s buying SpaceX and Anthropic? in ~finance

    skybrian
    Link Parent
    There are shenanigans in other parts of the world too, and you miss some big gains that way.

    There are shenanigans in other parts of the world too, and you miss some big gains that way.

  8. Comment on Decades of effort restore steelhead and salmon passage on California's Alameda Creek in ~enviro

    skybrian
    Link
    From the article:

    From the article:

    Last year, California Trout and Pacific Gas & Electric removed the final barrier to fish passage on California’s Alameda Creek with funding from NOAA Fisheries’ Office of Habitat Conservation. For the first time in 50 years, threatened Central California Coast steelhead and other migratory fish can reach spawning grounds and juvenile rearing habitat in the upper watershed.

    Construction crews relocated a Pacific Gas & Electric (PG&E) natural gas pipeline and removed its concrete covering. The pipeline had spanned the creek and created an 8-foot drop in the creek. They installed the new pipeline section deep below the creek bed, removed the old pipe section, and regraded the stream channel—restoring a natural pathway for fish.

    1 vote

  10. Comment on Clanker: A word for the machine in ~tech

    skybrian
    Link
    One reason I dislike "clanker" is that it doesn't work as a metaphor. "Clanker" implies a machine that makes a lot of noise. Chatbots are silent. They're also invisible unless you go online. "Bot"...

    One reason I dislike "clanker" is that it doesn't work as a metaphor. "Clanker" implies a machine that makes a lot of noise. Chatbots are silent. They're also invisible unless you go online. "Bot" seems straightfoward, but I also like "ghost."

    3 votes

  11. Comment on It's not just X. It's Y. in ~humanities

    skybrian
    Link
    From the article: [...] [...] [...] [...]

    From the article:

    Recent overuse by language models has led many to declare it bad writing. I'm not so sure. Nobody called JFK a lazy writer when he said, "ask not what your country can do for you – ask what you can do for your country." Negative parallelism is a rhetorical device, and any rhetorical device is only as lazy or inspired as what it contains.

    [...]

    Now, we have AI detectors that claim to protect you from the witch hunt by looking for these patterns. You take your own writing and you run it through Grammarly, which will analyze word patterns that AI detectors might flag. Then it offers ideas for how to change them, which a) gives Grammarly the power to write for you and b) makes your writing lose any sense of rhythm or intent.

    [...]

    Defining reasoning the way it has been used in LLMs assumes that the point of asking a question is to get an answer, that answers can be verified, and that nothing is lost in immediate closure. This has real effects on writing, and the openness to doubt is something we lose in the rapid prototyping of thought that occurs with a language model. Ambiguity, doubt, and uncertainty matter more to some ways of thinking than any immediate answer. The inner life grows in the spaces between the industrial complexes that harness every remnant of our externalized thought.

    Nonetheless, the language we use in these states is the same. When AI detectors flag text as AI-generated, is it because it follows a certain structural pattern of that reasoning? Pangram and reasoning models both detect structural patterns based on how humans reason when writing. Pangram's model is trained on pre-2021 data; it then inserts AI-generated versions of the same text into its training.

    So, if we publicly shame people whose text looks like it might have been written by a machine – because it mimics the language used for human reasoning – and people stop writing in ways that they internalize as "AI writing" out of fear of false detection, it sends a signal that your language for reasoning must be policed, or you too could be held up to public scrutiny.

    In the end, shaming people for writing that gets flagged as AI can lead people to sidestep structures the model has learned from us: structures that are effective tools for argumentation. We take the tools of critical thinking out of the kit at the time we most need them.

    [...]

    I'm not convinced by the old "if you haven't done anything wrong, you don't have anything to worry about" line. I've seen 99.8% cited as a measure of accuracy in automated surveillance systems since 2018. As Arvind Narayanan has noted, that is on a per-paper basis, which compounds every time we use it. So up to 10% of college students could be falsely accused. If we collectively run every bit of text through an AI model to check whether it is AI-generated, we will generate false positives on an even larger scale.

    [...]

    We create a culture of self-censorship and AI-detector-pressured rewriting and paraphrasing as people strive to avoid these witch hunts. That is the opposite of protecting human expression. We should resist normalizing a trust in any machine's ability to determine matters of guilt. If using AI to write is, at its worst, an industrialization of the mind, then AI detection, at its worst, becomes a surveillance system for thought.

    4 votes

  13. Comment on Who’s buying SpaceX and Anthropic? in ~finance

    skybrian
    Link
    Matt Levine wrote today about the SpaceX IPO and the effect of index funds: ...

    Matt Levine wrote today about the SpaceX IPO and the effect of index funds:

    That is, the maximally cynical approach is to sell as little stock as possible to price-sensitive investors, in order to keep the supply low and the price high, and then sell as much stock as possible to index funds, who can’t negotiate on price. Actually “can’t negotiate on price” understates the issue. If index funds need to buy 24% of your stock, and only 20% of your stock is available to buy, then the index funds are forced to chase it, driving up the stock to whatever price you like. You have effectively created a short squeeze for the index funds: They have to buy stock at any price, and there isn’t enough stock for them to buy. “Just keep bidding,” you tell Vanguard.

    I want to be really clear that this is the schematic maximally cynical approach, is not what SpaceX is doing, and is not actually possible. Pretty much every big stock index is “float-weighted” or “float-adjusted,” meaning that if only 20% of your stock is available to buy, index funds are not actually going to be trying to buy 24% of it. (As a first cut, they should try to buy 4.8% of it — 24% of 20% — though the actual number is slightly higher.)

    ...

    The index demand is not 100% of the stock available in the IPO, or 110%, or even 50%. But it’s plausibly more than 25%. It’s not a short squeeze, but it’s a lot. Add a reported 30% allocation to retail, and arguably a majority of the IPO is being sold to price-insensitive investors. That is one way to get a high IPO price.

    5 votes

  14. Comment on Hackers used Meta’s AI support bot to seize Instagram accounts in ~tech

    skybrian
    Link
    From the article: [...] [...] [...]

    From the article:

    The Instagram accounts for the Obama White House and the Chief Master Sergeant of the U.S. Space Force were briefly defaced with pro-Iranian images and messages over the weekend, after instructions began circulating on Telegram showing how to trick Meta’s “AI support assistant” bot into resetting account passwords.

    [...]

    On May 31, word began to spread on several Telegram instant message channels that Meta’s AI bot would happily add an email address to an existing account as part of the bot’s standard password reset flow.

    [...]

    Meta has not responded to requests for comment on the video’s claims, but Meta’s Andy Stone said on Twitter/X that the issue had been resolved and that they were securing impacted accounts. The security blog thecybersecguru.com reports that Meta pushed an emergency patch over the weekend, and clarified that no back end database was breached.

    “Instagram has notoriously poor human support infrastructure,” Cybersecguru wrote. “Recovering a locked account – especially a high-value one can take weeks of back-and-forth with an automated ticketing system. Meta’s solution was to deploy a conversational AI layer to handle common recovery workflows: relinking a lost email address, triggering a password reset, verifying account ownership. The assistant, presumably, was supposed to reduce friction for legitimate users stuck in account-access hell.”

    [...]

    Securing your various online accounts means taking full advantage of the most secure form of multi-factor authentication (MFA) offered (such as a passkey or security key). In this case, even using the least robust form of MFA that Instagram offers — a one-time code sent via SMS — likely would have blocked the exploit: The hackers who released the video on Telegram said their exploit failed to work against any accounts that had MFA enabled.

    2 votes

  16. Comment on Why Chevron and PG&E are spending millions in California to boost Xavier Becerra in ~society

  17. Comment on Why Chevron and PG&E are spending millions in California to boost Xavier Becerra in ~society

    skybrian
    Link
    From the article: [...] [...]

    From the article:

    A pro-Becerra committee recently received significant cash dumps from major companies like McDonalds, which contributed $500,000, and Meta, which gave $950,000. Realtors, the California Medical Association, and tribes have also collectively given millions to support Becerra, who most recently served as President Joe Biden’s Health and Human Services secretary.

    Energy companies in particular have shelled out to bump Becerra and block Steyer, a longtime climate activist who signed a pledge not to take fossil fuel money and said that he would take on big oil companies and break up the state’s major utilities as governor — long-shot promises that would require some assistance from the state Legislature but that have nonetheless appealed to young liberal voters.

    [...]

    Meanwhile, an anti-Steyer committee has raised some $33 million to block his rise, which includes more than $13 million from Pacific Gas and Electric and a union representing electrical workers and nearly $12 million from a pro-business committee that has received roughly $7 million combined from PG&E, Sempra Energy, Edison, and Chevron.

    [...]

    The flood of money comes as Becerra and Steyer duke it out in the polls for a shot out of the June 2 primary and onto the November ballot. Becerra, who was polling in the single digits only a handful of weeks ago, has emerged as the clear Democratic front-runner in recent polls, with Steyer and Republican Steve Hilton close behind.

    Becerra supporters argue that the money flowing into his campaign and supporting committees is nothing compared to the more than $210 million Steyer has dumped into his bid from his fortune. And while Steyer has pitched himself to voters as a climate champion who will run Big Oil out of business, Becerra’s backers have noted that his billions came from his success at Farallon Capital, the hedge fund he started in the 1980s that invests in the fossil fuel industry.

    2 votes

  19. Comment on Who’s buying SpaceX and Anthropic? in ~finance

    skybrian
    Link
    Google already owns some of Anthropic indirectly. The S&P 500 plans to add these new stocks six months after IPO. Other than that, no real plans. I'm curious about Anthropic's financial...

    Google already owns some of Anthropic indirectly. The S&P 500 plans to add these new stocks six months after IPO. Other than that, no real plans. I'm curious about Anthropic's financial statements, though.

    1 vote

  20. Comment on The fall of the theorem economy in ~science

    skybrian
    Link
    From the article: [...] [...] [...] [...] [...] [...] [...] [...]

    From the article:

    In my first Substack post, I (half-jokingly) declared that we had been wrong about mathematics for 2300 years, stuck in a false dilemma between formalism (“mathematics is a meaningless game of formal symbols”) and Platonism (“mathematics captures properties of actual entities living in the perfect world of ideas”).

    My proposed conceptualist resolution is a rephrasing of Thurston’s view: mathematics does rely on a meaningless game of formal symbols, but we only play this game because we project meaning onto it.

    [...]

    This is how the system worked for millennia. Mathematicians created value by introducing new concepts, but the rule was that only theorems could put bread on the table. The deal was fine because the two aspects almost always walked hand in hand. David, the social parasite who claimed credit for the 𝐾⁡(𝜋,1) conjecture, was the same person as the David who crafted Definitions 2.4 and 9.3.

    Solving a big conjecture was a cryptographic proof that you had come up with a genuine conceptual innovation.

    I am using the past tense because this is no longer the case. There is a structural vulnerability in the honor code of mathematicians and AI has started exploiting it in a systematic manner.

    [...]

    The problem with unintelligible proofs goes way beyond correctness, and cannot be resolved by autoformalization alone: even if correct, unintelligible proofs aren’t accretive to the mathematical corpus.

    [...]

    Yes, I know, that sounds counterintuitive. This is why outsiders are likely to miss the nuance and label the pushback as Luddism.

    A clear explanation can be found in Alex Kontorovich’s account of his own learning curve with formalized mathematics. In a nutshell: Mathlib, the dominant Lean library, is a human-curated formalization of an ever-growing fraction of existing human mathematics. It exposes clean APIs and abstractions, without which no autoformalization could take place. By contrast, Math Inc’s autoformalized proof of Viazovska’s results exposes no intelligible interface. Who in their right mind would merge a 200,000-line unaudited vibe-coded blob into the master branch of global human science?

    [...]

    This is what made the Mathlib community so angry. They had been working on a multiyear project to formalize Viazovska’s work. Math Inc jumped in on this collective effort, leveraged prior insights, then abruptly went silent, until they made their spectacular announcement.

    Is this necessarily bad news? Now that the brute-force autoformalization is done, why can’t the Mathlib community refocus on the value-creating canonization?

    Because of Hardy’s curse and the honor code of mathematicians. Math Inc captured the prize (“first formalization of a Fields-medal-level theorem”) and there is no social reward left for cleaning up after them. Hence this comment by Patrick Massot, a non-Luddite expert in formalized mathematics:

    [...]

    The likely outcome is that formalized mathematics will now develop in two separate layers, an intelligible layer embodied by Mathlib, and an unintelligible layer we might call Mathslop, a library of results that are known to be correct via proofs that no human has ever understood.

    [...]

    Why don’t we construct benchmarks that are fairer to humans? Because we can’t. Because the true value of mathematics, the collective and individual elevation of our worldviews, is ill-defined and intangible. This intangibility was the raison d’être of the honor code.

    For millennia, we had agreed to only benchmark human intelligence on its problem-solving facet, as we had found that it was the best objective proxy. Theorem proving is so inordinately difficult for our cognition that the only progress path was through patient concept building and neuroplastic internalization of these new concepts.

    Yet this was only ever a proxy. The thing we really care about is different in kind. Industrial robots are far stronger than humans, yet we still go to the gym. Blenders have been outchewing us for over a century, yet we still don’t eat exclusively through straws.

    [...]

    If theorem-proving remains their only official currency, all research mathematicians run the risk of becoming demonetized in the course of the next few years.

    [...]

    All the active mathematicians I have cited, from Tao to Avigad, from Litt to Kontorovich, are careful to note that there is something in mathematics that goes beyond theorem proving. Yet this is usually framed as a passing remark, not a major shift in the narrative.

    Well, this is a major shift in the narrative, and the public will never take it seriously if it isn’t properly explained. Right now, it looks like an excuse.

    4 votes