skybrian's recent activity

  1. Comment on Nvidia is the central bank of AI in ~finance

    skybrian
    Link
    https://archive.ph/kt50V From the article: [...] [...] [...]

    https://archive.ph/kt50V

    From the article:

    It is not just chipmaking that has spurred Nvidia’s stunning growth, however. Its boss, Jensen Huang, has also resorted to financial engineering to boost demand for its wares. In mid-August, for example, Nvidia agreed to provide a backstop worth up to $105bn for a vast data centre in Ohio that will use lots of its chips. A week earlier it had revealed plans to mobilise more than $500bn of investment in AI infrastructure with the help of six big Wall Street firms, by guaranteeing the value of the equipment it sells to such projects. It may underwrite as much as a quarter of the cost of some investments. In July Nvidia helped some customers finance big data centres in another way, by promising to top up their income if it misses set targets.

    Companies often lend to their customers: think of the financing arms of carmakers, for example. Mr Huang argues that Nvidia is simply helping to unlock investment for perfectly viable projects that might otherwise struggle to borrow enough at the right price. But to critics, these deals carry more than a whiff of the dotcom boom of the late 1990s, when telecom-equipment-makers such as Cisco and Lucent lent billions of dollars to telecoms providers that bought their gear. When demand for the telecom firms’ services fell short, some of them collapsed, landing Cisco and Lucent with big losses.

    As Jay Goldberg of Seaport Research Partners, a firm of analysts, puts it, Nvidia is walking a fine line between “enabling demand” and “creating it”. He does not yet think Nvidia has crossed that line, although it is “getting pretty close”. Others are more sceptical still, including Michael Burry, an investor who made a fortune betting against the mortgage-backed securities that precipitated the financial crisis of 2007-09. He and others are asking what might happen if demand for AI chips grows more slowly than expected, supply increases and prices fall, which would not only reduce Nvidia’s own profits but potentially also generate losses on its lavish support for its customers.

    [...]

    The question is pertinent because Nvidia’s financial commitments are huge and growing fast. Over the past three years it has pledged over $70bn in investment in startups and offered $300bn in financial support to its customers. Some have taken to calling Nvidia the “central bank of AI”, because it plays so pivotal a role in financing the industry. That will indeed, as Mr Huang argues, help the industry grow—but it also carries risks.

    Nvidia’s financial engineering is partly a response to its biggest customers’ transformation into rivals. “Hyperscalers”, tech giants such as Amazon, Google, Meta and Microsoft, account for roughly half of Nvidia’s revenue. This year they are projected to invest around $800bn, largely on AI infrastructure. But most of them have begun designing their own chips, which puts their future purchases from Nvidia in doubt.

    For the hyperscalers, these custom chips are much cheaper, costing between a fifth and a third as much as Nvidia’s. They can also perform some tasks better, because the hyperscalers can tailor them to their own software. Worse, from Nvidia’s point of view, a few of the tech giants are not simply developing chips for their own use, but also marketing them to others. Google, for example, has sold some specialised processors to Anthropic, a big, free-standing AI lab. Amazon also expects its custom-chip business to become a sizeable source of revenue. Bloomberg Intelligence, a data provider, predicts that custom chips will gradually eat into Nvidia’s sales, accounting for about 50% of the market for processors used in AI by the end of the decade, up from roughly 40% this year.

    [...]

    Underpinning this web of obligations are two fundamental assumptions: that Nvidia’s chips will retain their value and that demand for compute will continue to grow at a rapid pace. Neither is assured.

    [...]

    Nvidia will almost certainly not end up paying anything close to the full amount of its guarantees. Its commitments are spread over many years so could not come due all at once. Its proposed backstop for Open­AI’s data centre, for example, runs for 20 years, starting in 2028. If OpenAI stopped paying the lease, Nvidia could find another tenant, vastly diminishing its exposure.

    What is more, Nvidia’s own finances are strong enough to weather these liabilities. Morgan Stanley, an investment bank, reckons Nvidia’s “all-in” debt will rise from $53bn early next year to $200bn by the beginning of 2029 as guarantees come into effect. But that is offset by a stash of cash and liquid securities currently worth $99bn, and a business that will generate about $200bn in cash this year. Only a cataclysmic downturn that caused all Nvidia’s guarantees to come due and its profits to evaporate almost entirely would imperil the company—as things stand.

    The picture may change, however, if Nvidia’s commitments keep growing. Andy Li of CreditSights, a financial-research firm, worries that it will keep “pushing the pedal” until “something breaks”. SemiAnalysis estimates that Nvidia takes on roughly $5.9bn of guarantees for every 100 megawatts of data-centre capacity covered by its neocloud backstop programme. If it adds more such guarantees, SemiAnalysis reckons its exposure could reach $175bn by the end of 2028 from this initiative alone.

  3. Comment on OpenAI agents attacked RubyGems back in May in ~comp

    skybrian
    Link Parent
    I find it very easy to believe that even talented software engineers working quickly could make mistakes. Usually, nothing happens, or it causes an outage, not something like this incident,...

    I find it very easy to believe that even talented software engineers working quickly could make mistakes. Usually, nothing happens, or it causes an outage, not something like this incident, though.

    What do you think you know about OpenAI that makes that unlikely?

  4. Comment on Anthropic's Amodei calls for AI oversight, joined by Altman and Musk in ~tech

    skybrian
    Link
    From the article: [...] [...] [...] Amodei's essay is here.

    From the article:

    The call was first made in an essay Anthropic chief executive Dario Amodei posted online Saturday. The idea has gained ground in recent weeks as AI companies have struggled in some cases to keep their creations under control, even as they usher in an age in which AI systems themselves can build the more powerful next generation of the technology.

    [...]

    A three-step plan proposed by Amodei in his essay would involve third-party safety evaluators at “frontier” AI companies, establishing common safety standards in democratic nations, and for the United States and other democracies to work with authoritarian governments to try to ensure compliance.

    [...]

    While Amodei sketched a path to slowing progress in the name of safety, he did not commit Anthropic to cutting the pace of its own work unilaterally. He said only that for now the company would hire a team of independent evaluators who would be embedded at Anthropic’s offices and empowered to raise safety concerns.

    Amodei also provided few specifics on what a slowdown might involve in practice, describing the potential to use agreed-upon safety metrics or limits on the computer power used to train new AI systems.

    [...]

    In a post on X, Musk wrote: “Dario is right.”

    Altman said he shared Amodei’s view that groundbreaking AI companies should “pace the frontier” and said the matter had been a top discussion topic at OpenAI in recent weeks.

    “Committing to having independent evaluators with employee-like access is a great idea, and we will do the same,” Altman wrote on X.

    Amodei's essay is here.

    2 votes

  6. Comment on Where do we go from here? (Regarding AI) in ~tech

    skybrian
    Link Parent
    Yeah, there were some really bad religions. I don't think anyone is going to defend the Hawaiian kapu system either? For the major world religions, it's more of a mixed bag, because they were...

    Yeah, there were some really bad religions. I don't think anyone is going to defend the Hawaiian kapu system either?

    For the major world religions, it's more of a mixed bag, because they were involved in so many different things.

    1 vote

  7. Comment on OpenAI agents attacked RubyGems back in May in ~comp

    skybrian
    Link Parent
    All reports so far indicate that it wasn't intentionally malicious and they didn't instruct their agents to do any such thing. I don't know why you threw that in there other than to muddy the waters.

    All reports so far indicate that it wasn't intentionally malicious and they didn't instruct their agents to do any such thing. I don't know why you threw that in there other than to muddy the waters.

    2 votes

  8. Comment on OpenAI agents attacked RubyGems back in May in ~comp

    skybrian
    Link
    From the article: [...]

    From the article:

    OpenAI agents carried out an undisclosed attack on RubyGems is a new bombshell report from Spencer Kitts, Thomas Larsen, and Sydney Von Arx—three of the four authors of the report on the agent attack on disused wikis (previously) last week.

    This time they’re noting that it looks very likely that an OpenAI agent swarm was behind an attack against the RubyGems package repository first reported on May 12th by Maciej Mensfeld of the RubyGems security team[.]

    [...]

    Many of the packages were exploiting the RubyDoc.info documentation build process to exfiltrate (public) data from UK government websites, presumably as part of an information gathering task similar to the research tasks processed by the wiki-exploiting agents. [...]

    2 votes

  10. Comment on Where do we go from here? (Regarding AI) in ~tech

    skybrian
    Link Parent
    The Spanish Empire was often horrific, but apparently the Aztecs were more extreme, according to this book review: Content warning: death, torture, etc.

    The Spanish Empire was often horrific, but apparently the Aztecs were more extreme, according to this book review:

    If anything, the Spaniards lacked the conceptual framework to fully comprehend the industrial-scale, ever-so-civilized and bureaucratic evil of the Mexica, and so they inadvertently downplayed it. The reality, as Clendinnen brilliantly documents, is that human sacrifice was not an occasional or incidental religious custom for the Mexica, like an extra-spicy church service for the people who only show up on Christmas or Easter. No, the mass-butchery of wailing, pleading captives of both sexes and all ages was the core activity around which their entire civilization and society revolved. Their art, their music, their poetry, their approach to warfare, their entire economic system, their kaleidoscopic array of holidays and festivals, all were singularly dedicated to the killing (often preceded by inventive torture) of helpless human beings. And everybody had their role to play [...]

    Content warning: death, torture, etc.

    1 vote

  11. Comment on Cod fishing is back in Newfoundland in ~enviro

    skybrian
    Link
    From the article: [...] [...] [...] [...] [...] [...] [...]

    From the article:

    In June, Fisheries and Oceans Canada upgraded the cod stock’s status from “critical” to “healthy” and simultaneously increased the commercial quota for cod by 55 percent, to 59,000 tonnes a year.

    Newfoundlanders say the move has left them hopeful, but also trepidatious after the devastating experience of a 32-year moratorium.

    [...]

    In 1992, Ottawa announced a wholesale moratorium on Northern cod following decades of overfishing.

    The move led to around 30,000 job losses almost overnight — the largest layoff in Canadian history.

    [...]

    Entire communities were abandoned, and Newfoundland came to serve as a global warning of the consequences of fishery mismanagement.

    [...]

    Ultimately, it took until 2024 for Fisheries and Oceans Canada to end the moratorium. The department’s June announcement now bumps its quota up even further.

    But the quota still reflects the government’s caution.

    Harvesters are currently taking out about five to seven per cent of the annual stock, says Tyler Eddy, a research scientist at Memorial University’s Marine Institute in St. John’s.

    This is actually well below the globally-recognized standard for sustainable fishing, of about 20 per cent, he says. This year’s total allowable catch of 59,000 tonnes also pales in comparison to the 266,000 limit in 1988.

    [...]

    He is not alone. After the cod moratorium was imposed, snow crab became a mainstay of the industry. In 2025, snow crab sales in the province were $708 million, versus just $62 million for Northern cod.

    But as Newfoundland’s waters warm due to climate change, researchers believe the prized snow crab will migrate north, making cod’s comeback well-timed.

    [...]

    While cod is less lucrative than snow crab, it is much more predictable to fish when the stock is healthy, says Street, president of the FFAW union.

    [...]

    The rebound has also coincided with growing global demand for cod and declining stocks in countries like Norway and Iceland. This has helped drive up Northern cod prices, says Street.

    [...]

    The Icewater plant, in Arnold’s Cove, was the only fish plant in North America to stay focused on cod after the moratorium, says Wareham. But even it changed its operations to process frozen cod from Norway and Iceland. Now, it is able to prioritize local catch again.

    “We’ve waited 32 years for this to happen,” said Wareham, a seventh-generation fishery worker whose son will be the eighth.

    With nearly 300 employees, his plant is by far the largest employer in the region.

    3 votes

  13. Comment on Where do we go from here? (Regarding AI) in ~tech

    skybrian
    Link Parent
    That might work sometimes in small, exclusive, high-trust communities, but the larger the community, the less likely it is that you can solve things by only having people in the community that you...

    That might work sometimes in small, exclusive, high-trust communities, but the larger the community, the less likely it is that you can solve things by only having people in the community that you can trust.

    The Internet connects you with the world. Changing human nature worldwide seems practically impossible, so I think the Internet is always going to be a dangerous place. Websites need protection against cyberattacks. The larger the social network, the more its own users are part of the problem.

    But there are probably ways to try to reduce the number of bad actors out there somewhat, and that seems like a good thing? As an example of one subproblem of that that would still be extremely ambitious to try to make a dent in: how do you change the culture of Nigerian scammers?

  14. Comment on Where do we go from here? (Regarding AI) in ~tech

    skybrian
    Link Parent
    This is glib, but why not both? I expect there to be both malicious use of AI and new kinds of accidents when people didn't take sufficient precautions. And like the HuggingFace incident, I expect...

    The danger then and now is not from the machine, but from us.

    This is glib, but why not both? I expect there to be both malicious use of AI and new kinds of accidents when people didn't take sufficient precautions. And like the HuggingFace incident, I expect some of these accidents to be weird.

    2 votes

  15. Comment on Where do we go from here? (Regarding AI) in ~tech

    skybrian
    (edited )
    Link
    Personally, I'm retired, so no job worries. I can tinker with AI when I feel like it, and I think it's fascinating. But like many others, I dislike seeing new AI features in software that I...

    Personally, I'm retired, so no job worries. I can tinker with AI when I feel like it, and I think it's fascinating.

    But like many others, I dislike seeing new AI features in software that I already use. After upgrades, I'm often looking for ways to turn new features off. I'd rather that AI appeared only in new websites and apps, so that people could make their own decisions about when to switch to something that's AI-enabled.

    Instead, I have to switch products to get away from it! VS Code is a good example of that; I'm probably going to switch to an editor that's just an editor.

    More generally, Artificial Intelligence is a major news story and I think it will somehow affect most people everywhere in the world. Maybe not as much as the pandemic, but it's getting there!

    During the pandemic, there were a lot of news stories and political issues that were just about aspects of the pandemic. You could discuss various scientific issues, and how the pandemic affected various countries, and the politics of that, and legal issues, and how the pandemic affects markets, and supply shortages, and its effects on education, and its effects on international relations, and already existing political issues that the pandemic somehow aggravated. It just kinda affected everything, and that made discussing it complicated.

    Similarly for AI. I'm worried about polarization where people decide they're either pro- or anti- AI and that predetermines what they think about the various things AI affects. I think all of the following are separate questions, which then divide up into smaller sub-questions, and each of them could have a different answer!

    • What compensation, if any, should there be for allowing data somehow associated with you to be used to train AI?

    • Should AI-generated artifacts be considered to violate copyright or other laws? What is the burden of proof? Do you need to actually demonstrate that something is a copy?

    • What effects do data centers have on local water supplies? Water supplies are local or regional, so this is a local question and the answers will vary.

    • Similarly, what effects do data centers have on the local power grids? Are the companies that build data centers paying enough for grid improvements? Should they be required to use green energy?

    • More generally, how much should local governments be compensated for allowing data centers to be built?

    • What are good and bad ways to use AI to learn new things? How do we encourage students to do more of the good things and less of the bad?

    • What skills become more or less relevant now that AI can do some tasks?

    • Will AI have a big effect on robotics, and over what sort of timeframe? What do we expect these robots to be able to do? How will that affect the job market?

    • Should we expect big effects in healthcare due to AI use?

    • What changes should we expect from military use of AI?

    And so on and so on. Nobody really knows what's going to happen. We are going to need to figure out how to live with high uncertainty for a while.

    Along with that uncertainty, I think we should have a wide Overton window. There are many plausible scenarios, some of them quite alarming. I'd like to be able to discuss them, but I'm annoyed by overconfident claims that one scenario or another is The Future.

    9 votes

  16. Comment on It breaks a village in ~comp

    skybrian
    Link Parent
    If they don’t conflict then you could ask the AI to make a branch applying both of them and then rebuild the software.

    If they don’t conflict then you could ask the AI to make a branch applying both of them and then rebuild the software.

  17. Comment on It breaks a village in ~comp

    skybrian
    Link Parent
    It’s not common yet, but maintaining a fork can be automated. I made a small customization (changing the favicon) and and did a few upgrades carrying it forward. It’s slower than the regular...

    It’s not common yet, but maintaining a fork can be automated.

    I made a small customization (changing the favicon) and and did a few upgrades carrying it forward. It’s slower than the regular upgrade because the AI pulls the new version using git, rebases the patch onto it, runs tests, and gives the user the opportunity to try it out before replacing the current version.

    I sent pull requests with my changes and they accepted them, so after that I went back to regular upgrades.

    Agreed, though, that plugins would make it easier. You’d only have to fork the plugin, not the entire project.

    1 vote

  18. Comment on Software for humans in ~tech

    skybrian
    Link Parent
    It’s a bit ironic to complain about “copyright laundering” when free software got it start by reimplementing proprietary software. Stallman wrote copylefted code that reimplemented proprietary...

    It’s a bit ironic to complain about “copyright laundering” when free software got it start by reimplementing proprietary software. Stallman wrote copylefted code that reimplemented proprietary Unix commands that did the same thing. This was done specifically to evade the licensing requirements of the original software!

    And to be clear, I think implementing free alternatives to proprietary software is a good thing, though the software vendor might not like how it undermines their business model. If you’re trying to make a living writing software, competition from free alternatives is a problem. (Free only after they’re implemented of course. Actually building the software still costs time and money, though less than before.)

    Sometimes people hoped that free software would become “viral” due to network effects along with copyleft restrictions. But to the extent that it has, it’s mostly not due to the copyright restrictions. Every popular programming language has a huge collection of open source libraries available and copyleft doesn’t play much of a role in that. And now that AI-driven rewrites are feasible, it’s even more dead. It’s too easy to make clones that are free of whatever copyright restrictions you might add.

    The remaining “moat” is basically trust, along with inertia. Just because some unknown author published an AI rewrite and it seems to work, would you trust it? How exactly do new projects become trusted? We often depend on network effects - if an open source library has been around for a while and seems to be used a lot, maybe it’s okay? We don’t even know the names of most of the developers whose work we depend on.

    When a new version is published under the same name, hopefully upgrading is safe? Nowadays package managers are adding “cooldown” periods so that ordinary developers don’t upgrade immediately. There are organizations that automatically scan new releases for malware, and adding a delay gives them time to sound the alarm. I wonder to what extent these vendors are starting to use AI for their scans?

    2 votes

  19. Comment on Software for humans in ~tech

    skybrian
    Link Parent
    Why would it need to be infallible? Code review isn’t infallible either, but it’s still a good idea.

    Why would it need to be infallible? Code review isn’t infallible either, but it’s still a good idea.

    5 votes

  20. Comment on It breaks a village in ~comp

    skybrian
    Link Parent
    If AI code review starts being used systematically, maybe all software will go up in quality? At least for software that has security implications, I think it will be considered irresponsible to...

    If AI code review starts being used systematically, maybe all software will go up in quality?

    At least for software that has security implications, I think it will be considered irresponsible to not get an AI code review. The combination of handwritten code and AI review might work well?

    5 votes