29 votes

Overview of the proofs behind the Navier-Stokes $1 million millennium prize problem

18 comments

  1. [11]
    Barney
    (edited )
    Link
    This is deeply disturbing on many levels, which I will not dive into here, but perhaps in another post / topic, because I think there's an interesting conversation to be had here around AI proofs...

    This is deeply disturbing on many levels, which I will not dive into here, but perhaps in another post / topic, because I think there's an interesting conversation to be had here around AI proofs and the future of mathematics in general.

    Nonetheless, OpenAI disgusts me.

    While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models⁠.

    Yeah, right. As of the initial publishing of the paper, it only had 16 citations, which is laughable for a proof of this size, not to mention it didn't credit the team that has been working on this problem for years and were close to solving it, Diego Córdoba and Luis Martínez-Zoroa.

    OpenAI now amended the paper to include additional references, but at best, this is blatant plagiarism. Why do we allow this? Why are they not held accountable? Why can years or decades of hard work be stolen so easily? Breaks my heart.


    Here is a link to the originally published paper with its 16 references. 16 references for 165 pages.

    53 votes
    1. [4]
      psi
      Link Parent
      The larger issue is that OpenAI used millions of dollars of compute to scoop the researchers who were honing in on a solution. Admittedly those researchers were potentially still years away, but...

      The larger issue is that OpenAI used millions of dollars of compute to scoop the researchers who were honing in on a solution. Admittedly those researchers were potentially still years away, but OpenAI's behavior must be massively demotivating for mathematicians. Why spend your career working on a difficult problem if a lab will just throw 10,000 agents at that problem and solve it over the weekend? The entire episode shows that these labs are extremely capable but also not responsible stewards of their own technology.

      Also, regarding the citations: even in OpenAI's most recent draft of the proof, Levent Alpöge (Buckman's collaborator) still hasn't been cited.

      16 votes
      1. [3]
        thecakeisalime
        Link Parent
        This is basically how mathematics progresses. Mathematicians use computer-based tools all the time to help them, and the only difference here is that these specific mathematicians are backed with...

        Why spend your career working on a difficult problem if a lab will just throw 10,000 agents at that problem and solve it over the weekend?

        This is basically how mathematics progresses. Mathematicians use computer-based tools all the time to help them, and the only difference here is that these specific mathematicians are backed with more resources than anyone else gets.

        There are a lot of issues here with plagiarism and the lack of citations which shouldn't be excused, and it seems like they chose this specific problem explicitly to scoop the existing researchers, which is a pretty shitty thing to do, but using AI to find solutions to problems should not be the controversial part of this.

        Imagine that some billionaire has a grudge against a mathematician, and decides one day to spend as much money as necessary to hire a bunch of other mathematicians to just get together, solve the problem, and scoop the guy he hates. That's obviously a shitty thing to do, but it's good for mathematics as a whole. That's sort of what happened here, other than the plagiarism.

        9 votes
        1. [2]
          psi
          Link Parent
          There are no other computed-based tools comparable to an LLM. It's a totally different paradigm. Probably the previous largest technological breakthrough for mathematics was the internet, but the...

          There are no other computed-based tools comparable to an LLM. It's a totally different paradigm. Probably the previous largest technological breakthrough for mathematics was the internet, but the internet encouraged collaboration, not hoarding. I also wouldn't agree that simply having a solution is necessarily good for mathematics as a whole. As Terence Tao puts it:

          [...] The incentives may now be pointing in the direction of no longer sharing any promising research directions with the broader community, which would reverse centuries of traditions of open science and do serious long-term damage to the future of the field.

          In short, the indiscriminate use of powerful solution-extraction tools can achieve the immediate short-term goal of solving problems at hand, but at the cost of sustaining the ecosystem for the next wave of progress, or in understanding the progress already obtained. [...]

          There is much more to doing mathematics than just having the answer. Indeed, in mathematics the journey is usually at least as important as the destination. However, LLM-steered mathematics tends towards inelegant and clunky intermediate results, meaning that one loses much of the value that would normally be gained from having a solution to a difficult problem.

          14 votes
          1. thecakeisalime
            Link Parent
            I disagree with this conclusion. Most promising research direction gets shared in the form of published papers, and those get published (for better or worse) because if researchers don't publish,...

            The incentives may now be pointing in the direction of no longer sharing any promising research directions with the broader community

            I disagree with this conclusion. Most promising research direction gets shared in the form of published papers, and those get published (for better or worse) because if researchers don't publish, they lose funding and may be fired.

            In this case, it seems like OpenAI brute forced the solution, but I do think that with time, AI will be able to also "discover" some of the intermediary solutions. That probably depends on how researchers decide to use AI to tackle the problem. It's also potentially possible to take the solution and work backwards to discover things that may have been discovered during the journey.

            Brute forcing solutions is not new to mathematics. Sometimes people just want to know if something is true. I would be very surprised if no one was able to use this solution elsewhere.

            6 votes
    2. [6]
      Adys
      Link Parent
      There’s a lot of backlash against OpenAI on this and, to be clear, I really dislike OpenAI as a company. But knowing how ai training works, the way I read that sentence from their article is a...

      There’s a lot of backlash against OpenAI on this and, to be clear, I really dislike OpenAI as a company.

      But knowing how ai training works, the way I read that sentence from their article is a CYA. Probably maybe there can have been some formulation in the final result that was influenced by this training because it was part of the chain and they used 10k agents. But, it’s not at all realistic to think that this was actually meaningfully influenced by this usage, and it’s so ridiculously far from being plagiarised that it doesn’t make sense to write what you wrote.

      Like, people don’t understand how little influence individual datapoints have over ai training. It’s microscopic. The whole point is there’s billions of these datapoints to learn from… usually what has the most influence gets weighted out a ton.

      9 votes
      1. [4]
        SloMoMonday
        Link Parent
        I don't dispute that most individual inputs tend to get flattened out of general relevance and you're left with an approximate average through the model. But that uniformity is lost with context...
        • Exemplary

        I don't dispute that most individual inputs tend to get flattened out of general relevance and you're left with an approximate average through the model. But that uniformity is lost with context and almost disappears as you get to exceptionally granular topic and niche fields. I'd argue that the spesific input data can only be exaggerated as there are only so many established paths through those topics.

        Sort of how you can start by asking an LLM about a story of a boy and get a massive verity of answers. Make it about high school and the outcome window narrows. Set it in the UK, it narrows further and you might start to see some notable trends across output-generations. Take it far enough and eventually you can get to Harry Potter. To be fair there is corrective measures, architectures and formal-verification tools to catch this sort of thing, but the token paths have been (proven to exist)[https://arxiv.org/html/2601.02671v1] in these models training results (and I don't think there's any way around that because these are language-boxes, not analytic engines).

        That doesn't even consider the fact that in this case the plagiarism doesn't need to be word for word. We just need to get all the agent logs to verify that the researches works were not ingested and there was no word-for-word reproduction in the reasoning/background processes. Like I said, there are methods to prevent reproduction, but that doesn't mean the research data was not used without consent and for financial gain. I'm actually curious if this whole situation backfires and academics drop the big AI labs entirely with how its looking.

        The sequence of events is strange:

        OpenAI admits that their work on the Navier-Stokes equation was inspired by rumors that Alpöge and Buckmaster had solved a Millennium Prize problem.

        That's immediately a major red flag.

        From oAI own report

        Nearly 100 agents worked together for approximately 50 hours to produce our Euler regularity disproof.

        Once we saw the Euler solution, we thought that Navier–Stokes was the most promising problem to work on. Thus, we decided to devote our resources to Navier–Stokes. To do so, we shifted agents away from the other Millennium Problems and prompted these agents with the Euler resolution. When a further trained version of our internal model became available over the course of the effort, we updated our agents to that model.

        So, they had an "internal model" with x-thousand agents running problems. At some point they decided to re-orientate their entire research effort towards a single problem and employed a fresh training dataset. The delta between those training sets is a critical consideration.

        According to Buckmaster's statement

        My work with Levent has been a purely personal collaboration, free of any
        institutional agreements or official involvement by either of our employers. We
        used several LLMs throughout: Anthropic’s Claude, OpenAI’s Codex, especially
        with GPT-5.6 Sol and, more recently, Astra. The latter was only used for
        writeups and auditing our arguments.
        ...
        I asked when the first prompt had been sent by them. This question was
        not answered directly by OpenAI for some time. Eventually it was agreed that
        it had been sent in the past few days, after information about our work had
        reached OpenAI.
        I asked whether the model had been trained on, or had access to, our sessions
        in Codex, into which we had been putting all our drafts for the whole of this
        project. I was told the model did not look up user data. I asked again, about
        training, and I did not get an answer.

        There's also the direct conflict with the previous hype-push of "AI is so smart that it escapes containment and does a bunch of stuff without us knowing". So is that problem fixed and all the AIs pinky promise to be good. We also cant forget the fact that openAI is in a bit of a sticky situation and since revenue is out of the question, the only way for them to IPO is with hype.

        And the fact that this is an entire industry built on IP theft. And there the whole issue of how an LLM can be used to generate proofs...

        21 votes
        1. [3]
          Adys
          Link Parent
          But like … I don’t know how this can possibly matter. The dissenting researchers are not claiming they did any work by themselves at any rate; their claim is on the usage of OpenAI as the...

          But like …

          I don’t know how this can possibly matter.

          The dissenting researchers are not claiming they did any work by themselves at any rate; their claim is on the usage of OpenAI as the inference provider.

          So this isn’t even so much about training on OpenAI inputs … it’s training on its own outputs which would be the only relevant part of the work there.

          Which is super irrelevant since its own outputs are by definition reproducible with the right inputs.

          3 votes
          1. [2]
            psi
            (edited )
            Link Parent
            But most people wouldn't know "the right inputs". That's almost entirely the controversy. Buckman and Alpöge allege (in short) that OpenAI stole their research program and scaled it up 10000 times.

            But most people wouldn't know "the right inputs". That's almost entirely the controversy. Buckman and Alpöge allege (in short) that OpenAI stole their research program and scaled it up 10000 times.

            6 votes
            1. Adys
              Link Parent
              According to them, there has been minimal actual input going into this. I wouldn’t bet my life on it but I’d still put money on the “inputs” being nothing more than vague directions, the problem,...

              According to them, there has been minimal actual input going into this. I wouldn’t bet my life on it but I’d still put money on the “inputs” being nothing more than vague directions, the problem, and a harness that can launch 10k agents. Bunch of compute that is.

              3 votes
      2. TangibleLight
        Link Parent
        If you're talking about regions well in-distribution, sure, but this is not always true. Especially in high-dimensional situations, data points out of distribution generally have more control over...

        Like, people don’t understand how little influence individual datapoints have over ai training. It’s microscopic.

        If you're talking about regions well in-distribution, sure, but this is not always true. Especially in high-dimensional situations, data points out of distribution generally have more control over the model. Hundreds of billions of parameters (or probably trillions of parameters for an internal fronteir model) is certainly high-dimensional, and a particular counterexample to a millennium prize problem is certainly out-of-distribution. All that to say: if that proof is in the training data, it is almost certainly an influential data point in that region.

        Of course there are ways to mitigate this to varying effect. I'm sure OpenAI employs some of them, but without insider information it is impossible to say unconditionally that "it's microscopic."

        https://en.wikipedia.org/wiki/Leverage_(statistics) talks about this but mostly in the context of simple regression.

        Probably maybe there can have been some formulation in the final result that was influenced by this training because it was part of the chain and they used 10k agents.

        The takeaway to me is that 10k agents wandered around with the model distribution to finally land on the one thing which answers the question, then regurgitate it.

        8 votes
  2. [5]
    Carrow
    Link
    I recall once in grad school, a prospective professor was showing their work (it was even fluid related too). Part of it involved fitting data to a line. They used a polynomial with like 8 terms,...

    I recall once in grad school, a prospective professor was showing their work (it was even fluid related too). Part of it involved fitting data to a line. They used a polynomial with like 8 terms, none of the coefficients had any connection to real world properties, not derived from first principles at all. But it drew the line well, and they were proud of it.

    One professor there in the same subfield had derived an equation from first principles that fairly accurately represented those graph types based on actual physical properties reduced to only one fitting parameter. It wouldn't draw the line quite as well though.

    Nonetheless, the field seemed to favor these ugly polynomials when it came to what was used in papers. Computer simulations with more silicon, more digits, not looking at the real world.

    So what I've been keen on finding out is, are these AI solutions more like the prospective prof's, where they just threw more numbers on the board until everything fit expectations regardless of the underlying physics, or our prof, where they actually model the behavior based on principles? Is there a novel model, or are they over fitting the data to get a solution? The "smooth" aspect seems analogous at the least.

    (Fun aside: my profs also complained of how scientists didn't get how to mathematically model vortexes accurately, so of course they show up in the solution.)

    19 votes
    1. [3]
      trobertson
      Link Parent
      By design, they are the first type. The Universal approximation theorem, which is the foundational theory in ML/LLMs, is all about universal approximation without care for subject matter knowledge...

      are these AI solutions more like the prospective prof's, where they just threw more numbers on the board until everything fit expectations regardless of the underlying physics, or our prof, where they actually model the behavior based on principles?

      By design, they are the first type. The Universal approximation theorem, which is the foundational theory in ML/LLMs, is all about universal approximation without care for subject matter knowledge or understanding. Quoting wikipedia:

      the universal approximation theorems (UATs) state that neural networks with a certain structure can, in principle, approximate any continuous function to any desired degree of accuracy.

      Universality is achieved by increasing the number of neurons in the hidden layer, making the network "wider." Other versions of the theorem show that universality can also be achieved by keeping the network's width fixed but increasing its number of layers, making it "deeper."

      a.k.a., just chuck more variables at it.

      7 votes
      1. [2]
        stu2b50
        Link Parent
        That's kind of conflating multiple different concepts. For one, the foundational theory in ML is not the universal approximation theorem. Neural networks are just one area; most of ML is about...

        That's kind of conflating multiple different concepts.

        For one, the foundational theory in ML is not the universal approximation theorem. Neural networks are just one area; most of ML is about convex optimization, for one. A gradient forest or linear regression has no relationship to a neural network, and certainly does not have the same number of parameters. There's the whole field of generative models (which, unfortunately, now has a namespace collision with generative AI), which are models where you explicitly define the prior probabilities rather than just trying to recreate the posterior probability.

        As an aside, I would also say that I wouldn't even consider that the "foundational theory" in neural networks. It's a somewhat trivial proposition. What is more relevant is empirical observation that local minima are unlikely to occur in high-dimensional spaces, since the obvious theoretical issue with high dimensional models is that the optimization space is absolutely NOT concave, and therefore optimizing it is a NP-complete problem. What makes them work is that you can use a greedy approach and mostly get away with it in high dimensions.

        Secondly, although that has relevance to the generated text, the way that an LLM would go about solving something like Navier-Stokes has nothing to do with it. As you can see from their whitepaper, they were iterating on closed-form solutions and made heavy use of Lean to ensure it was formally correct.

        15 votes
        1. psi
          Link Parent
          These are good points, but I think you risk underselling the importance of the universal approximation theorem. For a small neural network (e.g. reproducing a 1-dimensional function), you don't...

          These are good points, but I think you risk underselling the importance of the universal approximation theorem. For a small neural network (e.g. reproducing a 1-dimensional function), you don't even need backpropagation; you can use whatever minimizer you'd like.

          1 vote
    2. scarecrw
      Link Parent

      With four parameters I can fit an elephant, and with five I can make him wiggle his trunk.

      3 votes
  3. uaeaaie
    Link
    [...] [...]

    The Navier-Stokes equations account for the fact that fluids can have viscosity, or friction. (Fluids with more viscosity, like honey, flow slowly, while those with less viscosity, like water, flow more quickly.) A simpler, related set of equations called the Euler equations describe fluids with zero viscosity, which flow without friction. [...] in 2013, when Thomas Hou of the California Institute of Technology and Guo Luo, now at the Hang Seng University of Hong Kong, derived a groundbreaking result showing that the Euler equations can “blow up,” as mathematicians like to say, in a cylinder if the top and bottom halves are set spinning in opposite directions.

    [...]

    There are a few intellectual steps to get from there to the present day. The first is the question of a boundary. The Millennium Prize version of the problem asks what happens in three-dimensional space that extends indefinitely in all directions. [...] The next major step involves modeling the forces that cause fluids to move. [...] [Mathematicians have] sometimes tried to introduce awkward, ungainly forcing functions to get fluids to behave in odd ways. But the Millennium Prize version of the problem asks what can happen when the forcing function is mathematically well-behaved, or “smooth.”

    [...]

    By 2023, the pair [Martínez-Zoroa and Córdoba] had proved that a version of the Euler equations with a messy forcing function displayed singularities. [...] In broad outline, the pair’s technique relies on creating an infinite sequence of “layers,” each of which is a non-singular solution to the equation they are studying. [...] They then combine those solutions in what Martínez-Zoroa calls an “infinite cascade” to produce a new solution.

    That new solution, they showed, contains the desired singularity. However, even though each individual layer relies on a smooth forcing function, combining them together can cause the forcing function to have undesirable mathematical properties. The remaining hurdle was to figure out how to create a similar infinite cascade that resulted not only in a singularity, but also in a smooth forcing function.

    That’s the step that both competing AI groups appear to have had success with.

    9 votes
  4. psi
    Link
    A couple dozen Fields Medal recipients have since issued a statement spurred by this development. "A Severe Misalignment of AI in Mathematics"

    A couple dozen Fields Medal recipients have since issued a statement spurred by this development.

    Over the last few months, the mathematical capabilities of LLMs have improved dramatically, to the point that they can solve major outstanding problems in many fields of mathematics. However, the push by AI companies to solve mathematical problems as a benchmark is detrimental to the science of mathematics, and to the mathematical community. The goals of the AI companies and the goals of the mathematical community are severely misaligned. We see these as part of broader alignment issues impacting other scientific and creative professions, as well as the whole of society.

    4 votes