skybrian's recent activity

  1. Comment on Git at any scale in ~comp

    skybrian
    Link
    From the article: [...] [...] [...] [...] [...] [...]

    From the article:

    Over the years, companies that tried hosting Git repositories at scale noticed that this packfile-based design was a major limitation on both availability and scalability. Packfiles are large binary files that must exist on a filesystem for Git to access them. The simple approach of having an HTTP server in front of a repository on disk has a very low ceiling. Ideally you'd want the repository to exist on many disks and many machines (this lets you run many Git operations in parallel, and keeps your repository available when a server crashes). But how do you do that?

    There are broadly three possible approaches to accomplish this, in increasing order of complexity: distribute the filesystem, distribute the packfiles, or distribute Git itself.

    [...]

    Spokes was originally developed at GitHub around 2013, and it has since become an industry standard. Most Git hosting services use a variant of the Spokes approach (application-level replication for Git repositories) in their architecture. The main reason Spokes has worked well for many years is that it made three fundamental choices that, over time, have been proven to be optimal:

    [...]

    Spokes is a consensus-based distributed system. It works by storing several copies of your Git repository on different servers. Whenever you push new data, an orchestrator fans out your push so that every instance of your repository receives a copy. The "fan-out" is synchronized with a classic consensus algorithm called 3PC (three-phase commit) so that a push is only accepted if a majority of the nodes acknowledge it.

    [...]

    This scalability constraint also applies the other way. When agents work with Git repositories at scale, they often operate outside of a monorepo by creating vast numbers of small repositories, many of them throwaway, and most of them barely touched. Spokes struggles here because it still requires three replicas for every one of these repositories. Three mostly idle replicas, which cannot be trimmed down because then the system wouldn't be fully consistent and data loss would be possible. With three-phase commit, the floor is always too high, and the ceiling too low.

    Another flaw, impossible to see up front, but painfully obvious after having suffered through it, is that Spokes can be rough to operate at scale. Because the repositories on disk are always the source of truth for consensus, every copy of every repository is very important. You have to treat repositories as pets, not cattle.

    [...]

    Continuity is a simple system (a system cannot be easy to operate if it is not simple). The core primitive behind it is a write-ahead log, which we store in S3-compatible object storage. In production, we run directly on S3, but we designed it so it can be deployed on any cloud.

    [...]

    The local copy of the repository is, of course, a normal Git repository stored on a very fast NVMe drive. We do the same thing that Spokes does because I think Spokes got that exactly right. It allows us to reuse all the amazing OSS work of the Git community, including the upstream Git client and its many performance optimizations. It lets us focus on shipping new features, instead of doing weird stuff with Git.

    [...]

    We've seen that one thing that makes a Spokes cluster hard to operate is that it's very important to keep track of the location of every repository on each server. Continuity does this very differently. Where does every repository live? The answer is "anywhere". It doesn't matter! We treat repositories like a warm cache on disk, but the source of truth is always the write-ahead log in S3. The system is stateless, and there are no routing tables (and no relational database to operate — hashtag blessed). If a repository is missing from the local disk when accessed on a host, we just materialize it from the WAL. We can do this very efficiently, but of course we don't want to do this all the time, because it'd be wasteful. In production, we use rendezvous hashing to map a repository ID to the list of nodes where we expect it to be. All the state we require to route repositories is the repository ID and the current set of healthy nodes in a cluster. But if this state gets out of sync (e.g., a node becomes unhealthy), that's perfectly fine too. We'll just materialize the repository on whichever node comes next.

  3. Comment on Ukraine planned to swarm Moscow airports with AI-guided drones (gifted link) in ~society

    skybrian
    (edited )
    Link
    Maybe this explains the leadership shakeup? I guess they went with attacking Wildberries warehouses instead. Neither side in this war seems quite willing yet to escalate it into a wider war, so...

    Both sources told me planning for the operation was halted in July, when Zelensky abruptly fired its mastermind, Defense Minister Mykhailo Fedorov. As the architect of Ukraine’s military drone program, Fedorov had worked on the plan for most of this year, keeping it a secret even from some of his closest aides. He helped assemble a team of Ukrainian computer programmers to develop the AI guidance system, and he secured financing from Ukraine’s allies in Europe to produce the autonomous drones at scale, the two sources said.

    Maybe this explains the leadership shakeup? I guess they went with attacking Wildberries warehouses instead.

    Neither side in this war seems quite willing yet to escalate it into a wider war, so it's happening slowly as they test the boundaries. But escalation is a risk and civilian aviation seems rather vulnerable if the drone war gets further out of hand.

    7 votes

  4. Comment on Founder of collapsed Chinese property giant Evergrande sentenced to life in prison in ~finance

    skybrian
    Link Parent
    I would expect there to be a deposit held in escrow. I wouldn’t expect the builder to effectively be borrowing money from the home buyer. They’d get loans from a bank for that, or maybe raise...

    I would expect there to be a deposit held in escrow. I wouldn’t expect the builder to effectively be borrowing money from the home buyer. They’d get loans from a bank for that, or maybe raise money from investors.

    3 votes

  5. Comment on Founder of collapsed Chinese property giant Evergrande sentenced to life in prison in ~finance

    skybrian
    Link Parent
    I’m wondering why this pre-paying was a thing at all. It seems like a weird thing to do to use a real estate company as a bank if you’re saving up for an apartment? Why were there no government...

    I’m wondering why this pre-paying was a thing at all. It seems like a weird thing to do to use a real estate company as a bank if you’re saving up for an apartment? Why were there no government protections?

    2 votes

  6. Comment on Moderna, Merck say mRNA vaccine prevents melanoma from returning in ~health

    skybrian
    Link
    From the article:

    From the article:

    The study is the most advanced of a number of trials testing the approach against a variety of cancers, including lung, bladder, kidney and pancreatic cancer. While small trials have shown the promise of the approach, the Moderna and Merck trial was seen as a bellwether in demonstrating the technology — and a first step toward transforming cancer care with treatments that are individualized to a person’s tumor and that may ultimately be able to stay ahead of it as it changes.

    3 votes

  7. Comment on Car break-ins drop in San Francisco as drone use increases in ~transport

    skybrian
    Link Parent
    I don't know anything about them but the articles I shared mentioned a few.

    I don't know anything about them but the articles I shared mentioned a few.

  8. Comment on AI usage patterns in software teams in ~tech

    skybrian
    Link
    From the article: [...] [...] [...]

    From the article:

    Executives are personally active on AI at rates that match or beat their teams. CEOs at companies of 201 or more people went from 9% to 36% in six months, the largest jump of any cut in this report, suggesting the most senior leaders are learning the technology by using it rather than reading about it. Company size comes from third-party enrichment, so this cut covers fewer workspaces than the rest of the report.

    [...]

    Two years ago, fewer than one issue in a thousand was created by AI. Teams now use AI to write just under half of everything created in Linear, and at the current pace it will soon author more than people and integrations combined.

    [...]

    Teams that connected a coding agent roughly tripled their weekly pull requests over two years, from 21 to 65, while teams without one went from 8 to 10. These teams were already higher-output before coding agents existed, so the levels aren’t directly comparable, but each cohort against its own baseline tells a clean story, and nearly all the growth sits on the agent side.

    [...]

    The clearest indication of AI’s influence on product development is the dramatic output gains experienced by teams using coding agents over the last two years. We have no way of knowing whether this increased output led to positive business outcomes, but it shows a very clear correlation between AI adoption and acceleration.

    Perhaps more intriguing is the makeup of that adoption, and how it appears to be blurring roles. Senior leaders are doing more of the hands-on IC work, adopting AI aggressively to help them do it, and non-engineers are committing code. The suggestion that everyone in an organization is becoming a “builder” seems to be directionally true.

    Those gains haven’t shown up as time saved, though. Time spent on existing tasks in Linear held while AI usage appeared as a new layer of work, meaning the overall time spent on product development is going up rather than down. As far as we can observe, teams are working more, not less, suggesting AI has a Jevons paradox quality beyond token consumption.

    Many will rightfully argue that looking at pull requests indicates motion rather than value, which is certainly true, but it’s still a step forward from measuring tokens. A mechanical refactor might burn lots of tokens while a meaningful bug fix or code review doesn’t, so token spend and value don’t line up at all, and using one as a proxy for the other will be remembered as a relic of AI’s early days.

    11 votes

  10. Comment on Guyana’s oil-driven economy has seen the world’s fastest growth in GDP per capita in recent years in ~finance

    skybrian
    Link
    From the article: [...] [...] From the "some kinds of economic growth are better than others" department. But it sounds like they're handling it ok so far?

    From the article:

    Guyana, a small country in South America, has seen the fastest growth in gross domestic product (GDP) per capita in the world over the past decade.

    [...]

    A large and sudden expansion in oil production has driven most of this growth. Between 2020 and 2025, the country’s oil production grew 860%, making it a key contributor to global crude oil supply growth.

    [...]

    It’s too early to see the full effects of this, and hard to measure how far the oil boom has translated into better living standards in the country. Official poverty estimates, for example, have not been published since production began. But there is early evidence of the government channeling oil revenue toward citizens, for instance, through cash grants for every adult, free tuition at public universities, and increasing health spending.

    From the "some kinds of economic growth are better than others" department. But it sounds like they're handling it ok so far?

    5 votes

  12. Comment on How a giant battery is transforming a town centre in Cannington, Ontario in ~enviro

    skybrian
    Link
    From the article:

    From the article:

    The glowing phone booth, it turns out, is actually a giant battery. Built by Toronto-based startup Civic Grid on behalf of The Nourish and Develop Foundation (TNDF), it helps power the neighbouring food bank for cheap while providing residents with a mini town square to gather and recharge. For Civic Grid, the pilot project is a demonstration of a new kind of community infrastructure.

    “This square provides … lighting, shelter, seating, [and] an open courtyard to be an extension and to strengthen the TNDF’s existing programming,” Civic Grid founder and CEO Max Fine told BetaKit in an interview on Thursday. “Having these spaces within these communities, big or small, I think, are really essential as we electrify our systems and we go through this energy transition.”

    The courtyard’s 60 kWh energy system charges during the cheap, off-peak night hours, then powers the TNDF’s two buildings for six hours the following day. That keeps its fridges and community kitchen running at less cost, while also providing insurance if the power goes out.

    Fine said a battery like this shows how important energy access is for people to participate in society. Already, multiple people are charging their phones in the courtyard in the morning, and it has helped an electric wheelchair user get moving when they ran out of juice. This philosophy is also reflected in the design, with the stone base meant to evoke a community water well.

    3 votes

  14. Comment on We economists have done the maths: ‘growth’ is a doomed strategy – there is a better way in ~finance

    skybrian
    Link Parent
    If a purely socialist system succeeded at exporting, say, cars that consumers in other countries will buy, that would be quite an achievement! And tough to do now when competing with China. But...

    If a purely socialist system succeeded at exporting, say, cars that consumers in other countries will buy, that would be quite an achievement! And tough to do now when competing with China.

    But the place to start is probably something like manufacturing clothing. The bottom isn't a country that has a clothing export industry like Bangladesh. It's the countries that can't even do that. There was an attempt to build a clothing industry in Haiti after the earthquake, but it's collapsed further since then.

    6 votes

  15. Comment on We economists have done the maths: ‘growth’ is a doomed strategy – there is a better way in ~finance

    skybrian
    Link Parent
    "intrinsic inevitable" is doing a lot of work there. You can rule out a lot of failures by saying they weren't intrinsic or inevitable, but they're still failures, at least until the country...

    "intrinsic inevitable" is doing a lot of work there. You can rule out a lot of failures by saying they weren't intrinsic or inevitable, but they're still failures, at least until the country finally succeeds.

    Meanwhile, the track record for growth due to export-led global trade is pretty good. It's certainly not "pure" capitalism since it's often due to intentional government policy. (China is an example!) But it's embedded in a trade-based global framework.

    5 votes

  16. Comment on Fifty years ago, cosmic rays may have caused Apollo astronauts to see lights (2023) in ~space

    skybrian
    Link
    From the article:

    From the article:

    The mechanism behind the flashes described by Apollo astronauts remains a mystery. Perhaps the particles, which are components of c­osmic rays, emit radiation as they pass through part of the eye. Or perhaps they trick nerve cells to create the illusion of light. However the flashes happen, they are still an issue for astronauts. In 2006, about 80 percent of NASA and European Space Agency astronauts reported experiencing the flashes. How the phenomenon impacts astronaut health after months or years in deep space is unclear. As NASA plans crewed missions to Mars and the moon, scientists are devising new ways to protect astronauts from radiation (SN: 7/4/20 & 7/18/20, p. 18), such as portable magnetic shields that can deflect cosmic rays.

    5 votes

  18. Comment on We economists have done the maths: ‘growth’ is a doomed strategy – there is a better way in ~finance

    skybrian
    (edited )
    Link Parent
    When you write “growth of the economy as a whole,” do you mean for the entire world? Generally speaking, less economic growth in one country doesn’t imply more economic growth in a different...

    When you write “growth of the economy as a whole,” do you mean for the entire world? Generally speaking, less economic growth in one country doesn’t imply more economic growth in a different country. It’s more like economic setbacks are sometimes contagious. Example:

    Asian Financial Crisis

    A financial crisis started in Thailand in July 1997 and spread across East Asia, wreaking havoc on economies in the region and leading to spillover effects in Latin America and Eastern Europe in 1998.

    More spending in the US on imports has helped growth in many other countries that have exporting industries. Cutting back on spending (as threatened by Trump’s tariffs) doesn’t help.

    A major exception is that increased carbon emissions are bad worldwide. That really is zero-sum. But that’s not true of most kinds of economic activity.

    5 votes

  19. Comment on We economists have done the maths: ‘growth’ is a doomed strategy – there is a better way in ~finance

    skybrian
    (edited )
    Link Parent
    Yes, there are many success stories. I’m saying that economic growth for the poor countries where it hasn’t happened yet (mostly in Africa) is very important.

    Yes, there are many success stories. I’m saying that economic growth for the poor countries where it hasn’t happened yet (mostly in Africa) is very important.

    9 votes

  20. Comment on We economists have done the maths: ‘growth’ is a doomed strategy – there is a better way in ~finance

    skybrian
    (edited )
    Link
    "Economic growth" is an extremely vague concept. What kind of growth is it? Some kinds of economic growth have to be better than others, and we should make distinctions. There are some kinds of...

    "Economic growth" is an extremely vague concept. What kind of growth is it? Some kinds of economic growth have to be better than others, and we should make distinctions.

    There are some kinds of economic growth that I'm quite dubious of. The gold rush to build datacenters as rapidly as possible, for example. What's the hurry? Seems like it's due to greed and competition.

    However, I don't see how anyone could deny that some poor countries are severely lacking in economic growth of nearly all kinds. To say that we don't need any more economic growth seems like the sort of "luxury belief" that would only make sense to rich people in rich countries.

    Edit: clarified wording.

    9 votes