14 votes

Breaking my hand forced me to write all my code with AI for 2 months

30 comments

  1. [4]
    Deely
    Link
    Interesting article. But I can't help but feel suspicious. It is written by Member of Technical Staff at Anthropic, and he uses his own product Claude to do all these stuff. ...write 3`000 lines...

    Interesting article.
    But I can't help but feel suspicious.
    It is written by Member of Technical Staff at Anthropic, and he uses his own product Claude to do all these stuff.

    By using a combination of voice-to-text and Claude, I was still able to write a tremendous amount of code at my job at Anthropic, including one week when I wrote over 3,000 lines of (admittedly, boilerplate-heavy) code!

    ...write 3`000 lines of boilerplate-heavy code... Honestly, that's just a bit insane for me. Even with both hands working. Did he at least understand what he writes?

    Migrations and refactors were perfect cases for this. I would migrate one instance manually and then use that as an example for Claude to transform the rest of my inputs. By following this format, I was able to refactor about 3,000 lines of code pretty quickly.

    So, maybe he refactored 3,000 lines of code instead of writing new code?

    For me this article looks like marketing ad.

    41 votes
    1. timo
      Link Parent
      Definitely feels like an ad. “Oh Claude is just so much better than Copilot”. “Even when it was wrong, it was actually me that gave the wrong input”. No downsides, like talking about the trial and...

      Definitely feels like an ad. “Oh Claude is just so much better than Copilot”.

      “Even when it was wrong, it was actually me that gave the wrong input”.

      No downsides, like talking about the trial and error vibes you get with AI. The lack of joy. Having an AI produce hard to find bugs, which you need to find and correct.

      23 votes
    2. [2]
      krellor
      Link Parent
      Depending on what it is 3,000 lines in a week of generated code isn't even all the much. When I was working with .net MVC framework, I would build the database first, and then have visual studios...

      Depending on what it is 3,000 lines in a week of generated code isn't even all the much. When I was working with .net MVC framework, I would build the database first, and then have visual studios generate CRUD methods for all the object classes generated from the database. It generated the C# class files with the get, list, update, delete methods as well as the client side razor files. Any decently sized database would give you thousands of lines of generated, working, boilerplate. I could see an LLM prompt to take it further and generate a frontend with asynchronous calls to the REST backend.

      But without them sharing the code they generated, it doesn't sound much more impressive then what good procedural tools can do within frameworks.

      That said, I have used Gemini in Google colab to generate some throwaway code, and it works well. One example is I wanted to scrape data from a paginated table of data with almost a hundred pages. I have a detailed prompt, and got a function that scrapped the data perfectly using beautiful soup. So that was nice not to fuss with.

      The bigger question I have around AI use here is that AI can be helpful to those who are knowledgeable and can give detailed educated prompts. But how does the next generation learn if we tell them to stop getting practice reps and just keep fiddling with prompts? I worry about the future skill of the average developer declining.

      7 votes
      1. creesch
        Link Parent
        This is also my concern. I am able to leverage LLMs quite well in areas where I have plenty of experience to know what to prompt for and to validate the code. Without that experience I would be...

        The bigger question I have around AI use here is that AI can be helpful to those who are knowledgeable and can give detailed educated prompts. But how does the next generation learn if we tell them to stop getting practice reps and just keep fiddling with prompts? I worry about the future skill of the average developer declining.

        This is also my concern. I am able to leverage LLMs quite well in areas where I have plenty of experience to know what to prompt for and to validate the code. Without that experience I would be just flying blind and given some of the results I am getting. I would get code that might work either not exactly as expected or in ways that will hugely impact performance or other aspects of the application.

        3 votes
  2. [7]
    patience_limited
    Link
    I fear the death of genuine expertise. Not because people are intrinsically unwilling to do difficult or boring work, but because there will no longer be paid employment for students and juniors...

    I fear the death of genuine expertise. Not because people are intrinsically unwilling to do difficult or boring work, but because there will no longer be paid employment for students and juniors to make every mistake and learn from it.

    Why teach millions of humans the basic (or intermediate, or advanced) reasoning and methods, when you can turn the work over to a few already trained LLMs, supervised by an ever-diminishing handful of experienced devs (or doctors, or lawyers, or...)?

    What happens when no one knows enough to catch errors before they're released into the world? Admittedly, humans don't have a perfect record on this either, e.g. manual vs. autonomous driving. Given the commercial deprecation of "safety" in LLM development, forgive me if I have a jaundiced view of the outcomes of unrestrained capitalism here.

    12 votes
    1. [5]
      teaearlgraycold
      (edited )
      Link Parent
      So far I feel like LLMs have accelerated my ability to learn. I recently set up a GPU cluster with used Cisco networking hardware. GPT-4 was able to guide me through the process of setting up...

      So far I feel like LLMs have accelerated my ability to learn. I recently set up a GPU cluster with used Cisco networking hardware. GPT-4 was able to guide me through the process of setting up everything, including configuring switches over serial console, resetting the password when I screwed that up, setting up link aggregation over SSH, dividing up a QSFP+ port into 4 SFP+ ports with arcane commands, etc. Googling around was not super helpful and the official documentation is not approachable.

      Here's a sequence of messages and response excerpts:

      Okay. So my switch and its downstream devices do not appear to be online. Why might that be? The transceivers are connected together by fiber.

      main-switch(config-if)# show interface ethernet 1/53
      Ethernet1/53 is down (Link not connected)
      admin state is up, Dedicated Interface
        Hardware: 1000/10000/25000/40000/50000/100000 Ethernet
      

      ...

       

      ChatGPT: If your Cisco switch interface shows as "down" and "Link not connected," even though the fiber transceivers are connected, there could be several reasons why the connection is not established. Here are some common troubleshooting steps to identify and resolve the issue:
      ...

      ^ Wasn't helpful

      So, what we did was take a 40G QSFP+ transceiver with a 4x breakout cable and connect just one of those to a 10G SFP+ port on our router.

       

      ChatGPT: Enable Breakout Mode: On your Cisco 9000 series switch, you need to configure the QSFP+ port to operate in breakout mode. This might involve setting the specific port to split into four 10G links. The command may vary slightly depending on the NX-OS version, but generally, you'd use something like:

      interface breakout module 1 port 53 map 10g-4x

      Which worked perfectly!

      Now, I'm not an expert on NX-OS by any means, but I have learned quite a bit even though I didn't have to struggle through reading manuals. Even just knowing the tidbit that I need to manually configure breakout mode instead of expecting the switch to magically detect that is useful. Taking a software engineer that's competent with Linux and turning them into a junior network engineer is pretty awesome!

      I also ask GPT-4 questions on fields of science, for which I should be careful not to trust it. But in the case of software and hardware I know it's correct because I can see the changes it allows me to make.

      5 votes
      1. creesch
        Link Parent
        I think you might be reinforcing the point the previous person was making as well, though. Specifically when you mention this line You are able to learn from it as you already have a knowledge...

        I think you might be reinforcing the point the previous person was making as well, though. Specifically when you mention this line

        Taking a software engineer that's competent with Linux and turning them into a junior network engineer is pretty awesome!

        You are able to learn from it as you already have a knowledge base to build on and at the very least know enough to be critical and ask the right questions. Not only that, it also gives you a much better framework to contextualize new knowledge and experience. People who start with much less knowledge and experience don't have that. As a result, they will struggle a lot more even getting to the right questions. And be satisfied much sooner by eventually getting a working answer without actually having internalized why it works and what got them there.

        The curse of knowledge is a real thing there ;)

        11 votes
      2. [3]
        patience_limited
        Link Parent
        That does sound helpful for solving specific problems and learning. I know how thoroughly Cisco's documentation can hide the useful bits. I suspect Cisco actively prevents Google search indexing,...

        That does sound helpful for solving specific problems and learning. I know how thoroughly Cisco's documentation can hide the useful bits. I suspect Cisco actively prevents Google search indexing, all in the interest of forcing learners to take their insanely expensive courses. Same with VMWare, Oracle, and other top-tier vendors. I'd suspect those vendors are readying to unleash their sharp-fanged legal departments over leakage of proprietary information for LLM training.

        That being said, there's no substitute for the fingertip feel and intuition you can develop from specialized learning and years of practice. In high-risk large scale infrastructure management, you need to have a very good understanding of change impacts, how to test, how to roll back, etc. It's not just knowing the correct command line strings, but how to troubleshoot and how to fix what breaks even if you did everything by the book.

        [It's been years since I've done this stuff and wouldn't dream of calling myself an expert anymore, but I remember what expertise feels like.]

        4 votes
        1. [2]
          teaearlgraycold
          Link Parent
          Exactly. It’ll only take me to “junior” or intern status. But being able to become a jack of all trades is really powerful.

          Exactly. It’ll only take me to “junior” or intern status. But being able to become a jack of all trades is really powerful.

          3 votes
          1. patience_limited
            Link Parent
            Oh, absolutely - that's what I now do for a living. I have just enough domain knowledge in many different areas to be very effective. And I do rely on Google and Copilot to refresh my memory and...

            Oh, absolutely - that's what I now do for a living. I have just enough domain knowledge in many different areas to be very effective. And I do rely on Google and Copilot to refresh my memory and stay up to date; I'm woeful at remembering how to do things in SQL without lookup and testing, for instance. I'm not saying LLMs aren't useful, but it's more like getting help with restaurant Spanish than being a fluent Spanish speaker.

            1 vote
    2. chocobean
      Link Parent
      Same. Cut out all the bridging that brings the young towards expertise, and eventually we'll have LLMs churn out and automatically test millions of versions of code to land on one with the fewest...

      Same. Cut out all the bridging that brings the young towards expertise, and eventually we'll have LLMs churn out and automatically test millions of versions of code to land on one with the fewest side effects. Lots of black boxes. Mystery code that seem to do a slight thing we want so let's go with that and try to mitigate side effects with other black box code that treat these symptoms

      And if our young people have to learn coding without best practices, but from trying to read meta meta meta meta code, it can still be done, except if no one is going to pay for it, very few people can afford the time to learn it. It'll be like any other craft lost to time.

      4 votes
  3. [3]
    LetsBeChooms
    (edited )
    Link
    If you're not writing the code yourself, you're not writing code. You're copy/pasting and mushing pieces together that probably don't fit or flow correctly. If this is going to be the new norm,...

    If you're not writing the code yourself, you're not writing code. You're copy/pasting and mushing pieces together that probably don't fit or flow correctly.

    If this is going to be the new norm, then it needs to have its own verb, because "writing code" and "combining code snippets" are not functionally nor effectively the same.

    7 votes
    1. Minori
      Link Parent
      The line between copying and editing from StackOverflow versus the documentation can be pretty thin.

      The line between copying and editing from StackOverflow versus the documentation can be pretty thin.

      7 votes
    2. vczf
      Link Parent
      Similar to photobashing.

      Similar to photobashing.

  4. bln
    Link
    I find this take on AI coding interesting, it’s not just about autocompleting boilerplate. Note: I submit with the first sentence rather than the page title because it’s more descriptive IMO.

    I find this take on AI coding interesting, it’s not just about autocompleting boilerplate.

    Note: I submit with the first sentence rather than the page title because it’s more descriptive IMO.

    3 votes
  5. [15]
    DefinitelyNotAFae
    Link
    Not being able to take that time off work, or be given more accessible tasks is really demonstrating what American employment is like- whether that's about the actual benefits themselves or the...

    Not being able to take that time off work, or be given more accessible tasks is really demonstrating what American employment is like- whether that's about the actual benefits themselves or the inability to use them.

    You can't even rest when your hand is broken.

    61 votes
    1. [5]
      infpossibilityspace
      Link Parent
      Exactly, this isn't the accomplishment the author thinks it is. I think it's telling that this mentality is so normalised that it's not even mentioned as a problem that shouldn't exist.

      Exactly, this isn't the accomplishment the author thinks it is. I think it's telling that this mentality is so normalised that it's not even mentioned as a problem that shouldn't exist.

      33 votes
      1. [4]
        DefinitelyNotAFae
        Link Parent
        Yeah and due to a convo in another thread, it's made me think that it's why pregnancy leave is such "problem" in the US, from discrimination against women applicants to actual issues at the...

        Yeah and due to a convo in another thread, it's made me think that it's why pregnancy leave is such "problem" in the US, from discrimination against women applicants to actual issues at the workplace. A pivot to parental leave doesn't even help because it's the pregnant person that pretty much has to take time off for recovery. But if you can push/expect people with broken limbs to work through it, and that is just normal, then that pregnant or formerly pregnant person out for six (unpaid) weeks is just whining and not as dedicated to their work. Sorry a bit tangential.

        Idk I have a colleague who has been living life on a knee scooter and I know she's feeling internally driven to be at work, but also we're all a)telling her to rest more, including going home and b) trying to adjust to meet her needs. But she could take more time off. I just wish folks would.

        18 votes
        1. [3]
          ShroudedScribe
          Link Parent
          Yeah, I've realized I shouldn't ask about benefits when interviewing. Ask about PTO? They might think you're a slacker. Ask about parental leave policy? They might think you're looking to use...

          Yeah, I've realized I shouldn't ask about benefits when interviewing. Ask about PTO? They might think you're a slacker. Ask about parental leave policy? They might think you're looking to use those benefits soon which wouldn't provide the greatest benefit to the company. Ask about health plans? They might think you have a chronic illness and won't be able to work as hard as other employees.

          These are mostly things that are illegal to discriminate against, but the reality is that companies can do so as long as they don't get caught.

          7 votes
          1. [2]
            teaearlgraycold
            Link Parent
            You might be able to get that info from GlassDoor. Or if you’re in a 1:1 interview and the interviewer doesn’t seem like a narc you could ask. If someone asked me about parental leave I’d keep...

            You might be able to get that info from GlassDoor. Or if you’re in a 1:1 interview and the interviewer doesn’t seem like a narc you could ask. If someone asked me about parental leave I’d keep that to myself.

            The interview training at Google specifically mandated interviewers steer interviewees away from those topics. Like, even if they mention something about their kid, just say “okay” and change the subject.

            1 vote
            1. DefinitelyNotAFae
              Link Parent
              Well kids are on the list of "don't ask or talk about " because they're a potential discrimination lawsuit flag. We like candidates to be authentic but I edit out when they share they did X with a...

              Well kids are on the list of "don't ask or talk about " because they're a potential discrimination lawsuit flag.

              We like candidates to be authentic but I edit out when they share they did X with a church group or whatever. We talk a lot about wanting to make sure our team takes their time, and we mean it. Interview and after. But I know not everywhere is like that. (See also my colleague and her scooter feeling like she gotta be there. See also me capping my vacation accrued. We're a work in progress) But it's about to be move-in so everyone is tired from training and about to be tired so time off will be after next week.

              2 votes
    2. chocobean
      Link Parent
      Code reviews could have been easy to switch to for two months. Unless his job is specifically to work on the product he talked about here. Then it would be valuable data to see how easy it is to...

      Code reviews could have been easy to switch to for two months.

      Unless his job is specifically to work on the product he talked about here. Then it would be valuable data to see how easy it is to use for accessibility reasons

      2 votes
    3. [8]
      skybrian
      (edited )
      Link Parent
      That’s making some assumptions. Some folks are pretty gung-ho and we don’t know his work situation. Maybe they would have made more accommodations if he had asked? I think it’s a good question...

      That’s making some assumptions. Some folks are pretty gung-ho and we don’t know his work situation. Maybe they would have made more accommodations if he had asked?

      I think it’s a good question though: what does Anthropic do in a situation like this? It’s not mentioned in the article at all, and it should be.

      1 vote
      1. [7]
        DefinitelyNotAFae
        Link Parent
        He said it "forced him" to write his code with AI. He may be choosing to work despite his broken hand, but is that his office culture or his actual choice. Why aren't his bosses encouraging him to...

        He said it "forced him" to write his code with AI.

        He may be choosing to work despite his broken hand, but is that his office culture or his actual choice. Why aren't his bosses encouraging him to take time off? Why isn't HR like "not til you're cleared for work?" Does he have the time off in the first place?

        Why doesn't he value his well being more? Or have idk hobbies that he likes to do when not at work that he could be doing?

        Being this "gung ho" is a symptom. And I say that as someone who recognizes the problem within myself.

        5 votes
        1. [2]
          ThrowdoBaggins
          Link Parent
          I’d love more than anything for some version of this article to exist, and the plot twist at the end is that they love coding and did it as a hobby while they were on paid leave the whole time....

          I’d love more than anything for some version of this article to exist, and the plot twist at the end is that they love coding and did it as a hobby while they were on paid leave the whole time.

          That this article isn’t that, I agree is definitely telling. Even if this person happens to have a healthy relationship with work and has options for workplace adjustments and paid time off; the article existing without those ideas being brought up makes it easier for some boss at some toxic workplace to read this and be like “oh cool, so injury isn’t even enough to keep the person from being productive” and enforcing the shitty version at their workplace.

          2 votes
          1. DefinitelyNotAFae
            Link Parent
            Yeah, I can't swear his situation is toxic, but it highlights the toxicity of employers in general, and possibly the tech sector in particular.

            Yeah, I can't swear his situation is toxic, but it highlights the toxicity of employers in general, and possibly the tech sector in particular.

            2 votes
        2. [4]
          skybrian
          Link Parent
          I asked about it on Twitter and here is what he said:

          I asked about it on Twitter and here is what he said:

          my manager encouraged me to take time off if I wanted, but I didn't just want to sit at home doing nothing

          1. [3]
            DefinitelyNotAFae
            Link Parent
            Yeah like I said.

            He may be choosing to work despite his broken hand, but is that his office culture or his actual choice. Why aren't his bosses encouraging him to take time off? Why isn't HR like "not til you're cleared for work?" Does he have the time off in the first place?

            Why doesn't he value his well being more? Or have idk hobbies that he likes to do when not at work that he could be doing?

            Being this "gung ho" is a symptom. And I say that as someone who recognizes the problem within myself.

            Yeah like I said.

            1. [2]
              skybrian
              Link Parent
              Uh, no, read that again. It doesn't confirm your suspicions. Specifically, you wrote: "Why aren't his bosses encouraging him to take time off?" And he said that his boss did encourage him to take...

              Uh, no, read that again. It doesn't confirm your suspicions.

              Specifically, you wrote: "Why aren't his bosses encouraging him to take time off?" And he said that his boss did encourage him to take time off.

              You're still welcome to your suspicions - just because he didn't say anything is wrong doesn't mean there's nothing wrong. But we don't have anything confirming it.

              1. DefinitelyNotAFae
                Link Parent
                My suspicions are not alleviated. I wrote multiple things. Being this gung ho is a symptom. Whether it's internalized or externalized. Why doesn't he have hobbies that he enjoys that aren't his...

                My suspicions are not alleviated. I wrote multiple things.

                Being this gung ho is a symptom. Whether it's internalized or externalized. Why doesn't he have hobbies that he enjoys that aren't his job that would keep him busy but might not hurt him? Is this coming from a work culture that nominally says to take your time but doesn't reward that? What about the tech sector in general?

                To an extent his individual response doesn't matter because this is a larger critique.

                But also, as someone who works when they shouldn't. Who literally today has been helping with move-in and moving things when I should not be doing that. Who exhausted myself and came home and had to exhaust myself for my partner again. My boss encourages me to take time off. But student affairs has a hustle culture problem. And I have internalized a "I'm not doing enough" attitude. I am recognizing this specific to response from a different sector and calling myself out in turn.

                1 vote