It turns out interviewing was broken long before I learned the trade, and despite the many attempts to band-aid it, it’s still broken today. It has managed to survive in spite of that. But it is finally dying on its own. People are a bit unclear on what’s next, so we’ll talk about some of our options.
[...]
We’ve already established that the gold standard of assessment is working directly with someone on real work, in a real environment, for as long as needed to make the call.
So the short answer is, stop simulating. Post real pieces of work, let the candidate do it, look at what they actually produced, and decide from that work.
[...]
If you spend any time doomscrolling on X, the new meta in SF is “come work with us for a few days.” Paid, real codebase, real ticket, real team. From what I’ve heard, it generates the best signal people have ever seen, by a mile. I don’t think it beats a six-month co-op, but it’s a hell of a lot better than a standard interview loop.
So there’s already a movement away from traditional interviewing, which is why I feel comfortable making the claim that it’s already starting to die out on its own. It’s being replaced by bringing people in for longer stints. I’ve been calling this working model the campfire: pull up a log, build something together, see how it feels. Bringing in outside contributors is pretty easy, as they’ll have agents and should be able to come up to speed very quickly. If they can’t, there’s your signal.
But even though people are experimenting with it, the way it’s done today is a mess. Every shop is reinventing a bespoke solution. And it’s unportable: candidates do brilliant work in a trial, they don’t get the offer for reasons, and the signal evaporates. The next company they campfire-interview at starts them at zero.
OK, but is that really how it works? Does doing mountains of free work actually correlate with your likelihood of getting hired in the next interview? I hear people have to send out hundreds of...
working directly with someone on real work
a six hour investment in a permanent small reputation bump
OK, but is that really how it works? Does doing mountains of free work actually correlate with your likelihood of getting hired in the next interview? I hear people have to send out hundreds of CVs to even have a chance of getting hired these days. Even just a hundred interviews with "6 hours of work" as part of the process would be more than two months' worth of unpaid work, full time...
Should we maybe figure out a way to do the not-simulation once and share the results with all interviewers? Or at least for all sufficiently similar job applications.
I think the idea is that it should be paid temp work, like a paid internship but shorter. More like a gig? But if it were possible to build software with temp workers, why isn't someone doing it?...
I think the idea is that it should be paid temp work, like a paid internship but shorter. More like a gig?
But if it were possible to build software with temp workers, why isn't someone doing it? Maybe it's more feasible now with AI?
How would anyone with a job do this? I don’t doubt that you get great signal about the candidate, but you also filter out everyone who already is employed. Which is probably the more talented...
If you spend any time doomscrolling on X, the new meta in SF is “come work with us for a few days.” Paid, real codebase, real ticket, real team.
How would anyone with a job do this? I don’t doubt that you get great signal about the candidate, but you also filter out everyone who already is employed. Which is probably the more talented segment of the potential hiring pool. That seems counterproductive.
Like if you’re a hot, young AI startup you’ve already filtered out anyone already at OpenAI or Anthropic or Deepmind or any of the other startups (Thinking Machines, etc).
Trying to compete with those companies for workers seems kinda tough, though, so maybe the moneyball approach would be not to try and look for talent that hasn't been recognized yet?
Trying to compete with those companies for workers seems kinda tough, though, so maybe the moneyball approach would be not to try and look for talent that hasn't been recognized yet?
I'm going to put my skeptics hat on and highlight this portion: Nowhere but to accept precarious, at will, provisional employment under any sort of fancy name: co-op, internship, probation, temp,...
I'm going to put my skeptics hat on and highlight this portion:
If provisional employment is the answer [...] then why aren’t more companies requiring internships before making their hiring decisions?
The reason is, hiring engineers has historically been so competitive that you couldn’t convince a senior engineer to do an internship. [...] they’ll go elsewhere.
[...] but if a sizable slice of the skilled-labor pool ends up “between gigs” at the same time, there may be no elsewhere for them to go
Nowhere but to accept precarious, at will, provisional employment under any sort of fancy name: co-op, internship, probation, temp, campfire... Whatever you call it.
And then, importantly, each work item also counts once for the candidate: they walk away with a permanent, portable record of what they did and how well they did it, signed by you, whether or not you make an offer.
Walking away with anything except an offer still only means you're waving around a "no hire" failed campfire flag. It's exactly the same as being asked why didn't they hire you at the end of the internship.
Companies whose stamps mean anything will be the ones known for only handing them out for quality work. Being a hard, fair judge is an advantage for everyone.
This means not everyone walks away with that shiny One Piece, only star performers. So the rest would have done barely paid unprotected temp work they don't get to show off like they said.
So, Mechanical Turk for engineers. Gigwork. Sure enough that's what was mentioned next.
Walking away with anything except an offer still only means you're waving around a "no hire" failed campfire flag. It's exactly the same as being asked why didn't they hire you at the end of the internship.
The best tech interview I ever did was an on site interview on a desktop pc in a small window office with the whole stack live on it and the interviewer told me to figure out what variable “blah”...
The best tech interview I ever did was an on site interview on a desktop pc in a small window office with the whole stack live on it and the interviewer told me to figure out what variable “blah” was for.
This was in like 2015 and I didn’t get the answer but i tried my best and talked through my thinking and I got the job.
The world had changed with these remote interviews. Way too easy to cheat and steal company code with remote interviews.
First job i had in software did this. Interview process started with a 2hr paired activity where you worked with another candidate on some busy work and an interviewer listened. Rotate 3 times, go...
We’ve already established that the gold standard of assessment is working directly with someone on real work, in a real environment, for as long as needed to make the call.
First job i had in software did this. Interview process started with a 2hr paired activity where you worked with another candidate on some busy work and an interviewer listened. Rotate 3 times, go home. Nobody checked the work, it was just an eval of how people interacted with others. Incredibly effective. Second step was 1 full day of pairing on real work, paid. Followed by a 3 week trial.
The 3 week bit killed it for a lot of potentially good candidates, people have families to feed. But the rest of the process was always very revealing. Watching people go from professional and respectful to impatient, rude, and/or argumentative in under an hour as they work alongside you was always fascinating. Absolutely worth the handful of hours to find out if was a bad fit than have to put up with a bad coworker for months or years.
Imagine my surprise when I finish this article and notice the author, Steve Yegge, who created what was probably my favorite PC game of all time: the early 2000's graphical MUD called Wyvern....
Exemplary
Imagine my surprise when I finish this article and notice the author, Steve Yegge, who created what was probably my favorite PC game of all time: the early 2000's graphical MUD called Wyvern.
Unfortunately, after abandoning the game for many years, a time during which it flourished under the care of a few independent devs, Steve came back around 2020, alienated the very people who kept it going all this time, ported it to mobile and subsequently ran it into the ground and then disappeared again. These days I believe the servers are still up but there is nobody playing most of the time. I guess Steve is too busy writing articles like this one that just come off as kind of icky.
From the article:
[...]
[...]
OK, but is that really how it works? Does doing mountains of free work actually correlate with your likelihood of getting hired in the next interview? I hear people have to send out hundreds of CVs to even have a chance of getting hired these days. Even just a hundred interviews with "6 hours of work" as part of the process would be more than two months' worth of unpaid work, full time...
Should we maybe figure out a way to do the not-simulation once and share the results with all interviewers? Or at least for all sufficiently similar job applications.
The author is not recommending volunteer temp labor.
I think the idea is that it should be paid temp work, like a paid internship but shorter. More like a gig?
But if it were possible to build software with temp workers, why isn't someone doing it? Maybe it's more feasible now with AI?
How would anyone with a job do this? I don’t doubt that you get great signal about the candidate, but you also filter out everyone who already is employed. Which is probably the more talented segment of the potential hiring pool. That seems counterproductive.
Like if you’re a hot, young AI startup you’ve already filtered out anyone already at OpenAI or Anthropic or Deepmind or any of the other startups (Thinking Machines, etc).
Trying to compete with those companies for workers seems kinda tough, though, so maybe the moneyball approach would be not to try and look for talent that hasn't been recognized yet?
Why would anyone do that? Who is still even using the trash heap formerly known as twitter...and why? Steve is, I guess, but surely nobody else.
I'm going to put my skeptics hat on and highlight this portion:
Nowhere but to accept precarious, at will, provisional employment under any sort of fancy name: co-op, internship, probation, temp, campfire... Whatever you call it.
Walking away with anything except an offer still only means you're waving around a "no hire" failed campfire flag. It's exactly the same as being asked why didn't they hire you at the end of the internship.
This means not everyone walks away with that shiny One Piece, only star performers. So the rest would have done barely paid unprotected temp work they don't get to show off like they said.
So, Mechanical Turk for engineers. Gigwork. Sure enough that's what was mentioned next.
Ya this is where the article lost me as well.
The best tech interview I ever did was an on site interview on a desktop pc in a small window office with the whole stack live on it and the interviewer told me to figure out what variable “blah” was for.
This was in like 2015 and I didn’t get the answer but i tried my best and talked through my thinking and I got the job.
The world had changed with these remote interviews. Way too easy to cheat and steal company code with remote interviews.
First job i had in software did this. Interview process started with a 2hr paired activity where you worked with another candidate on some busy work and an interviewer listened. Rotate 3 times, go home. Nobody checked the work, it was just an eval of how people interacted with others. Incredibly effective. Second step was 1 full day of pairing on real work, paid. Followed by a 3 week trial.
The 3 week bit killed it for a lot of potentially good candidates, people have families to feed. But the rest of the process was always very revealing. Watching people go from professional and respectful to impatient, rude, and/or argumentative in under an hour as they work alongside you was always fascinating. Absolutely worth the handful of hours to find out if was a bad fit than have to put up with a bad coworker for months or years.
Imagine my surprise when I finish this article and notice the author, Steve Yegge, who created what was probably my favorite PC game of all time: the early 2000's graphical MUD called Wyvern.
Unfortunately, after abandoning the game for many years, a time during which it flourished under the care of a few independent devs, Steve came back around 2020, alienated the very people who kept it going all this time, ported it to mobile and subsequently ran it into the ground and then disappeared again. These days I believe the servers are still up but there is nobody playing most of the time. I guess Steve is too busy writing articles like this one that just come off as kind of icky.
Bane of my existence, good riddance!