alt Hacker NewsThe problem with work-sample testing (which is commonly administered as a take-home problem for the developer candidate to solve) is two-fold:
a) it discriminates against people who cannot spare 4+ hours of focused time on evenings/weekends to work on the problem. People with multiple jobs, single parents, etc.
b) in the age of AI it is no longer a reliable measure of someone's skill, for obvious reasons
Unlike Yegge, I haven't worked at FAANG, but the companies I have worked at all followed the same hiring practices and suffered from the same problems as he describes.
Provisional employment (or, if that's not possible, then well-paid internships) solve all of those issues. The candidate gets 3-6 months of stable employment, you as the employer get a large number of work-sample tests, and you can see how they use AI and how much.
Replies
I think a very real problem is that these take home problems are often way more than 4 hours. And to that they often add the traditional 4-6 hour interview loop.
Provisional employment doesn’t work for most cases, though. It might attract people who have no job and it might attract people who have so much saved that they are okay with potentially being let go after 90 days. But I imagine the vast majority of the potential employment pool are not willing to quit their current job to accept a “maybe” job from another company.
A standard interview loop kills an entire work day, and is preceded by phone interviews that eat several hours. Properly budgeted work samples are strictly better from the candidate's time perspective, not to mention that you can do them from your couch rather than under flourescent lights in a confeence room.
The AI thing is an interesting problem, but a solvable one. We continue to hire resume-blind.
We do all our work samples in person at a our offices as part of the in person interviews. Takes 2-3 hours, never been a problem so far.
If you are going to take a day off to do 3-4 in person interviews at a company then this slots in well.
>The candidate gets 3-6 months of stable employment
To me, "stable" implies I don't even worry about having a full-time job for the next 3-5 years. Anything less isn't stable, unless this would be my second job immediately after the McDonald's burgerflipper one where I was scheduled for random 4 hour shifts seemingly designed to maximize personal inconvenience.
If I knew that my job would only last 6 months, I would have to immediately begin conducting a job search and prepping for it. I put my odds at finding another job before the 6 months is up at less than 60%. Even that number seems naively optimistic now that I've typed it out. Offering me 3-6 months of employment only sounds like a deal if I'm currently unemployed and rent was due yesterday.
And coding interviews often bias against people who can’t spend days of time grinding on hackerrank or whatever to prepare for the interview. Provisional employment biases against people who currently have jobs.
There is no perfect interview process, so it’s important companies think about what they’re biasing for or against. A work sample gives a more realistic picture of someone’s ability than a coding interview, but less than provisional employment. But, it’s much easier for people to commit to finding the time for it than to spending 3-6 months in limbo.
Regarding AI and work samples, it seems like this problem is largely solved by having an in-depth conversation with the candidate about the assignment, no? They should be able to explain why they made the decisions they did, pros and cons of their approach, etc. If they can do that, and their solution is good, does it matter if it was LLM-assisted?