Our software engineering interviews are changing a little

so, i've gone through a bunch of iterations on how to interview in days of LLMs and i came up with a process that is quite good:

first, a coding problem that is solved live (right now, online) -- it's not a hard one and most LLMs can easily one shot it.

BUT: if you are not a senior developer, you need to do it without LLM, and if you are a senior+ and do it via LLM, the bar is super raised and now you need to explain in details everything the LLM wrote as if you wrote it yourself.

the main component here is to know if the candidate understands coding in general, understands problem solving and how to dive into an existing (although quite small) codebase. solving the problem is not really the goal -- in fact, most candidates we hired did not solve it 100%, but were able to understand and explain everything that they put it on code. this usually takes 1h30.

(we also ask the candidate to open a PR against the repo just to see if they know the basics of git -- you would be surprise how many people cannot do it).

the second is an architecture interview, where we build a system together BUT instead of doing the whole spec, miro board, whatever, we do it in more of a free way: we start with a really simple, MVP level, spec and we ask how the candidate would build it. then, we add another feature, and we ask how they would go for it. during this whole thing, i ask random questions like "why would you use x instead of y?" or "why do you prefer this approach?".

one important thing: we allow the candidate to say "i don't know" and we tell the candidate that there's no gotcha questions (which is true, i hate these kind of questions). this lets the candidate be super comfortable (feedback we had from real people).

in general, this flow helped us a ton and i really really like running it :)

(edit: spelling)

Does it work? How do they know it works?

There are a lot of people telling you how they interview and why. Almost none of them have shown even the slightly indication that they are aware there there is scientific research on what works for an interview, much less that they apply it. They also are not showing any indication that they have done any testing - to see if their questions help hire better people.

If I said they could get equal results by picking resumes at random and hiring them without any other screening could they refute that? If so have they justified every part of their process or just one step that is useful and the rest they don't know?

This is the third Jane Street-related thing I've seen on here this week after not seeing much about them that I can remember any time recently beforehand I'm aware of their use of OCaml and probably do other things that might be of interest to the HN community, but it seems like a lot in a very short time out of nowhere. Is there some sort of marketing push or something going on?

This is a completely side comment, but I don't understand why stories about Jane Street keep making the front page day after day:

Today: https://news.ycombinator.com/item?id=50467297

Yesterday: https://news.ycombinator.com/item?id=50455394

Two days Ago: https://news.ycombinator.com/item?id=50438261

It's a bit odd (?!).

My guess is that they'll end up hiring the same people regardless of the interview format. They want to hire top graduates from elite institutions anyway.

A friend recently got through Google after getting interviewed in Meta and several other companies. He said interview styles are changing in most places but there is no common pattern. Every company is taking their own custom approach. Some are giving open access to AI IDE, some are asking to not ask everything and monitoring what you ask. One company give access to LLM (probably custom) that would provide wrong code intentionally and candidate has to correct it.

Pretty vague blogpost tbh. Apart from the fact that it's changing, it doesn't really describe much

I'll be sad or maybe it's just a transition of an era that I just need to adapt when SWE interviews no longer write code on a whiteboard.

Perhaps one day, interviews will require candidates to "bring your own harness/prompt/skill/mcp"

Code Interviews are a big waste of time for the employee and the employer. look what the human can bring to the table.

This sort of announcement has made me very happy I left the software world. I still code as part of my job, but it's not my main part and luckily I don't have to use LLMs are part of my job. I still code 100% by hand.

In some ways, although I detest AI, it has clarified the sort of people and organizations I want to associate with. Any latent characteristics that I abhor have been made obvious with the existence of AI and of course, I would want nothing to do with Jane Street.

Back then, people who were bad at interviews were often undervalued. I think interview culture itself will soon be devalued and eventually become obsolete. People who build real technology and solve real problems will be unaffected, while the glorified Valley slop built around interview performance will become obsolete.

My optimism comes from AI's ability to solve genuinely hard mathematical problems this year. September and October could go down as two of the most remarkable months in the history of AI If, and only if, those proofs are correct and independently verified by real experts. Even if the proofs have been formally verified in Lean, expert scrutiny still matters.

This will always be the shortcoming of how "knowledge work" sort of job interviews put so much focus on trying to get the exact unicorn candidate from the outset, instead of just seeing what happens and either keeping the worker or letting them go like how every other job works. Most shift work on the other hand doesn't have to worry about you lying on your resume or interview. If you lie, and you can't do the job the next day, you are fired. That's it. If you lie, and you can do the job the next day, all is well anyhow, right?

It is like an unhackable way to do it. Just hiring the person, seeing how it goes, and letting them go or not. It isn't rocket science. The guys digging a ditch outside the office know more about hiring than the career HR professionals inside I guess.

Jane Street's Addiction

What's up with the obsession with Jane Street today ?

The linked article has almost no info to take away besides being so wordy.

"interviews in our onsites for campus candidates."

ie. Not actually interviewing software engineers.

I see much buzz lately about Jane Street. Probably they had a good year and now they’re looking for fresh meat, trying to expand even more.

employing humans? wow

Use AI or not, quite being half in half out.

the thought of going into an interview and getting asked to write code is a joke at this point. i haven't written more than a handful of lines in over a year. the interview should be related to the job.

did you get paid for that slop?