A few years ago, IT hiring was the easiest thing on the calendar.
The engineering side would hand the recruiters a tidy rubric — a handful of theory questions, a few tricky ones with traps baked in — discriminative enough to run the early rounds. Answer smoothly and you moved on; don’t, and you were out. Only the final round needed a dev manager, or a solid engineer, to judge for real.
Then the AI era arrived, and every candidate suddenly “knew.” That is exactly when hiring got hardest.
The old rubric rested on an assumption that no longer holds
The old rubric worked because of a quiet assumption: whoever answers correctly understands, and whoever doesn’t understand answers wrong — or stays silent.
That used to be true. To talk about microservices, idempotency, or eventual consistency, you had to have read, built, and wrestled with them. Knowledge was evidence of effort.
Not anymore. One prompt yields an answer good enough to clear the first round — not because the candidate understands it, but because AI wrote something that sounds like understanding.
Impressive on the surface, disjointed underneath
These answers sound heavy: dense technical vocabulary, complex sentences, the rhythm of a rehearsed lecture. A non-technical interviewer can easily be overwhelmed.
But pull them apart and they don’t hold together:
- Disjointed. Each term sits in the right place, but nothing connects them into an argument. The next answer undercuts the previous one, and the candidate never notices.
- Thin on practice. They can recite the trade-offs between two architectures, then stall on “so which one should your project pick, and why?” Ask “when did you last hit this?” and the answer drifts somewhere else.
- Mechanical. One formula applied to every problem, with no sense of which ones need a careful trade-off and which just need to ship.
That is rote knowledge. It sounds like an expert with no experience standing behind it.
The interviewer is stuck between two things they cannot verify
The problem isn’t the candidate. It’s that the person across the table has no way to tell the difference.
The old rubric was simple because it assumed wrong answers were obvious: a wrong concept, missing vocabulary, an answer that wandered off. Now the right answers are everywhere — and the wrongness hides deeper, in the reasoning, not in the words.
Judging that takes exactly the expertise the recruiting team doesn’t have. Score against the rubric and everyone passes. Push back until it falls apart and you need someone who knows the craft — usually busy, and parked at the final round.
The result: the filter runs backwards. The strong candidate and the rote one walk through the same door, and only at the last, most expensive stage does anyone separate them.
Move the filter, don’t just add questions
A few more tricky questions won’t fix it. AI handles those too. The filter has to move.
One simple move that works: ask straight into the use case of each technical feature. Don’t ask “what is idempotency” — AI answers that in three seconds. Ask: “in your system, where does this belong, and what breaks if you remove it?” A definition can be copied; a real use case has to be understood before it can be produced. That is where mindset, creativity, and practical judgment show through — the things no amount of definition-searching with AI can fake. And here is the good part: the interviewer doesn’t need to be technical, only insistent on getting a concrete example.
A few other things to move along with it:
- Score what they can do, not what they know. A small exercise, a snippet to read, a bug to fix, a choice to defend. Rote knowledge can’t simulate the act of building.
- Bring technical judgment in earlier. No longer “the dev manager’s final round” — the first one, even if only for a few minutes.
- Ask follow-ups. Don’t stop at the correct answer. Ask “why,” ask “what if the constraints change,” and press on the exact point where the story contradicts itself.
- Ask for concrete experience. A system that went down, a time they argued the team out of the wrong call. People who have done it tell it in specifics; rote candidates tell it in concepts.
- Let “I don’t know” be an answer. Sometimes the clearest signal isn’t the answer — it’s how someone faces what they haven’t met yet.
One more stage: the test
The IT test used to be about syntax and algorithms: write the loop, remember how to reverse an array, solve a few puzzle problems. And because the test is usually done live, being able to do it on the spot was taken as proof of real ability.
The AI era flips that. Syntax and algorithms are no longer worth testing — once someone is hired, AI writes that part for everyone. Weak or strong, these tasks get handled in seconds. Making them the test means screening for a skill the job itself is about to stop needing.
What is still worth testing is the work AI can’t do for you:
- Hand them a technology and demand a use case. Pick exactly one thing — a message queue, a cache, idempotency, event sourcing — then ask: in a system you’ve actually worked on, where does this belong, what problem does it solve, and what breaks if you remove it. AI can answer “what it is”; only someone who has been through it can answer “when to use it, and why not the other option.”
- Test classification and architectural thinking. Vibe coding makes AI spray code everywhere — new functions, new classes, another abstraction layer at every turn. Without classification thinking (what belongs where, what splits out, what merges back) and architectural thinking (boundaries between parts, data flow, where it breaks), that output rots into garbage — it runs once, then quietly decays. Hand out a problem big enough to force decomposition, then watch where they draw the boundaries and how they group things.
And I’d let candidates use AI — the base model, with a time limit attached. Anyone dependent on it gets carried away: accept one suggestion, patch the next spot, and the clock runs out before they’re done. Someone who actually knows the work uses it as a second pair of hands and still finishes in time.
There is no trick here. Just an uncomfortable one: when everyone can talk, the only thing left that sorts people is what they have actually done.
If you are hiring for IT in the AI era and your old rubric is starting to slip, send us a sentence — we are wrestling with the same problem.
