How to Hire Engineers When AI Can Pass Your Coding Test

When Claude can generate production-quality code, what exactly are you testing in a technical interview? The old approach — give a candidate a problem, watch them solve it, grade the code — collapses when AI tools can produce the same output in seconds.

Mark Hay, CTO and co-founder of TextQL — a 40-person AI data analytics startup with roughly 15-16 engineers — has rebuilt his hiring process around a different question. Not “can you code?” but “what can you add that the AI can’t?”

The Net Improvement Framework

TextQL’s engineering interviews allow full AI usage. Candidates get a take-home assignment. They can use Claude, Copilot, or any other tool. No restrictions.

Then the evaluation team measures the delta.

“What’s the net improvement that they made over a solution that we assume was just 100% Claude Coded?” Hay explains. The baseline isn’t zero — it’s what a competent person could produce by pointing AI tools at the problem and accepting the output.

The signal is everything above that baseline. Did the candidate improve the architecture in a way the AI wouldn’t have suggested? Did they catch an edge case? Did they make a design choice that shows taste and judgment? Did they do something that makes the system “much more interesting,” as Hay puts it?

What This Tests For

The framework shifts hiring from testing knowledge to testing judgment. In the old model, you tested whether someone knew how to implement a binary search tree or design a REST API. Those are commodity skills now — any AI tool can produce them reliably.

The new test surfaces a different set of abilities. Can the candidate direct an AI tool toward a good outcome? Can they evaluate whether the output is correct? Can they make the architectural decisions that the tool won’t make on its own?

Hay’s take-home is intentionally open-ended. Instead of a specified problem with a right answer, candidates get a codebase and the instruction to “make some cool improvement that you think makes everything much more interesting.” This openness is the test — it measures whether the candidate can identify what matters, not just execute what’s prescribed.

The Broader Implication

This approach isn’t specific to TextQL or to startups. Any company hiring engineers in 2026 faces the same question: if the candidate can’t do something meaningfully better than the AI baseline, why are you hiring them?

The answer defines what engineering skill means now. It’s not typing speed, memorized algorithms, or syntax knowledge. It’s the ability to see the system that the AI can’t see — the architectural trade-off, the user experience implication, the edge case that only matters at scale.

Hay’s career advice for engineers reflects the same philosophy: “Try to operate at either end of the spectrum of AI maximalism, but not so much in the middle.” Push AI tools to their absolute limit to understand what they can do. Then separately, build deep understanding of systems by reading code without AI, building mental maps of how things actually work. The middle — sort of using AI, sort of understanding the code — produces mediocrity at both.

FAQ

How do companies evaluate engineers when AI can write code?

Some companies now allow full AI usage in take-home assignments and measure what the candidate adds beyond the AI baseline. TextQL’s approach: give candidates a codebase, let them use any AI tool, then evaluate the “net improvement” over what Claude or Copilot would produce alone. The signal is judgment, taste, and architectural thinking — not syntax.

What is the “net improvement” hiring framework?

The net improvement framework assumes a baseline: what a competent person could produce using AI tools alone. The hiring evaluation measures everything above that baseline — architectural decisions, edge case identification, design taste, and the ability to make a system meaningfully better. First used at TextQL, this approach works for any team hiring engineers in an AI-assisted environment.

Should coding interviews allow AI tools like Copilot and Claude?

Yes, according to several engineering leaders. Restricting AI tools in interviews tests a skill set that no longer reflects how engineers actually work. Allowing AI tools and measuring the delta above the AI baseline tests what matters: can the candidate direct AI effectively, evaluate its output, and make decisions the tool won’t make on its own?

What skills matter for software engineers in 2026?

The core engineering test has shifted from “can you write the code?” to “can you direct AI, check its work, and make the right architectural decisions?” Specific skills: evaluating AI-generated code for correctness and security, identifying architectural trade-offs, understanding system-level implications that AI tools miss, and producing creative improvements AI wouldn’t suggest.

How should engineers learn AI tools without losing fundamentals?

Train at both extremes, not in the middle. Spend dedicated time pushing AI tools to their limit — run multiple agents, automate aggressively, see what the tools can actually do. Separately, spend time reading code deeply without AI assistance, building mental maps of how systems work at the architecture level. The gym analogy: isolate each skill rather than blending them into mediocre practice.

What makes a good take-home coding assignment in 2026?

Open-ended assignments that test judgment, not knowledge. TextQL gives candidates a codebase and asks them to “make some cool improvement that you think makes everything much more interesting.” This measures: can the candidate identify what matters, make architectural decisions, and produce something meaningfully better than what AI alone would generate?

Is the traditional coding interview dead?

Traditional whiteboard and leetcode-style interviews are losing relevance because AI tools can solve those problems reliably. The interview format is evolving toward take-homes with AI access, system design discussions, and evaluation of how candidates collaborate with AI tools. The core question shifts from testing memorized solutions to testing engineering judgment.

How big is TextQL’s engineering team?

TextQL has approximately 40 total employees with 15-16 engineers — just under half the company. The team operates without dedicated data scientists or analysts, with data analytics distributed as a shared responsibility. Engineers contribute to the shared ontology as they ship features, blurring the line between software engineering and data engineering.