The AI Engineering Interview: How to Prepare and What to Expect (Part 2 of 2)

From building the project to walking into the room

In Part 1, Byron’s top advice was to build a project with a specific company and its customers in mind. Doing that forces you to research what the company’s customers need and what value the company adds, and that understanding shows up in the interview when you explain the problem you solved. It doesn’t guarantee a job, but it goes a long way if you get it in front of the company. This session picks up from there: once you have the interview, how do you get ready?

The old engineering role is gone

Byron opens with something he admits sounds like clickbait but has seen often enough to say with confidence: the engineering role from four years ago is gone and not coming back. He misses parts of it, like late nights finding clever coding patterns. But engineers have to accept the change and figure out what the role is now. Based on what Gauntlet hears from the companies it works with, he sees three things every company looks for in AI engineers: AI engineering skills, system design, and communication.

Evals come first

Byron deliberately lists evals first among AI engineering skills. Unlike unit tests, which only verify that code works, evals tell you whether changes to a prompt, a model swap, or a new model will hurt your users. Without them you’re going on gut instinct, and he has seen enough engineers add evals only to learn their gut was way off. At a high level, strong evals include a golden set (if these fail, you don’t release), pass/fail checks rather than scoring, a domain expert who defines what good looks like, LLM judges you check on and can trust, and release gates that make evals part of your CI/CD pipeline. For an example of evals done well, he recommends studying how Notion approaches them.

Reading code is now part of the interview

AI generates the code, so how do you know it works? The best engineers Byron knows can read code efficiently, but they also write clear instructions in their CLAUDE.md or AGENTS.md files and build good skills that shape how the code gets written in the first place. Better planning means they already know what to expect when it’s time to review. Companies are now testing this directly: interviewers hand you code and expect you to find errors and inefficiencies and then fix them.

Harnesses are the skill of 2026

If agents were the thing of 2025, Byron says harnesses are 2026, and he expects them to be table stakes next year. A harness is everything you build around an LLM to get useful outputs: evals, tracing and logs, tools, and guardrails. Writing a coding harness is a good place to start, as is automating any process from your work or life. He’s also seeing companies built on harnesses tuned to a specific role or industry. He uses Claude Code for coding and Grokbot for communications and pulling data from sources like Notion and Slack into reports, and keeps each harness doing the job it was built for.

System design starts with a North Star

Because engineers code less, they have more time to think about what they’re building and how it affects customers, and companies are asking them to think like system designers. Many engineers never had to worry about a million requests a minute or traffic spikes, so this is new ground for a lot of people. Byron’s starting point is the North Star metric: the one measure that proves what you built is working, supported by others like latency. If your goal is customer engagement with a chatbot, engagement is the North Star and round-trip response time is a supporting metric. From there you prove impact, and if something isn’t working you pivot. Only then do you get to the architecture: where the cache goes, the database behind it, the load balancer, the services. Then ask what breaks first and how you’ll handle a spike without crashing. You don’t have to be the CTO, but companies want engineers who can help guide those decisions, and they’re asking about it in interviews.

Learning system design with a Claude skill

Byron built a system design learning skill you can load into Claude Code, Claude Desktop, or another chat interface. It figures out where you are, teaches concepts, and mixes in case studies and quizzes to check what you’ve retained. What he likes most is being able to stop at any point and ask questions, like how a concept relates to something he’s seen in his own work. He still reads books, which he finds better for breadth, while the skill helps him go deep and lock in details.

Communication is where most engineers fail

You can be a great engineer, but if you can’t get your ideas across, no one will know it, and no one will want to work with you. Communication matters even more now because companies need fewer engineers, so they hire the ones who communicate well. Byron’s most impressive interview ever was a candidate who said he had written a CSV parser. It sounded like a school project until he explained the problem: the company’s biggest paying client was trying to load 500,000 rows of data and it broke every time. The way he explained the problem and how he solved it made it clear he was someone Byron wanted on his team. A good answer covers the what, the why, the trade-offs, how you measured it, and what breaks.

Telling your story with the STAR method

The STAR method keeps your answers focused and stops you from going off on tangents. Situation: make them feel the pain, in a minute or two at most. Task: what you were responsible for. Say “I,” not “we.” Humility is good, but in an interview they want to know about you. Action: where you spend most of your time, covering how you solved the problem and the new problems that came up along the way. Result: short and concrete, like loading all 500,000 rows and keeping the client. Byron cares most about the action, but only if you nailed the situation. An alternative structure covers the problem, the architecture, trade-offs, results with real numbers, what broke or would have broken, and what you’d change at scale. That last question tests whether you think ahead. You want that answer ready long before the fire hits, not worked out in the moment.

Practice out loud, don’t memorize

Communication isn’t a gift some people have and others don’t. It’s a muscle. Practice out loud, in a mirror or with a spouse, friend, or parent. Work on eye contact, check whether your story makes sense, and make your mistakes now so you can fix them. Don’t memorize your stories, because memorized answers come off as rote and hurt more than they help. Know the key points you want to hit and make sure you can check each one off.

Resources to prepare

Byron shared a set of AI engineering learning skills for attendees. The system design skill is used at Gauntlet to get people up to speed on system design assignments. The AI interview coach asks you questions across topics like evals, harnesses, and reading code, gives feedback on each answer, and shows you where your gaps are. You can pause it to dig into a topic, then come back and try again. The prep guide is a large markdown file of likely interview questions, useful for self-study or for handing to a friend running a mock interview. If you want to know what to learn next, Byron says the topics in these resources are what companies interview on, and they’re also what you’d be expected to do on the job.

Q&A highlights

Asked how a forward-deployed engineer differs from an AI strategist, Byron describes the FDE as a product manager and software engineer in one: someone who works with different parts of a business, listens to customer needs, builds, and iterates on feedback. An AI strategist likely stays more on the technical side, working out what pieces are needed to build something like a new chatbot interface. Asked how to get experience with enterprise AI and eval tools while job searching, he suggests free or cheaper tiers, such as Datadog’s or Cloudflare’s, and notes that the concepts carry over even when the exact tool doesn’t. Asked whether learning these tools as a non-engineer is a stretch, he says no: he expects every other part of the business to grasp harnesses within two years the way engineers are now, and many companies already struggle because everyone uses AI but no one shares skills or frameworks. Asked for harness resources, he points to LangChain, whose writing on harness principles from years ago describes what nearly every harness looks like today, and says he’ll add harness material to the shared repo.