Two years ago, “AI engineer” meant someone training models. Today it usually means someone who builds applications around models that already exist: assistants that answer from company documents, tools that extract data from messy text, agents that take actions. That is the job this roadmap is about.
The order matters because it is easy to jump straight to agents and end up with something that demos well and fails on the third question. Here is the order I would follow.
What an AI engineer does, and does not do
An AI engineer designs, builds and runs systems that use language models. The work is mostly engineering: connecting a model to data, deciding what to put in front of it, checking the answers, and keeping the whole thing reliable and affordable.
This is different from a data scientist, who explores data and builds models, and from a machine learning researcher, who trains new ones. You do not need a mathematics degree to do this job well. You do need to be a good engineer.
Stage 1: Solid Python and API habits
You will spend your days calling APIs, handling failures, parsing responses and writing tests. If those feel natural, the rest is much easier. Learn to read documentation, handle timeouts and rate limits, and keep secrets out of your code. Getting your first LLM API key is a good first exercise.
Stage 2: Understand what an LLM is
You do not need to build one, but you should know what it does. A model predicts text from the text it is given. It has no memory between calls, it can be confidently wrong, and what you put in the prompt changes what you get out. How LLMs work internally explains it without maths, and ChatGPT is not just an LLM shows how much sits around the model in a real product.
Stage 3: Prompts and structured output
Learn to write instructions a model can follow, and to get answers back in a format your code can use. Reliable structured output is one of the most useful skills in this field, because everything downstream depends on it. See how to extract reliable JSON from LLMs.
Stage 4: Embeddings and retrieval
Most business applications need a model to answer from your data, not from its training. That is retrieval-augmented generation, or RAG. Learn what embeddings are, how documents are split and stored, how retrieval finds the right pieces, and why the quality of retrieval usually matters more than which model you choose.
Also learn when RAG is the wrong tool. RAG versus fine-tuning is a good decision guide.
Stage 5: Evaluation, the skill that separates demos from products
This is the stage most tutorials skip. A system that answers well in your five test questions tells you almost nothing. Build a set of realistic questions with known good answers, measure the system against them, and rerun it every time you change something. Evaluation of LLMs introduces the approach.
If you remember one thing from this article, make it this: you cannot improve what you do not measure.
Stage 6: Agents and tools
An agent is a model that can decide to call tools and act on the results. It is powerful and easy to over-use. Learn the building blocks first: tool calling, memory, and workflows that loop and branch. What are agentic workflows is a clear starting point. Then learn to add limits, so an agent cannot take an action it should not.
Stage 7: Ship it
A notebook is not an application. Learn to expose your system as an API, handle many users, log what happens, trace each step, control cost, and add guardrails for safety. Exposing an agent with FastAPI shows the smallest working version.
Projects that prove it
- A document assistant with evaluation. Answer questions from a set of documents, cite the sources, and report a score on a test set you built.
- A structured extraction tool. Turn messy text into clean records, and measure how often it is right.
- A small agent with limits. Give it two or three tools, tracing, and a clear list of what it may not do.
For each one, write down what you tried that did not work. That is what an interviewer wants to hear about.
Common mistakes
- Starting with agents. Without retrieval and evaluation, you are guessing.
- Judging quality by feel. Build a test set early.
- Ignoring cost and speed. A system nobody can afford to run is not finished.
- Reaching for AI when you do not need it. Sometimes a query or a rule is the right answer, and asking whether you even need AI saves a lot of effort.
Do not skip the data foundations
Every AI application depends on clean, well-structured, reliable data. If yours is not, no model will fix it. The data engineer roadmap covers that side.
If you would like guidance
The DataKern AI Engineering program follows this path: a RAG pipeline with evaluation, then a multi-tool agent with memory, guardrails and observability. Sessions are live on Saturdays in small batches. If you want to try it first, the Masterclass is one three-hour session, and you can compare all formats here.