What does an AI engineer do?
An AI engineer builds software products and internal tools powered by foundation models. Day to day, that means:
- Designing prompts and structured outputs that behave reliably.
- Building RAG pipelines so models answer from company data.
- Wiring models to tools and APIs — increasingly as agents and via MCP.
- Writing evaluation suites so quality is measured, not guessed.
- Shipping to production: latency, cost, monitoring, security and guardrails.
Most AI engineers rarely train models from scratch. The hard problems are integration, data quality, evaluation and reliability.
AI engineer vs ML engineer vs data scientist
| AI engineer | ML engineer | Data scientist | |
|---|---|---|---|
| Core work | Build apps on foundation models | Train, deploy and maintain ML models | Analyse data, experiments, predictive models |
| Key skills | APIs, prompting, RAG, agents, evals | ML frameworks, feature pipelines, MLOps | Statistics, SQL, experimentation, ML |
| Typical output | AI assistant, extraction pipeline, agent | Fraud model in production | Insights, forecasts, A/B test results |
| Maths depth | Moderate | High | High (statistics) |
| ailearnist path | LLM App Developer | ML Engineer / MLOps | AI Business Analyst |
The AI engineer roadmap: 6 stages
Stage 1 — Engineering foundations (3–6 weeks)
Python (functions, classes, async basics, virtual environments), HTTP APIs and JSON, Git, SQL, and basic testing. If you already code professionally, skim and move on.
Ship: a CLI script that calls a public API, cleans the JSON and writes it to a database.
Stage 2 — LLM fundamentals (2–3 weeks)
How LLMs generate text, tokens and pricing, context windows, temperature and sampling, hallucinations, and the trade-offs between model sizes and providers. Call two or three model APIs directly — before touching any framework.
Ship: a small app that sends the same task to two models and compares quality, latency and cost.
Stage 3 — Prompt engineering and structured output (2–3 weeks)
Roles, few-shot examples, delimiters, chaining, and JSON-schema structured output validated in code. See the prompt engineering guide and practise in the Prompt Builder.
Ship: a document-extraction pipeline (e.g., invoices → validated JSON) with a 30-example test set.
Stage 4 — RAG and embeddings (4–6 weeks)
Embeddings, vector databases, chunking, hybrid search, re-ranking, metadata filtering, and retrieval evaluation (recall@k, faithfulness). Understand RAG vs fine-tuning.
Ship: a cited Q&A assistant over 100+ real documents with an evaluation report comparing two chunking strategies.
Stage 5 — Agents and tool use (3–5 weeks)
Function/tool calling, the agent loop, memory, planning, multi-agent patterns, MCP servers, and guardrails: approvals, budgets, prompt-injection defences.
Ship: an agent with 3–4 tools (e.g., SQL, docs search, ticket creation) and human approval for write actions, plus traces of real runs.
Stage 6 — Production and LLMOps (ongoing)
Evaluation suites in CI, observability and tracing, caching and cost control, latency optimisation, model routing, security reviews, deployment (containers, serverless), and basic fine-tuning awareness.
Ship: deploy one of your projects publicly with logging, cost tracking and a regression eval that runs on every change.
Interactive skills checklist
Portfolio projects that get interviews
| Project | Proves | Make it stand out |
|---|---|---|
| Policy / docs Q&A assistant | RAG, chunking, citations | Publish an eval table comparing two retrieval setups |
| Support ticket triage | Classification, structured output | Show accuracy vs a labelled set and cost per 1,000 tickets |
| Text-to-SQL analyst | Tool use, guardrails | Read-only access, query validation, refusal on unsafe requests |
| Research agent | Agents, multi-step reasoning | Traces, step budgets, cited final report |
| Custom MCP server | Integration, protocols | Open-source it with clear tool descriptions and tests |
Need realistic briefs? AI on the Job has work-style tickets — claims document Q&A, text-to-SQL guardrails, support triage — with sample datasets.
A realistic 6-month plan (≈10 hours/week, already coding)
| Month | Focus | Milestone |
|---|---|---|
| 1 | Foundations refresh + LLM fundamentals | Model comparison app |
| 2 | Prompting + structured output | Extraction pipeline with test set |
| 3 | Embeddings + RAG basics | Working RAG assistant |
| 4 | Advanced RAG + evaluation | Eval report with measurable improvement |
| 5 | Agents, tools, MCP | Guarded agent with traces |
| 6 | Production + job prep | Deployed project · interview prep · certification |
Mistakes that slow people down
- Starting with frameworks. Learn raw API calls first; frameworks make sense once you know what they abstract.
- Tutorial hopping. Ten half-finished notebooks prove less than one deployed, evaluated project.
- Skipping evaluation. “It looked good on three questions” is the most common reason AI projects fail in production.
- Over-investing in theory. You don’t need to derive backpropagation to ship a great RAG system.
- Ignoring cost and security. Employers ask about both — have answers.
Discussion