48 articles · 8 stages · free
AI & LLM engineering — a reading order
Everything on this blog about building with language models, arranged as a path rather than a feed: what the model is doing, how to talk to it, how to give it your data, how to let it act, and what it takes to run one in production.
If you want the computer-science side instead — operating systems, data structures, databases, distributed systems — that is taught in degree order at /cs. Everything ever published is listed in the archive.
01What the model is actually doing
Start here. Almost every surprising LLM behaviour makes sense once you can picture the next token being chosen.
- 01Next-Token Prediction: How an AI Actually Writes Text (Not Magic — Just Probability)An LLM never sees a finished sentence. It answers one tiny question, over and over: given everything so far, what's the next token? It builds a probability distribution over the whole vocabulary, samples from it (not always the top score), glues the winner on, and re-runs from scratch. That loop is the entire engine — and it explains why the same prompt can give two different answers, what 'personalization' really is, and where hallucinations come from.
- 02Transformers — The Architecture That Changed AI (Part 1 of 3)A deep dive into the Transformer architecture — from attention mechanisms to self-attention, multi-head attention, positional encoding, and why this single paper reshaped all of modern AI.
- 03Perplexity — the one number that says how surprised the model isPerplexity measures how surprised a language model is by real text—expressed as an effective number of equally-likely token choices. Learn the math from scratch: probability → surprise → cross-entropy → perplexity, and why lower is better only for fluency, not correctness.
- 04Chain of Thought — why 'think step by step' actually worksChain-of-thought prompting works because it buys your model more forward passes and external scratch space, not because it tries harder. Learn the mechanism from first principles.
- 05LLM Model Types — Reasoning, Thinking, and BeyondA practical guide to understanding the different types of language models: base models, instruction-tuned, reasoning models, thinking models, and MoE architectures — what they are, how they differ, and when to use each.
- 06Mixture of Experts (MoE): How a 400B Model Runs Like a 40B OneMixture of Experts (MoE) lets a 400B-parameter model compute like a 40B one by routing each token to only 2 of 8 expert sub-networks. Learn how sparse routing trades memory for compute efficiency.
- 07Vision Transformers — How Transformers Learned to See (Part 2 of 3)From CNNs to Vision Transformers — how splitting images into patches and applying self-attention revolutionized computer vision. A deep dive into ViT, DeiT, Swin, and the models that followed.
- 08Vision Language Models — When AI Learns to See and Talk (Part 3 of 3)The final piece: combining vision and language into unified models. From CLIP to GPT-4V, LLaVA, and Gemini — how VLMs understand images and text together, and why this changes everything.
02Prompting, and why output breaks
The part everyone does first and almost nobody does deliberately — including the failure modes that look like model problems and are not.
- 01Anthropic's Prompt Engineering Best Practices, DistilledAnthropic published one consolidated page covering everything they recommend about prompting Claude. It's long, mixed with model-migration trivia, and easy to skim past. Here are the techniques that survive past the demo — what to do, when each one matters, and the snippet you can paste.
- 02Anthropic Prompt Engineering: The Production Checklist (copy-paste templates)Anthropic's docs teach you the techniques. This is the other half — the checklist you run before you ship, and four blocks you paste into a file: a system prompt that does not destroy your cache, a few-shot block, a tool definition plus the loop around it, and an untrusted-content wrapper for prompt-injection hardening. Written against the current models, where several moves from a two-year-old tutorial now return HTTP 400.
- 03Zero-Shot vs Few-Shot Prompting: Why Your LLM Output Keeps Breaking (and the 1-Minute Fix)Learn why your LLM output drifts in production: zero-shot asks without examples, few-shot pastes worked examples so the model copies your exact format. Same weights, different prompt.
- 04Why Your LLM Keeps Returning Garbage JSON (And How to Stop It)Every LLM-powered feature breaks the same way in production: the model returns almost-JSON. Markdown fences, trailing commas, a chatty preamble, a missing closing brace. Here's the 3-layer fix that ships — native structured outputs, Pydantic validation, and json_repair + retry loops.
- 05Prompt Compression: 30,000 Tokens → 9,200, Same Answer (and the Caching Trap)Prompt compression shrinks token counts by 70% using extractive and abstractive techniques, but breaks prompt caching and costs latency upfront. Learn when the tradeoff wins.
- 06Prompt Injection Explained: Why There Is No 'Parameterised Prompt' (LLM Security)Prompt injection occurs because LLM apps send instructions and user data as a single text stream with no channel separation, unlike SQL which has parameterised queries. Learn why there is no fix and what defences actually work.
- 07Prompt Versioning Without Langfuse — Three Lean Paths from prompts.yaml to PostgresLangfuse, PromptLayer, and Promptfoo will all sell you a prompt registry. There are three reasonable ways to ship prompt versioning in 2026, and the choice is about team shape, not technology. This post walks through all three — Langfuse Cloud, prompts.yaml + git, and a 2-table Postgres schema with an 80-line FastAPI router — on the same customer-support classifier, running on Claude Haiku 4.5.
03Retrieval and RAG
Giving a model your data. The demos stop at "embed and search"; the difficulty is everything on either side of that.
- 01RAG vs Knowledge Graphs — How to Give LLMs the Right ContextA practical comparison of Retrieval-Augmented Generation and Knowledge Graphs for grounding LLM responses, with architecture patterns, code examples, and guidance on when to use each approach.
- 02Semantic vs Keyword vs Hybrid Search: What Every RAG Demo SkipsEvery RAG demo shows embeddings and stops there. Real production search almost always mixes keyword and semantic retrieval. Here's what's happening under the hood, why hybrid wins, and a runnable Postgres example in ~40 lines.
- 03RAG Ingestion & Chunking — The Missing Engine Behind Hybrid SearchYou tuned the embedding model. You went hybrid. Your RAG still misses. The bug is upstream — in how you split documents. Five chunking strategies, when each wins, and how to actually evaluate them.
- 04Vector Search — how HNSW finds nearest neighboursHNSW turns nearest-neighbor search from brute-force (2M comparisons, 1.2s) into a hierarchical graph walk (1.8k comparisons, 2ms). Learn how greedy navigation trades exact answers for speed.
- 05Embedding Dimensionality: Cut Your Vectors 4× Without Re-Embedding Anything (Matryoshka)Embedding vectors don't have to stay at full width. Matryoshka representation learning lets you truncate from 3,072 to 768 dimensions after the fact, cutting storage 4× with minimal quality loss—no re-embedding required.
- 06Long Context vs RAG — When to Stuff Gemini's 2M Window vs Build a Vector DBGemini 2M and Claude 1M made 'just paste it all' a real engineering option. Here's the cost math, the latency curve, the quiet failure mode of context dilution, and the rule for when stuffing beats RAG — and when it silently hurts.
- 07RAG vs Fine-tuningRAG retrieves documents at query time; fine-tuning changes model weights to teach behavior. Learn which solves knowledge updates vs. consistent output style.
- 08From Toy to Truth: Building Production-Grade RAGYour RAG demo works on 100 documents and dies in production — not because retrieval is hard, but because nobody measures it. Here's the two-layer eval framework, the three-stage ranking funnel, and the diversity algorithms (MMR vs VRSD) that separate a toy from a system you can trust.
04Agents, tools and protocols
Letting the model act. Built from scratch first, so the frameworks read as conveniences rather than as magic.
- 01How LLM Function Calling Actually Works — From Tokens to Tool OrchestrationHow LLMs return structured data through function calling, how constrained decoding works under the hood, and what happens when the model needs to call multiple tools in a single turn.
- 02Building AI Tool-Calling Agents from Scratch with PythonA hands-on guide to building LLM agents that call real tools — from a single weather function to a multi-tool database agent with safety constraints and interactive chat.
- 03ReAct From Scratch: Make Your LLM Reason and Act in ~40 Lines (No LangChain)ReAct (Reason + Act) is already baked into the frontier models you use every day — it started as a 2022 paper and became the backbone of modern AI agents. Here's what it actually is: the Thought → Action → Observation loop, a real trace, and a ~40-line build from scratch in Python, TypeScript, and Java — without the model's native tool-calling — so every moving part is visible.
- 04Building a Verifiable AI Agent with the ReAct FrameworkHow to build an AI agent that does not just fire and forget — using the Check-Action-Verify loop, ReAct reasoning pattern, and paired action/verification tools to ensure every operation is confirmed.
- 05How AI Agents Do Deep Search — Building a Research Agent from ScratchWhy a single LLM call fails for complex research questions, and how to build an agent that iteratively searches, reads, reasons, and synthesizes — with full Python code you can run.
- 06What is LangChain? Build Your First Agent in 15 LinesLangChain is the most-used framework for building LLM apps — agents, RAG, multi-step workflows. This tutorial shows you what it actually is, the three primitives that matter, and your first working agent with one tool in under fifteen lines of Python — runnable today on Google's free Gemini tier, no credit card. Plus the same agent rewritten without LangChain for comparison, so you can see exactly what the framework is doing for you.
- 07MCP vs REST API vs Markdown — How Agents Should Actually Consume Your DataThree ways to hand data to an LLM agent: the Model Context Protocol, a boring REST API with an API key, or a curated Markdown file. Each is right some of the time and wrong a lot of the time. Here's the honest decision tree.
- 08The MCP Token Tax — Your Agent Is Burning Tokens Before It StartsEvery MCP server you connect loads its tool schemas into the context window before the first user turn. Here's the arithmetic on how expensive that gets, why most teams never measure it, and how to stop paying for tools the agent will never call.
05Memory
A four-part series on the thing a language model does not have, and what it takes to fake convincingly.
- 01When LLMs Learn to Remember — Part 1: LLMs Don't Remember AnythingLLMs are completely stateless - they forget everything after each call. So how does ChatGPT remember your preferences? This post breaks down the illusion of AI memory, compares current memory strategies (markdown trees, vector databases, hybrid systems), and explains context tree indexing.
- 02When LLMs Learn to Remember — Part 2: Building a Memory System for Your AI AssistantA practical guide to building persistent AI memory: Memory CRUD operations, post-conversation sweeps, context tree curation, prompt templates, and the unsolved problems nobody talks about.
- 03When LLMs Learn to Remember — Part 3: How OpenClaw Turns LLMs into an Operating SystemOpenClaw treats AI as an infrastructure problem. This deep dive covers its 3-tier memory architecture, MemOS, hybrid search, automatic memory flush, and what it means for the future of AI assistants.
- 04When LLMs Learn to Remember — Part 4: Why Your AI's Memory Shouldn't Be a Graph DatabaseGraph databases look like the obvious answer for AI memory — entities, relationships, multi-hop queries. So why did OpenClaw, MemOS, and every shipping system pick flat markdown instead? A contrarian deep dive into the real tradeoffs.
06Running and serving models
Inference as an engineering problem: what makes it fast, what makes it cheap, and what the two trade against each other.
- 01Running LLMs Locally — Small Models, Quantization, and Your 4GB GPUA hands-on guide to running language models on consumer hardware. What fits on a 4GB GPU, what quantization actually does, how llama.cpp and Ollama work, and whether local models can replace your API subscription.
- 02Ollama vs vLLM: Local Dev Tool or Production Inference Engine?Ollama and vLLM both run LLMs locally, but optimize different axes: Ollama trades throughput for simplicity on one laptop; vLLM trades setup complexity for production concurrency. Learn which to use and why they're not competitors.
- 03Quantization vs DistillationQuantization keeps the same model at lower precision (no retraining); distillation trains a new, smaller student to mimic a teacher. Learn when to use each technique to shrink your LLM.
- 04Speculative Decoding, Explained: Free LLM Speed With Zero Quality LossSpeculative decoding uses a small draft model to propose K tokens, then verifies all K+1 in one big-model forward pass. Same weight load, mathematically identical output, zero quality loss — typical 2–3× speedup.
- 05Continuous Batching: How One GPU Serves Hundreds of Chat Users at OnceContinuous batching schedules LLM inference requests at each token generation step instead of locking the batch. This lets one GPU serve dozens of concurrent users by evicting finished sequences and admitting queued ones mid-flight, trading per-user latency for throughput.
- 06Concurrent LLM Calls in Python: asyncio.gather + a Semaphore500 LLM calls in a for-loop takes 17 minutes — and your CPU does nothing for all of it. await is not concurrency. Here's how asyncio.gather puts every request in flight at once, why it's named for the results and not the speed, and why asyncio.Semaphore is the one line that stops you from DDoS-ing yourself into a wall of 429s.
- 07How Should Your LLM Stream Tokens? SSE vs WebSockets vs PollingYour chatbot dumps the whole answer after an 8-second stare. ChatGPT types it out token by token. The difference is one transport decision — and for LLM streaming the answer is almost always Server-Sent Events, not WebSockets. Here's why, with the FastAPI code.
- 08The LLM Gateway: One Endpoint Instead of Five SDKsAn LLM gateway is the reverse proxy your AI stack is missing — one endpoint that handles fallback, cost control, caching, and observability across every model provider. Here's what it buys you, the 2026 landscape, where it's heading (agents + MCP), and when to skip it.
07Getting it into production
The work between a demo that impresses and a system you are willing to be paged for.
- 01The Guardrail Stack You Actually Need in LLM ProductionFive layers of LLM guardrails get talked about. Two of them carry the weight in most shipped features: output schema validation and tool/action policy. The other three are mostly theater until specific triggers fire. A practical breakdown drawn from running LLM features in production.
- 02Stop Vibe-Checking Your Prompts - Building an Eval Harness That Catches Regressions Before Your Users DoMost LLM features ship on vibes. The first time you regret it is the day a prompt change quietly breaks half your traffic and nobody notices for a week. Here is what an honest eval harness actually contains - golden datasets, LLM-as-judge, prompt versioning - and the real cost of running it.
- 03Build Your Own Model Registry in a Weekend — FastAPI + Postgres, No MLflowMost ML teams reach for MLflow before they need it and pay the operational tax for years. The custom model-registry pattern from a real labeling platform — version, compare, roll back models with a 200-line FastAPI service. When the DIY version is the right answer, and the three signals that say it isnt anymore.
- 04Build Your Own Local Text-to-Speech Stack with PythonA hands-on guide to deploying 3 TTS engines (Edge TTS, Piper, Coqui XTTS v2) and a speech-to-text service — from a simple API to desktop keyboard shortcuts that read any selected text aloud.
08Computer vision
The other half of "AI" — how detection got from hand-built pipelines to a single transformer.