A large language model (LLM) is a neural network trained on a very large amount of text to predict what comes next. Given “The capital of France is”, it assigns a probability to every possible next token and picks one. Then it adds that token to the input and does it again. It has no database of facts and no separate reasoning engine — what looks like knowledge and reasoning is a statistical model of language that has become very good. That one fact explains both why LLMs are useful and why they fail in the ways they do.
1 · From Prompt to Answer in Five Steps
| Step | What happens | Example |
|---|---|---|
| 1 · Input | The prompt, plus any system instructions, chat history and retrieved documents, joined into one sequence | “Explain quantum computing simply” |
| 2 · Tokenise | Text is split into tokens — whole words, word pieces or characters — each mapped to an ID | Explain · quantum · computing · simply → four IDs. English averages roughly four characters per token; Hindi and other Indic scripts usually need more tokens for the same meaning |
| 3 · Embed | Each token ID becomes a vector of numbers; position information is added so word order counts | “quantum” → a list of thousands of numbers that places it near “physics” and “qubit” |
| 4 · Transformer | Dozens of layers, each with attention (tokens look at each other) and a feed-forward network, with residual connections and normalisation | Each layer refines what each token means in this context |
| 5 · Generate | The last layer gives a probability for every token in the vocabulary; one is sampled, appended, and the loop repeats until a stop token | “Quantum” → “computing” → “uses” → … |
▸ Temperature, top-k and top-p only touch step 5. Low temperature picks the most likely token almost every time (repeatable, plain); high temperature samples more widely (varied, more error-prone). None of them makes the model know more.
2 · Attention in One Line
Attention lets each token decide which other tokens matter for its meaning. In “I went to the river bank”, the token “bank” attends strongly to “river” and weakly to everything else — which is how the model knows this is not a bank that holds money.
| Part | Plain meaning |
|---|---|
| Query (Q) | What this token is looking for |
| Key (K) | What each other token offers |
| Q·KT ÷ √dk | How well each query matches each key, scaled so the numbers stay stable |
| softmax | Turns the match scores into weights that add up to 1 |
| Value (V) | The information actually passed along, weighted by those scores |
| Multi-head | Many attentions run in parallel, each free to track a different relation (grammar, reference, topic) |
| Masked | In generators, a token may only attend to tokens before it — the model cannot peek at the answer |
3 · Three Architecture Families
| Family | Examples | Good at |
|---|---|---|
| Encoder-only | BERT and its descendants | Understanding: classification, search, spam and phishing detection, embeddings |
| Decoder-only | GPT, Llama and most of today’s chat models | Generation: chat, writing, code, agents |
| Encoder–decoder | T5 | Transforming one text into another: translation, summarisation |
A newer variant, mixture of experts (MoE), keeps a decoder but routes each token through only a few of many sub-networks, so a very large model costs less to run.
4 · How a Model Is Trained
- Data collection — very large volumes of text and code: web pages, books, papers, repositories.
- Pre-processing — remove duplicates, filter low-quality and unsafe content, strip personal data where possible, tokenise.
- Pre-training — predict the next token across the whole corpus, adjusting billions of parameters by gradient descent. This is the expensive part, run on thousands of GPUs or TPUs.
- Fine-tuning — train further on curated examples of instructions and good answers (supervised fine-tuning). Cheap variants such as LoRA change only a small add-on set of weights.
- Alignment — teach the model which answers people prefer and which to refuse, with methods such as RLHF (reinforcement learning from human feedback) or direct preference optimisation.
- Evaluation & red-teaming — benchmarks for capability; adversarial testing for safety and security before release.
5 · Twelve Terms That Come Up Constantly
| Term | Plain meaning |
|---|---|
| Parameters | The learned weights; “70B” means 70 billion of them |
| Context window | The most tokens the model can consider at once — prompt, history and documents together |
| Inference | Running a trained model to get an answer |
| KV cache | Stored keys and values from earlier tokens, so each new token is cheap to compute |
| Quantisation | Storing weights with fewer bits (e.g. 16-bit → 4-bit) to run faster on smaller hardware |
| Distillation | Training a small model to imitate a large one |
| Embedding | A vector that captures meaning; the basis of semantic search |
| RAG | Retrieval-augmented generation — fetch relevant documents and put them in the prompt |
| Tool use / agents | The model outputs a structured call that software executes, then reads the result |
| MCP | Model Context Protocol — an open standard for connecting models to tools and data |
| Hallucination | Fluent, confident output that is false |
| Prompt injection | Text in the input that overrides the instructions the model was given |
6 · Where Security Risk Enters
Each risk maps to a specific step above, which tells you where the control has to sit.
| Risk | Enters at | Why it happens | Control |
|---|---|---|---|
| Prompt injection | Input | Instructions and data share one token sequence; the model cannot reliably tell them apart | Treat all retrieved content as untrusted; limit what tools can do; human approval for risky actions |
| Sensitive-data leakage | Input & output | Whatever enters the context can come out in an answer | DLP before the model; permission-aware retrieval; output filtering |
| Training-data poisoning | Data & fine-tuning | Malicious examples plant behaviours or back doors | Data provenance, vetting of fine-tuning sets, model evaluation |
| Memorisation | Pre-training | Rare strings — keys, personal data — can be reproduced verbatim | Scrub training data; do not fine-tune on secrets |
| Hallucination | Generation | The model optimises for plausible text, not true text | Grounding with RAG and citations; verification for anything consequential |
| Excessive agency | Tool use | A model with broad tool permissions acts on a wrong or injected instruction | Least-privilege tools, scoped credentials, audit logs |
▸ Prompt injection is not a bug that will be patched in the next release. It follows from step 1: an LLM reads everything in its context as one stream of text. Design as if any document, email or web page the model reads may contain instructions. More in AI Security: New Attack Surface and Shadow AI.
Bottom line
- One job, at scale — next-token prediction, repeated, is the whole mechanism.
- Attention is context — each token weighs every earlier token to decide what it means here.
- Training sets behaviour, alignment sets manners — pre-training gives ability, fine-tuning and RLHF shape how it is used.
- Sampling settings change style, not knowledge.
- Every risk has an address — input, data, generation or tools; put the control there.
Sources & method. Architecture from Vaswani et al., “Attention Is All You Need” (2017), and the published BERT, GPT and T5 papers; training stages as described in public model reports. Token-per-character figures are rough averages that vary by tokenizer. Risk mapping follows the OWASP Top 10 for LLM Applications; the step-by-step placement is the author’s analysis. No claim is made about the internal design of any specific commercial model. Original analysis; no third-party graphics or text reproduced.
← Back to Cyber security