Description
microgpt is an interactive visualisation of a very small GPT model that runs in the browser. It is an educational tool for seeing how a language model generates text one character at a time. The model is trained on a dataset of names and learns to produce new names that look like the training data. A training button adjusts the model's weights with gradient descent, and a generate button samples new text, while a step counter and loss value show progress. The page answers questions about the design. It explains the choice of 16 dimensions, four attention heads and a 64-unit multilayer perceptron as a trade-off between speed and capability, and it compares these with the much larger sizes used in modern language models. It explains what a cell in the weight heatmap is, how training uses the error to adjust weights, how attention lets each character focus on the others, and what RMSNorm does to a vector. The project's source code is on GitHub, and the page is a learning resource for people who want to understand the mechanics of a transformer.…
Highlights
- Interactive tiny GPT in the browser
- Training and generation controls
- Weight heatmap and loss display
- Explanations of attention and RMSNorm
- Open source code on GitHub
Ratings & Reviews
No ratings yet
Be the first to rate microgpt.
People also visit
microgpt FAQ
What is microgpt?
An interactive visualisation of a very small GPT model, running in your browser, for learning how language models work.
What does the model learn?
It learns from a dataset of names and generates new names that look like them.
How do the questions on the page help?
They explain the design choices, such as the number of dimensions and attention heads, and the mechanics of attention, training and normalisation.
Where is the code?
On GitHub at b44ken/microgpt.
What are the best microgpt alternatives?
Popular alternatives to microgpt include Gpt2 Webgl, LLM-Visualized and ChatLLM-Web. See all microgpt alternatives



