large language model

Typeset PDF version — the authoritative form of this entry

The term large language model (LLM) refers to machine learning (ML) methods that analyze or generate text data, represented as sequences of tokens, using a high-dimensional model with billions of model parameters. Many current LLMs use a variant of a transformer that is trained via self-supervised learning. The training task is to predict words that are intentionally removed from a large text corpus, a construction that yields large training sets of labeled data points with little human supervision. A trained LLM maps an input sequence of tokens to a probability distribution over the next token. A prominent application is conversational artificial intelligence (AI), which generates human-like text for tasks ranging from answering questions to writing computer code.

Definition

An LLM is an umbrella term for machine learning (ML) methods that use high-dimensional ML models (with billions of model parameters) trained on large collections of text data. LLMs are used to analyze or generate sequences of tokens that constitute text data. Many current LLMs use some variant of a transformer that is trained via self-supervised learning, i.e., the training is based on the task of predicting a few words that are intentionally removed from a large text corpus. Thus, labeled data points can be constructed simply by selecting some words from a given text as labels and the remaining words as features of data points. This construction requires very little human supervision and allows for sufficiently large training sets for LLMs (Brown et al., 2020; Devlin et al., 2019). A prominent application is conversational artificial intelligence (AI) which generates human-like text for tasks ranging from answering questions to writing and debugging computer code.

Figure 1 of the entry llm — Figure 1: Signal-flow chart of an LLM. An input sequence of tokens is mapped to vectors by an embedding, refined by $N$ stacked transformer blocks, mapped back to vocabulary scores by the unembedding, and turned into a probability distribution over the next token via softmax function

See also: token, transformer, natural language processing (NLP), foundation model, embedding, softmax function.

References

Brown et al. (2020). Language Models are Few-Shot Learners. Adv. Neural Inf. Process. Syst..
Devlin et al. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proc. Conf. North American Chapter Assoc. Computational Linguistics (NAACL).

Cite this entry

@misc{dictml_llm,
  author = {Jung, Alexander},
  title = {large language model},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-08-06},
  url = {https://dictionaryofml.org/terms/llm.html}
}