Definition
A newspaper's archive holds a million articles. Nobody has time to tag each one, yet editors want to know which themes rose and fell over the years. Topic modelling reads the whole collection and discovers its themes without being told what to look for.
Topic modelling is an unsupervised method that finds groups of words that often occur together across a collection of documents. Each group is a topic, described by its most important words, and each document is described as a mixture of topics. Latent Dirichlet Allocation (LDA, 2003) is a probabilistic topic model that assumes each document mixes a few topics and each topic is a distribution over words. Non-negative matrix factorisation (NMF) factorises the document-word matrix, often TF-IDF weighted, into document-topic and topic-word matrices. Newer methods such as BERTopic cluster sentence embeddings.
In simple words
Imagine sorting a big pile of newspaper cuttings without reading them closely, just by noticing which words travel together: "wicket", "over" and "century" here; "rain", "monsoon" and "forecast" there. Each pile is a topic, and a cutting about rain stopping a cricket match belongs partly to both piles.
Overview
- Topic modelling discovers themes in a collection of documents without labels.
- Each topic is a group of words that often appear together.
- Each document is described as a mixture of topics.
- People read the top words and name the topics.
- The number of topics is chosen by trying several values.
- Claims based on topics should be checked by reading documents.
Why it matters
Topic models summarise large text collections: customer reviews, complaints, research papers, parliamentary debates and social-media posts. They help analysts explore, route support tickets, track trends over time and decide which categories a classifier should use.
Key terms
- Topic
- A set of words that tend to occur together, such as rain, monsoon, forecast.
- Document-term matrix
- A table with one row per document and one column per word, holding counts or TF-IDF weights.
- LDA
- Latent Dirichlet Allocation: a probabilistic model of documents as topic mixtures.
- NMF
- Non-negative matrix factorisation: splitting the document-term matrix into topic parts with no negative values.
- Topic coherence
- A score of how often a topic's top words really occur together, used to judge quality.
- Number of topics (k)
- A setting chosen by the analyst; too few merge themes, too many split them.
Types
There are several types of topic model. They differ in how they represent text and find themes.
1. Latent Dirichlet Allocation (LDA)
A probabilistic model (Blei, Ng and Jordan, 2003) of documents as mixtures of topics.
- How it works: Each topic is a distribution over words and each document a distribution over topics; inference estimates both from word counts.
- Example: Finding themes such as monsoon, elections and cricket in a newspaper archive.
- When to use it: Use it for longer documents and when you want a probabilistic description.
2. Non-negative matrix factorisation (NMF)
Factorising the TF-IDF matrix into document-topic and topic-word parts with no negative values.
- How it works: Non-negativity makes topics additive and readable; it is fast and deterministic with fixed settings.
- Example: NMF finding clean cricket, election, rain and phone topics in twelve short news items.
- When to use it: Use it for short texts and quick, readable topics.
3. Latent semantic analysis (LSA)
Applying truncated SVD to a term-document matrix (Deerwester et al., 1990).
- How it works: It finds latent dimensions that group related words, though components can have negative values that are harder to read.
- Example: Improving search so that "car" also finds documents about "automobile".
- When to use it: Use it for dimensionality reduction of text and simple semantic search.
4. Embedding-based topic models (BERTopic)
Clustering sentence embeddings and describing each cluster with its key words.
- How it works: Documents are embedded, reduced with UMAP, clustered with HDBSCAN, and each cluster is summarised with class-based TF-IDF.
- Example: Grouping 30,000 citizen complaints by meaning even when they use different words.
- When to use it: Use it for short, varied texts where synonyms matter.
5. Dynamic and guided topic models
Models that track topics over time or are steered with seed words.
- How it works: Dynamic topic models let topic words drift by period; guided (seeded) models nudge topics towards themes the analyst cares about.
- Example: Tracking how parliamentary debates about agriculture changed from 1990 to 2020.
- When to use it: Use them for trend analysis and when you already know some themes to look for.
Explained in detail
How topic models work
Both LDA and NMF start from a document-term matrix. NMF looks for two smaller non-negative matrices whose product approximates it: W, which says how much of each topic every document contains, and H, which says how strongly each word belongs to each topic. Because the values cannot be negative, topics combine by adding words, which makes them readable. LDA instead imagines how documents could have been written: pick a mix of topics for the document, then for each word pick a topic from that mix and a word from that topic. Training works backwards to find the topic mixes and word distributions that best explain the real documents. In both cases you choose the number of topics, and the model labels them only with numbers; people read the top words and give names such as "cricket" or "elections".
Methods compared
| Method | Input | Strengths | Limits |
|---|---|---|---|
| LDA | Word counts | Probabilistic mixtures; well studied | Struggles with very short texts |
| NMF | TF-IDF weights | Fast; often cleaner topics on short texts | Less of a probabilistic story |
| BERTopic | Sentence embeddings, then clustering | Understands synonyms and context | Heavier to run; needs an embedding model |
| LLM labelling | Prompts over samples of text | Readable labels and summaries | Costly at scale; must be checked |
Using topic models well
Clean the text first: lowercase, remove stop words and very rare or very common words, and consider lemmatising. Try several values of k and judge the topics by reading them and by coherence scores; there is rarely one correct number. Topics are only as good as the data: a collection of hotel reviews will produce topics such as rooms, food, staff and location. Remember that topics reflect word co-occurrence, not true meaning, so a topic can mix two ideas that share vocabulary. Before publishing claims such as "complaints about delivery doubled", read a sample of documents assigned to the topic to confirm. In the code below, NMF on twelve short news items finds four clean topics, about cricket, elections, rain and phones, and assigns every item to the right one.
Real-world scenarios: researchers have used topic models to trace how debates in parliaments and newspapers shifted over decades; companies group thousands of app-store reviews to see which problems users mention most after each release; and libraries organise digitised archives so readers can browse by theme rather than by keyword alone.
How it works
- Collect documents and clean the text.
- Build a count or TF-IDF document-term matrix.
- Fit LDA or NMF with several numbers of topics.
- Read each topic's top words and a few documents, then name the topics.
- Use the topic mixtures for exploration, trends or routing, and check claims by reading samples.
The maths
- Documents × words matrix
- Documents × topics weights
- Topics × words weights
Suppose 1,000 reviews of a food-delivery app are modelled with k = 3 topics. Topic A's top words are late, delivery, hour, waiting, rider; topic B's are cold, taste, stale, quantity, packaging; topic C's are refund, support, call, response, app. An analyst names them "delivery delays", "food quality" and "customer support". A review saying "rider was an hour late and the food arrived cold" might be 0.6 delivery delays, 0.35 food quality and 0.05 customer support. If 45 per cent of all topic weight in January goes to delivery delays but 25 per cent in March, the company's new routing system is probably helping, which can be checked by reading samples.
In code
import warnings
from sklearn.decomposition import NMF
from sklearn.feature_extraction.text import TfidfVectorizer
warnings.filterwarnings("ignore")
news = [
"India won the cricket match after a century by the captain",
"The bowler took five wickets as the team won the test match",
"Fans cheered as the cricket team lifted the trophy",
"Heavy monsoon rain flooded roads and the weather office issued an alert",
"The weather department forecast more rain and thunderstorms this week",
"Farmers welcomed the early monsoon rain for sowing rice",
"Voters queued at polling booths as the election began",
"The election commission announced the date for counting votes",
"Candidates campaigned for votes before the state election",
"A new smartphone launched with a faster chip and better camera",
"The company released an app update with new AI features",
"Engineers built a faster chip for phones and laptops",
]
tfidf = TfidfVectorizer(stop_words="english")
X = tfidf.fit_transform(news)
nmf = NMF(n_components=4, init="nndsvd", random_state=0, max_iter=2000)
doc_topics = nmf.fit_transform(X)
words = tfidf.get_feature_names_out()
for t, weights in enumerate(nmf.components_):
top = [words[i] for i in weights.argsort()[::-1][:5]]
print(f"Topic {t}: {', '.join(top)}")
print("Topic of each news item:", [int(t) for t in doc_topics.argmax(axis=1)])Output
Topic 0: won, match, cricket, team, captain Topic 1: election, votes, campaigned, state, candidates Topic 2: rain, weather, monsoon, week, thunderstorms Topic 3: chip, faster, new, smartphone, better Topic of each news item: [0, 0, 0, 2, 2, 2, 1, 1, 1, 3, 3, 3]
Try it yourself
Name the topic
Which label fits each list of top words?
- wicket, over, century, innings
- rain, monsoon, flood, forecast
- vote, election, candidate, poll
- refund, support, call, response
Listening to citizens' complaints
A city corporation receives 30,000 online complaints a month in Hindi and English.
Common mistakes
- Treating the number of topics as known instead of trying several.
- Skipping stop-word removal, so topics fill with words like "the" and "said".
- Publishing conclusions without reading documents from each topic.
Check yourself
Summary
- Topic models find groups of co-occurring words and describe documents as topic mixtures.
- LDA is probabilistic, NMF factorises TF-IDF, and BERTopic clusters embeddings.
- Clean text, try several k, name topics by reading them, and verify claims with samples.
Finished reading?
Sources: David Blei (2012), Probabilistic Topic Models, Communications of the ACM · scikit-learn: topic extraction with NMF and LDA