Definition
Could you now take raw text in any Indian language, clean and tokenise it, turn it into numbers, classify its topic or sentiment, build a chatbot around it and add a voice?
This review brings together NLP in Practice: the stages of NLP, tokenisation and cleaning, stemming and lemmatisation, parts of speech and named entities, bag of words, TF-IDF, word embeddings, sentiment analysis, text classification, intent-based chatbots, speech-to-text and text-to-speech and NLP for Indian languages, followed by a scored quiz.
In simple words
Read the recap table, try the activities, then take the quiz. Each answer comes with an explanation. If you miss one, go back to that topic.
Why it matters
Language is the most common way people interact with AI. These skills let you build useful text and voice tools and understand what large language models build on.
How it works
| Topic | Key idea |
|---|---|
| Stages | Clean, tokenise, normalise, tag, understand meaning and context, perform the task |
| Tokens | Words, sub-words or characters; keep negations and emojis for sentiment |
| Normalising words | Stemming chops by rule; lemmatisation returns dictionary forms |
| POS and NER | Grammatical tags and named entities with BIO labels |
| Representations | Bag of words and TF-IDF count words; embeddings capture meaning |
| Classification | Sentiment and topic labels; evaluate per class with macro F1 |
| Chatbots | Intents, slots, fallbacks and human hand-over |
| Speech | ASR measured by WER; TTS with a vocoder; consent for voices |
| Indian languages | Scripts, code-mixing, Roman typing, Unicode normalisation and native-speaker review |
Try it yourself
Match the task to the technique
Which technique fits best?
- Find all company names in news
- Search help articles by meaning
- Route complaints to departments
- Let users block a card by chat
- Turn a voice question into text
True or false?
Decide for each statement.
Bag of words keeps word order
Lemmatisation turns "went" into "go"
Removing "not" can flip sentiment
WER can never exceed 100%
Common mistakes
- Skipping a simple baseline before trying large models.
- Evaluating only in English or Hindi.
- Letting chatbots act without confirmation or human hand-over.
Check yourself
Summary
- NLP turns text into structure: tokens, tags, entities, vectors and labels.
- Start with simple baselines, evaluate per class and per language, and add context-aware models where needed.
- Chatbots and voice systems need confirmations, fallbacks, consent and human hand-over.
Finished reading?
Sources: Jurafsky and Martin, Speech and Language Processing (3rd ed. draft) · AI4Bharat: open-source AI for Indian languages