B7 · Data Literacy and Visualisation

Big Data in Simple Terms

Big data means collections of data so large, so fast-growing or so varied that ordinary tools such as a single spreadsheet cannot handle them. It is often described by three Vs: volume (how much), velocity (how fast it arrives) and variety (how many different kinds). Big data is one of the main reasons modern AI became possible.

6 min Beginner Jump to the activity

What you need: Nothing. This page starts from the beginning, with no programming and no formulas.

Definition

Every minute, people around the world send millions of messages, make huge numbers of online payments and upload hundreds of hours of video. Who could possibly look at all of it?

Big data means collections of data so large, so fast-growing or so varied that ordinary tools such as a single spreadsheet cannot handle them. It is often described by three Vs: volume (how much), velocity (how fast it arrives) and variety (how many different kinds). Big data is one of the main reasons modern AI became possible.

In simple words

Counting the mangoes in one basket is easy. Counting every mango sold in every market in India, every hour, as trucks keep arriving, and also reading the farmers' notes and photos of each crate, needs a completely different system. That is the jump from data to big data.

Why it matters

Big data powers maps, recommendations, fraud detection, weather forecasts and AI chatbots. It also raises serious questions about privacy and power, because whoever holds huge amounts of data about people can learn a great deal about them.

Key terms

Big data
Data too large, fast or varied for ordinary tools.
Volume
The sheer amount of data.
Velocity
How fast new data arrives.
Variety
The different forms of data: numbers, text, images, sound, video, locations.
Data centre
A building full of computers that store and process large amounts of data.
Gigabyte and terabyte
Units of data size. A terabyte is about a thousand gigabytes.

Explained in detail

The three Vs

VMeaningExample
VolumeHuge amountsEvery UPI payment in India in a year: many billions of transactions
VelocityArriving very fastLocation pings from every phone using a maps app during rush hour
VarietyMany kinds mixed togetherA hospital's records: numbers, doctors' notes, X-ray images and scans

Some people add more Vs, such as veracity (how trustworthy the data is) and value (whether it is useful). Veracity is important: a huge pile of messy, biased or wrong data is still messy, biased and wrong. More data helps only when it is good data.

Where big data comes from

  • Phones: locations, app use, photos and messages.
  • Payments: every card swipe, UPI transfer and online order.
  • Sensors: weather stations, satellites, traffic cameras, smart meters and fitness bands.
  • The internet: web pages, videos, posts, searches and clicks.
  • Organisations: hospital records, school results, government surveys.

How it is handled

No single computer can store or process data at this scale, so the work is split across thousands of computers in data centres working at the same time, each handling a small piece. The results are then combined. This idea of dividing a huge job into many small ones that run side by side is what makes big data, and training large AI models, possible. It is also why data centres use so much electricity and water, as you learned in the Responsible AI module.

Big data and AI

Machine learning improves with more examples. A model that learns to recognise speech from a few hundred recordings will be poor; one that learns from many thousands of hours across many accents will be far better. Large language models were trained on enormous collections of text. This is why the companies with the most data often have an advantage in AI, and why questions of who owns data, who agreed to its use and who benefits from it matter so much.

Big data is also used for public good: tracking the spread of disease, predicting floods and planning bus routes from travel patterns.

How it works

  • Ask how much data there is, how fast it arrives and what kinds it includes.
  • Ask whether it is accurate and representative, not just large.
  • Ask who collected it, with whose consent, and for what purpose.

Try it yourself

How a maps app knows about the traffic jam

You open a maps app and a road is shown in red. Nobody reported a jam. How does it know?

Which V is it?

Which V of big data does each example show best?

  • A farm app collecting soil numbers, crop photos and farmers' voice notes

  • Every book in a national library, scanned

  • All the photos ever uploaded to a social media site

  • Stock prices changing many times a second

  • Live readings from smart electricity meters across a city

Common mistakes

  • Thinking more data always means better answers.
  • Forgetting that big data about people raises privacy questions.
  • Assuming big data is only about size, and ignoring speed and variety.

Check yourself

Question 1

1. What does "velocity" mean in big data?

Question 2

2. Why is big data handled by many computers at once?

Question 3

3. A huge data set is full of errors. What is true?

Summary

  • Big data is too large, fast or varied for ordinary tools: volume, velocity and variety.
  • It is processed by many computers working together, and it powers modern AI.
  • Quality, consent and privacy matter as much as size.

Finished reading?

Sources: Open Government Data Platform India · Digital Personal Data Protection Act, 2023 (MeitY)