Time Series Data

🕐 Time Series Data: Your Database’s Memory of Time

Imagine your database is like a magical diary that remembers everything that happened, when it happened, and can tell you stories about the past!

🎯 The Big Picture

Think of Time Series Data like a video camera that records your height every birthday. Each measurement has two important things: what happened (your height) and when it happened (your birthday). NoSQL databases have super special tools to store and play back these “time recordings” really fast!

graph TD
    A["📅 Time Series Data"] --> B["📦 Collections"]
    B --> C["🪣 Bucketing"]
    C --> D["📉 Downsampling"]
    D --> E["🗄️ Retention"]
    E --> F["🗜️ Compression"]
    F --> G["⏰ TTL/Expiration"]

📦 What is Time Series Data?

The Story

Imagine you have a pet fish named Bubbles. Every hour, you write down:

• The water temperature 🌡️
• The time you checked ⏰

After a year, you have 8,760 notes! That’s time series data — measurements collected over time.

Real Examples

Key Characteristics

✅ Timestamp is king — Every piece of data has a “when” ✅ Data flows in — New data keeps arriving ✅ Old data is rarely changed — We mostly add, rarely edit ✅ Recent data matters most — Today’s temperature is more important than last year’s

📚 Time Series Collections

The Story

Remember your fish diary? Instead of using a regular notebook, imagine using a special time-diary that automatically:

• Sorts entries by date
• Groups similar entries together
• Makes finding “all temperatures from March” super fast!

That’s what a Time Series Collection does!

How It’s Different

Regular Collection (messy notebook):
📝 Page 1: Fish color = blue
📝 Page 2: Temp = 72°F at 3pm
📝 Page 3: Fish name = Bubbles
📝 Page 4: Temp = 71°F at 4pm

Time Series Collection (organized diary):
📅 3:00 PM → Temp: 72°F
📅 4:00 PM → Temp: 71°F
📅 5:00 PM → Temp: 73°F

Creating One (MongoDB Example)

db.createCollection("fishTemps", {
  timeseries: {
    timeField: "timestamp",
    metaField: "tankId",
    granularity: "hours"
  }
})

What this means:

• timeField → “This is where the clock time lives”
• metaField → “This tells us WHICH fish tank”
• granularity → “We check every hour”

🪣 Bucketing: Putting Data in Time Boxes

The Story

Imagine you collect candy wrappers. Instead of having 1,000 wrappers flying everywhere, you put them in monthly boxes:

• January Box 📦
• February Box 📦
• March Box 📦

Bucketing does exactly this with time data!

Why Bucketing is Smart

Without Bucketing:

Document 1: {time: "Jan 1, 9:00", temp: 70}
Document 2: {time: "Jan 1, 9:01", temp: 71}
Document 3: {time: "Jan 1, 9:02", temp: 70}
... 1,440 documents per day! 😰

With Bucketing:

Document 1: {
  date: "Jan 1",
  readings: [
    {time: "9:00", temp: 70},
    {time: "9:01", temp: 71},
    {time: "9:02", temp: 70}
    ... all in ONE document! 😊
  ]
}

Bucket Sizes

📉 Downsampling: Making Data Smaller

The Story

You took 1,000 photos on vacation. Do you show ALL of them to grandma? No! You pick the 10 best ones. That’s downsampling!

For data: Instead of keeping every minute’s temperature, we keep just the hourly average.

Before and After

Before (too much detail):

9:00 → 70°F
9:01 → 71°F
9:02 → 70°F
9:03 → 72°F
... 60 readings per hour

After (just right):

9:00 hour → Average: 71°F
           → Min: 70°F
           → Max: 72°F

Common Downsampling Methods

graph TD
    A["Raw Data: 1000 points"] --> B{Downsample}
    B --> C["Average: 71°F"]
    B --> D["Minimum: 68°F"]
    B --> E["Maximum: 74°F"]
    B --> F["Sum: 71000°F"]
    B --> G["Count: 1000"]

Real Example

// MongoDB aggregation for downsampling
db.temps.aggregate([
  {
    $group: {
      _id: { $dateTrunc: {
        date: "$timestamp",
        unit: "hour"
      }},
      avgTemp: { $avg: "$temp" },
      minTemp: { $min: "$temp" },
      maxTemp: { $max: "$temp" }
    }
  }
])

🗄️ Retention Policies: How Long to Keep Data

The Story

Your mom says:

• Keep your toys forever (important!)
• Keep homework for 1 year (might need it)
• Throw away food wrappers today (not useful)

Retention policies tell the database how long to keep different data!

Setting Up Retention

How It Works

graph LR
    A["New Data"] --> B["Hot Storage<br>All Details"]
    B -->|After 7 days| C["Warm Storage<br>Hourly Avg"]
    C -->|After 30 days| D["Cold Storage<br>Daily Avg"]
    D -->|After 1 year| E["🗑️ Delete"]

Real Example

// Create collection with 30-day expiration
db.createCollection("sensorData", {
  timeseries: {
    timeField: "timestamp",
    expireAfterSeconds: 2592000 // 30 days
  }
})

🗜️ Data Compression: Squishing Data Smaller

The Story

Imagine packing for vacation:

• Without compression: 10 big suitcases 🧳🧳🧳🧳🧳
• With compression: 2 small backpacks 🎒🎒

Compression makes your data take up less space while keeping all the information!

How It Works

Delta Encoding (smart trick!):

Before:
72°F, 73°F, 73°F, 74°F, 73°F

After (store only changes):
72°F, +1, +0, +1, -1

Much smaller! 🎉

Compression Types

Benefits

✅ Save storage — 10x less space ✅ Faster queries — Less data to read ✅ Lower costs — Less disk needed ✅ Automatic — Database does it for you!

⏰ TTL and Data Expiration

The Story

Milk has an expiration date. After that date, you throw it away. TTL (Time To Live) is the expiration date for your data!

How TTL Works

graph TD
    A["Data Created"] --> B["Clock Starts ⏰"]
    B --> C{TTL Reached?}
    C -->|No| D["Data Lives 💚"]
    C -->|Yes| E["Auto Delete 🗑️"]
    D --> C

Setting TTL

Option 1: Collection Level

db.createCollection("logs", {
  timeseries: {
    timeField: "timestamp",
    expireAfterSeconds: 86400 // 24 hours
  }
})

Option 2: Index Level

db.events.createIndex(
  { "createdAt": 1 },
  { expireAfterSeconds: 3600 } // 1 hour
)

TTL Use Cases

Important Rules

⚠️ TTL field must be a date type ⚠️ Deletion isn’t instant — runs every ~60 seconds ⚠️ Can’t undo — deleted data is gone forever!

🎯 Putting It All Together

Here’s how all these pieces work as a team:

graph TD
    A["📊 Sensor Reading Arrives"] --> B["📦 Time Series Collection"]
    B --> C["🪣 Grouped into Buckets"]
    C --> D["🗜️ Compressed to Save Space"]
    D --> E{Data Age?}
    E -->|Fresh| F["Keep All Details"]
    E -->|Older| G["📉 Downsample"]
    G --> H["🗄️ Apply Retention Policy"]
    H --> I{TTL Expired?}
    I -->|No| J["Keep in Storage"]
    I -->|Yes| K["⏰ Auto Delete"]

Quick Summary

🚀 You Did It!

You now understand how NoSQL databases handle time series data like a pro! These tools help us:

• 📈 Store billions of readings without running out of space
• ⚡ Query years of data in milliseconds
• 💰 Save money on storage costs
• 🤖 Automate cleanup so old data disappears magically

Remember: Time series data is everywhere — from your fitness tracker to weather apps to stock markets. Now you know the magic behind it! ✨