Iterators and Generators

🎭 The Magic Conveyor Belt: Understanding Iterators and Generators

Imagine you have a magical candy factory. Instead of making all the candies at once and filling up your entire room, this factory makes one candy at a time, only when you ask for it. That’s exactly what iterators and generators do in Python!

🎪 The Big Picture

Think of a conveyor belt at a factory:

• Regular lists = All items dumped on the floor at once (takes up space!)
• Iterators/Generators = Items come one by one on a conveyor belt (saves space!)

graph TD
    A["📦 Your Data"] --> B{How do you want it?}
    B -->|All at once| C["🗃️ List - Uses lots of memory"]
    B -->|One at a time| D["🎢 Iterator - Saves memory!"]

📜 The Iterator Protocol

What is it?

The Iterator Protocol is like a promise between Python and your object. It says:

“I promise I can give you items one at a time!”

To keep this promise, your object needs TWO special methods:

Simple Example

# A list is iterable
my_list = [1, 2, 3]

# Get the iterator (the conveyor belt)
my_iterator = iter(my_list)

# Get items one by one
print(next(my_iterator))  # 1
print(next(my_iterator))  # 2
print(next(my_iterator))  # 3
# next again? StopIteration!

🎯 Key Insight: When there are no more items, Python raises StopIteration - like the conveyor belt saying “That’s all folks!”

🔧 Creating Custom Iterators

Let’s build our own conveyor belt!

The Countdown Timer

Imagine a rocket countdown: 5, 4, 3, 2, 1, Blast off!

class Countdown:
    def __init__(self, start):
        self.current = start

    def __iter__(self):
        return self  # I am my own iterator!

    def __next__(self):
        if self.current <= 0:
            raise StopIteration  # Done!

        self.current -= 1
        return self.current + 1

# Use it!
for num in Countdown(5):
    print(num)
# Output: 5, 4, 3, 2, 1

Why Custom Iterators?

⚡ Generator Functions - The Easier Way!

Custom iterators work, but they’re like building a car from scratch. Generators are like buying a car - much easier!

The Magic Word: yield

Instead of return, use yield. It’s like saying:

“Here’s one item. Call me again for the next one!”

def countdown(start):
    while start > 0:
        yield start  # Give this, then pause
        start -= 1

# Use it the same way!
for num in countdown(5):
    print(num)
# Output: 5, 4, 3, 2, 1

yield vs return

graph TD
    A["Function Called"] --> B{return or yield?}
    B -->|return| C["Function ends forever 💀"]
    B -->|yield| D["Function pauses ⏸️"]
    D --> E["Next call? Resume! ▶️"]

🎯 The yield Statement Deep Dive

How yield Works

Think of yield as a bookmark in a book:

1. You read until the bookmark
2. You close the book (pause)
3. Later, you open it and continue from the bookmark!

def story_teller():
    yield "Once upon a time..."
    yield "There was a Python..."
    yield "The end!"

story = story_teller()
print(next(story))  # Once upon a time...
print(next(story))  # There was a Python...
print(next(story))  # The end!

Generators Remember Their State!

def counter():
    count = 0
    while True:
        count += 1
        yield count

# This generator remembers!
c = counter()
print(next(c))  # 1
print(next(c))  # 2
print(next(c))  # 3
# It never forgets where it was!

🔗 yield from - The Delegation Master

Sometimes you want to yield items from another iterable. Instead of:

def nested():
    for i in [1, 2, 3]:
        yield i
    for i in [4, 5, 6]:
        yield i

Use yield from:

def nested():
    yield from [1, 2, 3]
    yield from [4, 5, 6]

Real Example: Flattening Nested Lists

def flatten(nested_list):
    for item in nested_list:
        if isinstance(item, list):
            yield from flatten(item)
        else:
            yield item

# Flatten this mess!
messy = [1, [2, 3], [4, [5, 6]]]
print(list(flatten(messy)))
# Output: [1, 2, 3, 4, 5, 6]

🎯 yield from = “Hey, let this other thing yield for me!”

💨 Generator Expressions - One-Liners!

Remember list comprehensions? Generator expressions are their memory-efficient cousins!

Syntax Comparison

# List comprehension (creates entire list)
squares_list = [x**2 for x in range(5)]

# Generator expression (creates one at a time)
squares_gen = (x**2 for x in range(5))

Spot the difference? [ ] vs ( )

When to Use Which?

Example: Sum of Millions

# Bad: Creates 1 million numbers in memory
total = sum([x for x in range(1000000)])

# Good: One number at a time!
total = sum(x for x in range(1000000))

🧰 The itertools Module - Your Superpower Toolkit!

Python’s itertools is like a Swiss Army knife for iterators. It’s in the standard library - no install needed!

import itertools

🔢 Infinite Iterators

# count: 1, 2, 3, 4, ... forever!
for i in itertools.count(1):
    if i > 5: break
    print(i)  # 1, 2, 3, 4, 5

# cycle: A, B, C, A, B, C, A... forever!
colors = itertools.cycle(['red', 'green', 'blue'])
for _ in range(5):
    print(next(colors))

# repeat: same thing, forever (or n times)
for x in itertools.repeat("Hello", 3):
    print(x)  # Hello Hello Hello

🔗 Combining Iterators

# chain: connect multiple iterables
letters = itertools.chain("AB", "CD", "EF")
print(list(letters))  # ['A','B','C','D','E','F']

# zip_longest: zip with fill value
names = ['Alice', 'Bob']
ages = [25, 30, 35]
result = itertools.zip_longest(names, ages, fillvalue='?')
print(list(result))
# [('Alice',25), ('Bob',30), ('?',35)]

✂️ Filtering Iterators

# takewhile: take until condition fails
nums = [1, 3, 5, 7, 2, 4, 6]
small = itertools.takewhile(lambda x: x < 6, nums)
print(list(small))  # [1, 3, 5]

# dropwhile: skip until condition fails
big = itertools.dropwhile(lambda x: x < 6, nums)
print(list(big))  # [7, 2, 4, 6]

# filterfalse: opposite of filter
evens = itertools.filterfalse(
    lambda x: x % 2, range(10)
)
print(list(evens))  # [0, 2, 4, 6, 8]

🎰 Combinatoric Iterators

# permutations: all arrangements
perms = itertools.permutations('ABC', 2)
print(list(perms))
# [('A','B'),('A','C'),('B','A'),
#  ('B','C'),('C','A'),('C','B')]

# combinations: unique groups (order doesn't matter)
combs = itertools.combinations('ABC', 2)
print(list(combs))
# [('A','B'), ('A','C'), ('B','C')]

# product: cartesian product (like nested loops)
prod = itertools.product([1,2], ['a','b'])
print(list(prod))
# [(1,'a'), (1,'b'), (2,'a'), (2,'b')]

🎓 Quick Reference Table

🌟 The Golden Rule

“Don’t load what you don’t need.”

If you’re processing 1 million items but only need to look at them one at a time, use iterators and generators. Your computer’s memory will thank you!

graph TD
    A["Need to process data?"] --> B{How much?}
    B -->|Small| C["List is fine ✅"]
    B -->|Large/Infinite| D["Use Generators! 🚀"]
    D --> E["💾 Memory saved!"]
    D --> F["⚡ Faster start!"]
    D --> G["♾️ Can be infinite!"]

🎉 You Did It!

You now understand:

• ✅ The Iterator Protocol (__iter__ and __next__)
• ✅ Creating Custom Iterators
• ✅ Generator Functions with yield
• ✅ The yield from delegation
• ✅ Generator Expressions (x for x in ...)
• ✅ The itertools module superpowers

Remember: Generators are like a patient chef who cooks one dish at a time, rather than preparing everything and running out of counter space. Smart, efficient, and elegant!

Now go forth and generate some amazing code! 🚀