Asynchronicity - parallel observations

Welcome to Module 6, @name! Darwin here with an exciting topic - asynchronous programming in Python! ⚡🐍

Up until now, all your programs have been synchronous - executing operations one after another. Now you'll learn about asynchronicity - a technique that allows you to perform many tasks simultaneously! 🚀

Safari Analogy: Synchronous programming is like one guide observing animals one by one - first lions (waits 30 minutes), then elephants (waits 20 minutes), then cheetahs (waits 15 minutes). Asynchronous is like many guides observing simultaneously - one at the lions, another at the elephants, a third at the cheetahs - all in parallel! ⏱️🦁🐘🐆

The problem with synchronous code

Synchronous code - waiting in sequence

1import time
2
3def observe_lions():
4    """Observing lions - takes 3 seconds"""
5    print("🦁 Starting lion observation...")
6    time.sleep(3)  # Simulating wait
7    print("🦁 Lion observation complete!")
8    return {"species": "Lion", "count": 5}
9
10def observe_elephants():
11    """Observing elephants - takes 2 seconds"""
12    print("🐘 Starting elephant observation...")
13    time.sleep(2)
14    print("🐘 Elephant observation complete!")
15    return {"species": "Elephant", "count": 3}
16
17def observe_cheetahs():
18    """Observing cheetahs - takes 4 seconds"""
19    print("🐆 Starting cheetah observation...")
20    time.sleep(4)
21    print("🐆 Cheetah observation complete!")
22    return {"species": "Cheetah", "count": 2}
23
24# Synchronous execution - ONE BY ONE
25start = time.time()
26result1 = observe_lions()      # Waits 3s
27result2 = observe_elephants()  # Then waits 2s
28result3 = observe_cheetahs()   # Then waits 4s
29end = time.time()
30
31print(f"\nTotal time: {end - start:.1f}s")
32# Output: Total time: 9.0s (3 + 2 + 4)

Problem: Time = 9 seconds! We have to wait for each observation in sequence, even though we could observe in parallel! ⏰❌

The solution - asynchronous code

Asynchronous code - parallel waiting

1import asyncio
2
3async def observe_lions():
4    """Asynchronous lion observation"""
5    print("🦁 Starting lion observation...")
6    await asyncio.sleep(3)  # await instead of time.sleep!
7    print("🦁 Lion observation complete!")
8    return {"species": "Lion", "count": 5}
9
10async def observe_elephants():
11    """Asynchronous elephant observation"""
12    print("🐘 Starting elephant observation...")
13    await asyncio.sleep(2)
14    print("🐘 Elephant observation complete!")
15    return {"species": "Elephant", "count": 3}
16
17async def observe_cheetahs():
18    """Asynchronous cheetah observation"""
19    print("🐆 Starting cheetah observation...")
20    await asyncio.sleep(4)
21    print("🐆 Cheetah observation complete!")
22    return {"species": "Cheetah", "count": 2}
23
24async def main():
25    """Main async function"""
26    start = time.time()
27
28    # Execute in parallel!
29    results = await asyncio.gather(
30        observe_lions(),
31        observe_elephants(),
32        observe_cheetahs()
33    )
34
35    end = time.time()
36    print(f"\nTotal time: {end - start:.1f}s")
37    # Output: Total time: 4.0s (max of 3, 2, 4)
38    print(f"Results: {results}")
39
40# Run async main
41asyncio.run(main())

Result: Time = 4 seconds! All observations execute in parallel! ⚡✅

Savings: 9s → 4s = 5 seconds faster (55% reduction)!

Async/await - syntax

async def - defining an asynchronous function

1# Synchronous function
2def sync_function():
3    return "Sync result"
4
5# Asynchronous function (coroutine)
6async def async_function():
7    return "Async result"

async def

creates a coroutine - a special function that can be "suspended" (

await

) and resumed later.

await - waiting for a result

1async def fetch_species_data(species_id):
2    print(f"Fetching data for species {species_id}...")
3    await asyncio.sleep(2)  # Simulating I/O operation
4    return {"id": species_id, "name": "Lion", "population": 120}
5
6async def main():
7    # await suspends execution until a result is received
8    data = await fetch_species_data(1)
9    print(f"Received data: {data}")
10
11asyncio.run(main())

await

says: "Wait for the result, but in the meantime let other tasks run".

⚠️ IMPORTANT:

await

can only be used inside

async def

!

Event Loop - the async engine

The event loop is the heart of asynchronicity - it manages all coroutines and switches between them.

Analogy: The event loop is like a safari dispatcher - it assigns guides to different tasks, switches between them, and collects results! 🎯

1import asyncio
2
3async def task1():
4    print("Task 1 start")
5    await asyncio.sleep(1)
6    print("Task 1 end")
7
8async def task2():
9    print("Task 2 start")
10    await asyncio.sleep(0.5)
11    print("Task 2 end")
12
13async def main():
14    # Event loop manages these tasks
15    await asyncio.gather(task1(), task2())
16
17# asyncio.run() creates the event loop and executes main()
18asyncio.run(main())

Output:

1Task 1 start
2Task 2 start
3Task 2 end  (after 0.5s)
4Task 1 end  (after 1s)

The event loop was switching between task1 and task2!

asyncio.gather() - parallel execution

asyncio.gather()

executes multiple coroutines in parallel and returns a list of results:

1import asyncio
2
3async def get_species(species_id):
4    await asyncio.sleep(1)
5    return {"id": species_id, "name": f"Species {species_id}"}
6
7async def main():
8    # Fetch 5 species in parallel
9    results = await asyncio.gather(
10        get_species(1),
11        get_species(2),
12        get_species(3),
13        get_species(4),
14        get_species(5)
15    )
16
17    print(f"Fetched {len(results)} species:")
18    for species in results:
19        print(f"  - {species['name']}")
20
21asyncio.run(main())

Time: 1 second (instead of 5 seconds synchronously)! ⚡

gather()

returns results in the same order as you provided the coroutines!

asyncio.create_task() - run in the background

create_task()

starts a coroutine in the background (does not wait for the result):

1async def background_observation(species):
2    print(f"🔍 Starting observation of {species}...")
3    await asyncio.sleep(3)
4    print(f"✅ Observation of {species} complete!")
5
6async def main():
7    # Run in the background
8    task1 = asyncio.create_task(background_observation("Lion"))
9    task2 = asyncio.create_task(background_observation("Elephant"))
10
11    print("Doing other things in the meantime...")
12    await asyncio.sleep(1)
13    print("Still doing other things...")
14
15    # Wait for tasks to complete
16    await task1
17    await task2
18
19asyncio.run(main())

Output:

1🔍 Starting observation of Lion...
2🔍 Starting observation of Elephant...
3Doing other things in the meantime...
4Still doing other things...
5✅ Observation of Lion complete!
6✅ Observation of Elephant complete!

Async comprehensions

Python allows async list/dict comprehensions:

1async def get_population(species_id):
2    await asyncio.sleep(0.1)
3    return species_id * 10
4
5async def main():
6    # Async list comprehension
7    populations = [await get_population(i) for i in range(1, 6)]
8    print(f"Populations: {populations}")
9    # Output: Populations: [10, 20, 30, 40, 50]
10
11    # But this executes SEQUENTIALLY!
12    # Better to use gather():
13    populations = await asyncio.gather(
14        *[get_population(i) for i in range(1, 6)]
15    )
16    print(f"Populations (parallel): {populations}")
17
18asyncio.run(main())

Practical example - Safari API Client

An asynchronous Safari API client:

1import asyncio
2import aiohttp  # pip install aiohttp
3
4class SafariAPIClient:
5    def __init__(self, base_url: str):
6        self.base_url = base_url
7
8    async def get_species(self, species_id: int):
9        """Fetch species data"""
10        async with aiohttp.ClientSession() as session:
11            async with session.get(f"{self.base_url}/species/{species_id}") as response:
12                return await response.json()
13
14    async def get_multiple_species(self, species_ids: list[int]):
15        """Fetch multiple species in parallel"""
16        async with aiohttp.ClientSession() as session:
17            tasks = []
18            for species_id in species_ids:
19                task = session.get(f"{self.base_url}/species/{species_id}")
20                tasks.append(task)
21
22            responses = await asyncio.gather(*tasks)
23            results = []
24            for response in responses:
25                data = await response.json()
26                results.append(data)
27
28            return results
29
30async def main():
31    client = SafariAPIClient("https://api.safari-db.com")
32
33    # Fetch 10 species in parallel
34    species_ids = list(range(1, 11))
35    start = time.time()
36    results = await client.get_multiple_species(species_ids)
37    end = time.time()
38
39    print(f"Fetched {len(results)} species in {end - start:.2f}s")
40
41asyncio.run(main())

Synchronously: 10 requests × 0.5s = 5 seconds Asynchronously: max(0.5s) = 0.5 seconds ⚡

When to use async?

✅ Use async when:

• I/O operations - HTTP requests, databases, files
• Many parallel operations - 100+ API requests
• WebSockets - real-time communication
• Waiting - sleep, timers, long-running computations

❌ Do NOT use async when:

• CPU-bound - mathematical computations, image processing
• Simple code - small application without I/O
• No async libraries - the library doesn't support async

For CPU-bound use multiprocessing instead of async!

Async context managers

1class AsyncDatabaseConnection:
2    async def __aenter__(self):
3        print("Connecting to database...")
4        await asyncio.sleep(1)
5        return self
6
7    async def __aexit__(self, exc_type, exc_val, exc_tb):
8        print("Closing connection...")
9        await asyncio.sleep(0.5)
10
11    async def query(self, sql):
12        print(f"Executing: {sql}")
13        await asyncio.sleep(0.2)
14        return [{"id": 1, "name": "Lion"}]
15
16async def main():
17    async with AsyncDatabaseConnection() as db:
18        results = await db.query("SELECT * FROM species")
19        print(f"Results: {results}")
20
21asyncio.run(main())

Error handling in async

1async def risky_operation(species_id):
2    if species_id == 3:
3        raise ValueError(f"Species {species_id} does not exist!")
4    await asyncio.sleep(1)
5    return {"id": species_id}
6
7async def main():
8    try:
9        results = await asyncio.gather(
10            risky_operation(1),
11            risky_operation(2),
12            risky_operation(3),  # Error!
13            return_exceptions=True  # Return exceptions instead of raising
14        )
15
16        for i, result in enumerate(results, 1):
17            if isinstance(result, Exception):
18                print(f"Species {i}: ERROR - {result}")
19            else:
20                print(f"Species {i}: OK - {result}")
21
22    except Exception as e:
23        print(f"Error: {e}")
24
25asyncio.run(main())

Output:

1Species 1: OK - {'id': 1}
2Species 2: OK - {'id': 2}
3Species 3: ERROR - Species 3 does not exist!

Summary

In this lesson you learned:

• ✅ Sync vs Async - differences and benefits
• ✅

  async def

  and

  await

  - syntax
• ✅ Event loop - the engine of asynchronicity
• ✅

  asyncio.gather()

  - parallel execution
• ✅

  asyncio.create_task()

  - background tasks
• ✅ Async comprehensions
• ✅ Practical Safari API examples
• ✅ When to use async (I/O) vs multiprocessing (CPU)
• ✅ Async context managers and error handling

Final Safari Analogy: Async is like many safari guides observing different species simultaneously - instead of waiting 9 seconds observing one by one (sync), they observe in parallel and finish in 4 seconds (async)! The event loop is the dispatcher coordinating all the guides! ⏱️🦁🐘🐆

Next lesson: Darwin will show you FastAPI - a modern async framework for building blazing-fast APIs! 🚀🌐