Asynchronicity - parallel observations

Welcome to Module 6, @name! Darwin here with an exciting topic - asynchronous programming in Python!

Up until now, all your programs have been synchronous - executing operations one after another. Now you'll learn about asynchronicity - a technique that allows you to perform many tasks simultaneously!

Safari Analogy: Synchronous programming is like one guide observing animals one by one - first lions (waits 30 minutes), then elephants (waits 20 minutes), then cheetahs (waits 15 minutes). Asynchronous is like many guides observing simultaneously - one at the lions, another at the elephants, a third at the cheetahs - all in parallel! ⏱

The problem with synchronous code

Synchronous code - waiting in sequence

1import time
2
3def observe_lions():
4    """Observing lions - takes 3 seconds"""
5    print("Starting lion observation...")
6    time.sleep(3)  # Simulating wait
7    print("Lion observation complete!")
8    return {"species": "Lion", "count": 5}
9
10def observe_elephants():
11    """Observing elephants - takes 2 seconds"""
12    print("Starting elephant observation...")
13    time.sleep(2)
14    print("Elephant observation complete!")
15    return {"species": "Elephant", "count": 3}
16
17def observe_cheetahs():
18    """Observing cheetahs - takes 4 seconds"""
19    print("Starting cheetah observation...")
20    time.sleep(4)
21    print("Cheetah observation complete!")
22    return {"species": "Cheetah", "count": 2}
23
24# Synchronous execution - ONE BY ONE
25start = time.time()
26result1 = observe_lions()      # Waits 3s
27result2 = observe_elephants()  # Then waits 2s
28result3 = observe_cheetahs()   # Then waits 4s
29end = time.time()
30
31print(f"\nTotal time: {end - start:.1f}s")
32# Output: Total time: 9.0s (3 + 2 + 4)

Problem: Time = 9 seconds! We have to wait for each observation in sequence, even though we could observe in parallel! ⏰

The solution - asynchronous code

Asynchronous code - parallel waiting

1import asyncio
2
3async def observe_lions():
4    """Asynchronous lion observation"""
5    print("Starting lion observation...")
6    await asyncio.sleep(3)  # await instead of time.sleep!
7    print("Lion observation complete!")
8    return {"species": "Lion", "count": 5}
9
10async def observe_elephants():
11    """Asynchronous elephant observation"""
12    print("Starting elephant observation...")
13    await asyncio.sleep(2)
14    print("Elephant observation complete!")
15    return {"species": "Elephant", "count": 3}
16
17async def observe_cheetahs():
18    """Asynchronous cheetah observation"""
19    print("Starting cheetah observation...")
20    await asyncio.sleep(4)
21    print("Cheetah observation complete!")
22    return {"species": "Cheetah", "count": 2}
23
24async def main():
25    """Main async function"""
26    start = time.time()
27
28    # Execute in parallel!
29    results = await asyncio.gather(
30        observe_lions(),
31        observe_elephants(),
32        observe_cheetahs()
33    )
34
35    end = time.time()
36    print(f"\nTotal time: {end - start:.1f}s")
37    # Output: Total time: 4.0s (max of 3, 2, 4)
38    print(f"Results: {results}")
39
40# Run async main
41asyncio.run(main())

Result: Time = 4 seconds! All observations execute in parallel!

Savings: 9s → 4s = 5 seconds faster (55% reduction)!

Async/await - syntax

async def - defining an asynchronous function

1# Synchronous function
2def sync_function():
3    return "Sync result"
4
5# Asynchronous function (coroutine)
6async def async_function():
7    return "Async result"

async def

creates a coroutine - a special function that can be "suspended" (

await

) and resumed later.

await - waiting for a result

1async def fetch_species_data(species_id):
2    print(f"Fetching data for species {species_id}...")
3    await asyncio.sleep(2)  # Simulating I/O operation
4    return {"id": species_id, "name": "Lion", "population": 120}
5
6async def main():
7    # await suspends execution until a result is received
8    data = await fetch_species_data(1)
9    print(f"Received data: {data}")
10
11asyncio.run(main())

await

says: "Wait for the result, but in the meantime let other tasks run".

IMPORTANT:

await

can only be used inside

async def

!

Event Loop - the async engine

The event loop is the heart of asynchronicity - it manages all coroutines and switches between them.

Analogy: The event loop is like a safari dispatcher - it assigns guides to different tasks, switches between them, and collects results!

1import asyncio
2
3async def task1():
4    print("Task 1 start")
5    await asyncio.sleep(1)
6    print("Task 1 end")
7
8async def task2():
9    print("Task 2 start")
10    await asyncio.sleep(0.5)
11    print("Task 2 end")
12
13async def main():
14    # Event loop manages these tasks
15    await asyncio.gather(task1(), task2())
16
17# asyncio.run() creates the event loop and executes main()
18asyncio.run(main())

Output:

1Task 1 start
2Task 2 start
3Task 2 end  (after 0.5s)
4Task 1 end  (after 1s)

The event loop was switching between task1 and task2!

asyncio.gather() - parallel execution

asyncio.gather()

executes multiple coroutines in parallel and returns a list of results:

1import asyncio
2
3async def get_species(species_id):
4    await asyncio.sleep(1)
5    return {"id": species_id, "name": f"Species {species_id}"}
6
7async def main():
8    # Fetch 5 species in parallel
9    results = await asyncio.gather(
10        get_species(1),
11        get_species(2),
12        get_species(3),
13        get_species(4),
14        get_species(5)
15    )
16
17    print(f"Fetched {len(results)} species:")
18    for species in results:
19        print(f"  - {species['name']}")
20
21asyncio.run(main())

Time: 1 second (instead of 5 seconds synchronously)!

gather()

returns results in the same order as you provided the coroutines!

asyncio.create_task() - run in the background

create_task()

starts a coroutine in the background (does not wait for the result):

1async def background_observation(species):
2    print(f"Starting observation of {species}...")
3    await asyncio.sleep(3)
4    print(f"Observation of {species} complete!")
5
6async def main():
7    # Run in the background
8    task1 = asyncio.create_task(background_observation("Lion"))
9    task2 = asyncio.create_task(background_observation("Elephant"))
10
11    print("Doing other things in the meantime...")
12    await asyncio.sleep(1)
13    print("Still doing other things...")
14
15    # Wait for tasks to complete
16    await task1
17    await task2
18
19asyncio.run(main())

Output:

1Starting observation of Lion...
2Starting observation of Elephant...
3Doing other things in the meantime...
4Still doing other things...
5Observation of Lion complete!
6Observation of Elephant complete!

Async comprehensions

Python allows async list/dict comprehensions:

1async def get_population(species_id):
2    await asyncio.sleep(0.1)
3    return species_id * 10
4
5async def main():
6    # Async list comprehension
7    populations = [await get_population(i) for i in range(1, 6)]
8    print(f"Populations: {populations}")
9    # Output: Populations: [10, 20, 30, 40, 50]
10
11    # But this executes SEQUENTIALLY!
12    # Better to use gather():
13    populations = await asyncio.gather(
14        *[get_population(i) for i in range(1, 6)]
15    )
16    print(f"Populations (parallel): {populations}")
17
18asyncio.run(main())

Practical example - Safari API Client

An asynchronous Safari API client:

1import asyncio
2import aiohttp  # pip install aiohttp
3
4class SafariAPIClient:
5    def __init__(self, base_url: str):
6        self.base_url = base_url
7
8    async def get_species(self, species_id: int):
9        """Fetch species data"""
10        async with aiohttp.ClientSession() as session:
11            async with session.get(f"{self.base_url}/species/{species_id}") as response:
12                return await response.json()
13
14    async def get_multiple_species(self, species_ids: list[int]):
15        """Fetch multiple species in parallel"""
16        async with aiohttp.ClientSession() as session:
17            tasks = []
18            for species_id in species_ids:
19                task = session.get(f"{self.base_url}/species/{species_id}")
20                tasks.append(task)
21
22            responses = await asyncio.gather(*tasks)
23            results = []
24            for response in responses:
25                data = await response.json()
26                results.append(data)
27
28            return results
29
30async def main():
31    client = SafariAPIClient("https://api.safari-db.com")
32
33    # Fetch 10 species in parallel
34    species_ids = list(range(1, 11))
35    start = time.time()
36    results = await client.get_multiple_species(species_ids)
37    end = time.time()
38
39    print(f"Fetched {len(results)} species in {end - start:.2f}s")
40
41asyncio.run(main())

Synchronously: 10 requests × 0.5s = 5 seconds Asynchronously: max(0.5s) = 0.5 seconds

When to use async?

Use async when:

• I/O operations - HTTP requests, databases, files
• Many parallel operations - 100+ API requests
• WebSockets - real-time communication
• Waiting - sleep, timers, long-running computations

Do NOT use async when:

• CPU-bound - mathematical computations, image processing
• Simple code - small application without I/O
• No async libraries - the library doesn't support async

For CPU-bound use multiprocessing instead of async!

Async context managers

1class AsyncDatabaseConnection:
2    async def __aenter__(self):
3        print("Connecting to database...")
4        await asyncio.sleep(1)
5        return self
6
7    async def __aexit__(self, exc_type, exc_val, exc_tb):
8        print("Closing connection...")
9        await asyncio.sleep(0.5)
10
11    async def query(self, sql):
12        print(f"Executing: {sql}")
13        await asyncio.sleep(0.2)
14        return [{"id": 1, "name": "Lion"}]
15
16async def main():
17    async with AsyncDatabaseConnection() as db:
18        results = await db.query("SELECT * FROM species")
19        print(f"Results: {results}")
20
21asyncio.run(main())

Error handling in async

1async def risky_operation(species_id):
2    if species_id == 3:
3        raise ValueError(f"Species {species_id} does not exist!")
4    await asyncio.sleep(1)
5    return {"id": species_id}
6
7async def main():
8    try:
9        results = await asyncio.gather(
10            risky_operation(1),
11            risky_operation(2),
12            risky_operation(3),  # Error!
13            return_exceptions=True  # Return exceptions instead of raising
14        )
15
16        for i, result in enumerate(results, 1):
17            if isinstance(result, Exception):
18                print(f"Species {i}: ERROR - {result}")
19            else:
20                print(f"Species {i}: OK - {result}")
21
22    except Exception as e:
23        print(f"Error: {e}")
24
25asyncio.run(main())

Output:

1Species 1: OK - {'id': 1}
2Species 2: OK - {'id': 2}
3Species 3: ERROR - Species 3 does not exist!

Summary

In this lesson you learned:

• Sync vs Async - differences and benefits

• async def

  and

  await

  - syntax
• Event loop - the engine of asynchronicity

• asyncio.gather()

  - parallel execution

• asyncio.create_task()

  - background tasks
• Async comprehensions
• Practical Safari API examples
• When to use async (I/O) vs multiprocessing (CPU)
• Async context managers and error handling

Final Safari Analogy: Async is like many safari guides observing different species simultaneously - instead of waiting 9 seconds observing one by one (sync), they observe in parallel and finish in 4 seconds (async)! The event loop is the dispatcher coordinating all the guides! ⏱

Next lesson: Darwin will show you FastAPI - a modern async framework for building blazing-fast APIs!