Python course ยท Module 4: Data, APIs and Databases

REST API - data exchange architecture

11 min read ยท
In this lesson11

Welcome back! Darwin here with another lesson on data flow.

You've learned data formats (JSON, CSV, XML) and the HTTP protocol (GET, POST, PUT, DELETE). Now it's time for REST API - the standard architecture for building APIs used worldwide!

Safari Analogy: HTTP is the courier system. REST API is the standard procedures - the rules by which you organize research stations, data catalogs, and communication between them. Thanks to these rules, any biologist, regardless of where they come from, knows how to fetch lion data: GET /api/species/lion

What is REST API?

REST (REpresentational State Transfer) is an architectural style for APIs - a set of rules and conventions for designing APIs that use HTTP.

API (Application Programming Interface) is a programming interface - the way applications communicate with each other.

REST API = HTTP API designed according to REST principles

Why REST?

Without standards, every API would look different:

  • /getLion?id=123
  • /fetchLionData
  • /lion-information/get
  • /api/v1/animals/lion/retrieve

With REST everyone knows:

  • GET /api/species/lion - fetch a lion
  • POST /api/species - create a species
  • DELETE /api/species/lion - delete a lion

Predictable, consistent, intuitive!

REST Principles

1. Resources - nouns, not verbs

In REST everything is a resource - an object, entity, or data.

Good - nouns:

  • /species - species
  • /observations - observations
  • /habitats - habitats
  • /researchers - researchers

Bad - verbs:

  • /getSpecies
  • /createObservation
  • /deleteHabitat

The action is determined by the HTTP method, not the URL!

2. HTTP methods = actions on resources

MethodActionExample
GETRetrieve resource(s)GET /species - list, GET /species/lion - one
POSTCreate new resourcePOST /species - add species
PUTReplace entire resourcePUT /species/lion - replace lion data
PATCHUpdate part of resourcePATCH /species/lion - change population
DELETEDelete resourceDELETE /species/lion - delete lion

3. Stateless

Each request must contain all the information needed to process it. The server does not remember previous requests.

Example:

# Each request contains the authorization token
GET /api/species/lion
Authorization: Bearer abc123

GET /api/observations
Authorization: Bearer abc123

The server doesn't know "who sent this earlier" - each request is self-contained!

4. Resource hierarchy

Resources can be nested hierarchically:

/species/lion                    # Species: lion
/species/lion/observations       # Lion observations
/species/lion/observations/123   # Specific lion observation #123

/habitats/serengeti              # Habitat: Serengeti
/habitats/serengeti/species      # Species in Serengeti

Rule: /resource/id/subresource/id

5. API versioning

APIs evolve - versioning prevents breaking existing clients:

/api/v1/species
/api/v2/species

Change version when:

  • You remove fields from responses
  • You change the data format
  • You remove endpoints

RESTful Endpoints - standard patterns

CRUD (Create, Read, Update, Delete) for a resource

# LIST - list all species
GET /api/species
Response: [{"id": "lion", "name": "Lion"}, {"id": "elephant", "name": "Elephant"}]

# GET - one species
GET /api/species/lion
Response: {"id": "lion", "scientific_name": "Panthera leo", "population": 120}

# CREATE - create new species
POST /api/species
Body: {"scientific_name": "Gorilla gorilla", "population": 230}
Response: {"id": "gorilla", "scientific_name": "Gorilla gorilla", "population": 230}

# UPDATE (full) - replace species
PUT /api/species/lion
Body: {"scientific_name": "Panthera leo", "population": 125, "habitat": "savanna"}
Response: {"id": "lion", ...}

# UPDATE (partial) - change only population
PATCH /api/species/lion
Body: {"population": 125}
Response: {"id": "lion", "population": 125, ...}

# DELETE - delete species
DELETE /api/species/lion
Response: 204 No Content (or 200 with confirmation)

Nested resources

# List observations for a lion
GET /api/species/lion/observations
Response: [
  {"id": "obs1", "date": "2024-01-15", "count": 12},
  {"id": "obs2", "date": "2024-01-20", "count": 8}
]

# Specific lion observation
GET /api/species/lion/observations/obs1
Response: {"id": "obs1", "species": "lion", "date": "2024-01-15", "count": 12}

# Add observation for a lion
POST /api/species/lion/observations
Body: {"date": "2024-01-25", "count": 10, "location": "Serengeti"}
Response: {"id": "obs3", "species": "lion", ...}

Query Parameters - filtering, sorting, pagination

Filtering

# Endangered species
GET /api/species?endangered=true

# Species in savanna with population > 100
GET /api/species?habitat=savanna&min_population=100

# Observations within a date range
GET /api/observations?start_date=2024-01-01&end_date=2024-01-31

Sorting

# Sort by population ascending
GET /api/species?sort=population

# Sort by population descending
GET /api/species?sort=-population

# Sort by name, then population
GET /api/species?sort=name,population

Pagination

# Page 1, 20 results per page
GET /api/species?page=1&per_page=20

# Alternative: offset and limit
GET /api/species?offset=0&limit=20

# Response with pagination metadata:
{
  "data": [...],
  "pagination": {
    "current_page": 1,
    "per_page": 20,
    "total_items": 150,
    "total_pages": 8
  }
}

Field selection

# Only name and population
GET /api/species?fields=name,population

Response: [
  {"name": "Lion", "population": 120},
  {"name": "Elephant", "population": 450}
]

JSON Response Structure

Success - single resource

{
  "id": "lion",
  "scientific_name": "Panthera leo",
  "common_name": "Lion",
  "population": 120,
  "habitat": "savanna",
  "endangered": true,
  "created_at": "2024-01-01T10:00:00Z",
  "updated_at": "2024-01-15T14:30:00Z"
}

Success - resource list with pagination

{
  "data": [
    {"id": "lion", "name": "Lion", "population": 120},
    {"id": "elephant", "name": "Elephant", "population": 450}
  ],
  "pagination": {
    "page": 1,
    "per_page": 20,
    "total": 45,
    "pages": 3
  },
  "links": {
    "self": "/api/species?page=1",
    "next": "/api/species?page=2",
    "last": "/api/species?page=3"
  }
}

Error - standard format

{
  "error": {
    "code": "SPECIES_NOT_FOUND",
    "message": "Species 'unicorn' does not exist",
    "status": 404,
    "timestamp": "2024-01-15T14:30:00Z"
  }
}

Validation errors

{
  "error": {
    "code": "VALIDATION_ERROR",
    "message": "Data validation errors",
    "status": 400,
    "details": [
      {"field": "population", "message": "Must be a number greater than 0"},
      {"field": "habitat", "message": "Field required"}
    ]
  }
}

HTTP Status Codes in REST

2xx - Success

  • 200 OK - GET, PUT, PATCH - success with data
  • 201 Created - POST - resource created (+ Location header)
  • 204 No Content - DELETE - success without data

4xx - Client Error

  • 400 Bad Request - invalid data (e.g., bad JSON, validation)
  • 401 Unauthorized - no authentication
  • 403 Forbidden - no permissions (authenticated, but no access)
  • 404 Not Found - resource doesn't exist
  • 409 Conflict - conflict (e.g., duplicate ID)
  • 422 Unprocessable Entity - valid format, but logically incorrect data
  • 429 Too Many Requests - rate limit exceeded

5xx - Server Error

  • 500 Internal Server Error - server error
  • 503 Service Unavailable - service unavailable

Authentication and Authorization

API Key

GET /api/species
X-API-Key: your_api_key_here

Bearer Token (JWT)

GET /api/species
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...

Basic Auth

GET /api/species
Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=

Most common: Bearer Token (OAuth 2.0, JWT)

Safari example - full REST API client

import requests
from typing import List, Dict, Optional, Any
from datetime import datetime

class SafariRESTClient:
    """REST API client for Safari Data API"""

    def __init__(self, base_url: str, api_key: str):
        self.base_url = base_url.rstrip("/")
        self.session = requests.Session()
        self.session.headers.update({
            "Authorization": f"Bearer {api_key}",
            "Accept": "application/json",
            "Content-Type": "application/json"
        })

    # === SPECIES - /api/species ===

    def list_species(self, habitat: Optional[str] = None,
                    endangered: Optional[bool] = None,
                    page: int = 1, per_page: int = 20,
                    sort: str = "name") -> Dict[str, Any]:
        """
        GET /api/species
        List species with filtering, sorting, pagination
        """
        params = {
            "page": page,
            "per_page": per_page,
            "sort": sort
        }
        if habitat:
            params["habitat"] = habitat
        if endangered is not None:
            params["endangered"] = str(endangered).lower()

        response = self.session.get(f"{self.base_url}/api/species", params=params)
        response.raise_for_status()
        return response.json()

    def get_species(self, species_id: str) -> Dict[str, Any]:
        """
        GET /api/species/{id}
        Fetch a single species
        """
        response = self.session.get(f"{self.base_url}/api/species/{species_id}")
        response.raise_for_status()
        return response.json()

    def create_species(self, data: Dict[str, Any]) -> Dict[str, Any]:
        """
        POST /api/species
        Create a new species
        """
        response = self.session.post(f"{self.base_url}/api/species", json=data)
        response.raise_for_status()

        if response.status_code == 201:
            # Return created resource + ID from Location header
            location = response.headers.get("Location")
            print(f"Created: {location}")

        return response.json()

    def update_species(self, species_id: str, data: Dict[str, Any]) -> Dict[str, Any]:
        """
        PUT /api/species/{id}
        Replace entire species
        """
        response = self.session.put(
            f"{self.base_url}/api/species/{species_id}",
            json=data
        )
        response.raise_for_status()
        return response.json()

    def partial_update_species(self, species_id: str,
                              data: Dict[str, Any]) -> Dict[str, Any]:
        """
        PATCH /api/species/{id}
        Update part of a species
        """
        response = self.session.patch(
            f"{self.base_url}/api/species/{species_id}",
            json=data
        )
        response.raise_for_status()
        return response.json()

    def delete_species(self, species_id: str) -> bool:
        """
        DELETE /api/species/{id}
        Delete a species
        """
        response = self.session.delete(f"{self.base_url}/api/species/{species_id}")
        response.raise_for_status()
        return response.status_code == 204

    # === OBSERVATIONS - /api/species/{id}/observations ===

    def list_observations(self, species_id: str,
                         start_date: Optional[str] = None,
                         end_date: Optional[str] = None,
                         page: int = 1) -> Dict[str, Any]:
        """
        GET /api/species/{id}/observations
        List observations for a species
        """
        params = {"page": page}
        if start_date:
            params["start_date"] = start_date
        if end_date:
            params["end_date"] = end_date

        response = self.session.get(
            f"{self.base_url}/api/species/{species_id}/observations",
            params=params
        )
        response.raise_for_status()
        return response.json()

    def create_observation(self, species_id: str,
                          data: Dict[str, Any]) -> Dict[str, Any]:
        """
        POST /api/species/{id}/observations
        Add an observation for a species
        """
        response = self.session.post(
            f"{self.base_url}/api/species/{species_id}/observations",
            json=data
        )
        response.raise_for_status()
        return response.json()

    def get_observation(self, species_id: str, observation_id: str) -> Dict[str, Any]:
        """
        GET /api/species/{id}/observations/{obs_id}
        Fetch a specific observation
        """
        response = self.session.get(
            f"{self.base_url}/api/species/{species_id}/observations/{observation_id}"
        )
        response.raise_for_status()
        return response.json()

    # === STATISTICS - /api/statistics ===

    def get_statistics(self, group_by: str = "habitat") -> Dict[str, Any]:
        """
        GET /api/statistics
        Fetch statistics
        """
        params = {"group_by": group_by}
        response = self.session.get(
            f"{self.base_url}/api/statistics",
            params=params
        )
        response.raise_for_status()
        return response.json()

    def close(self):
        """Close session"""
        self.session.close()


# === DEMONSTRATION ===

print("=== SAFARI REST API CLIENT ===\n")

client = SafariRESTClient(
    base_url="https://api.safari-data.org",
    api_key="demo_bearer_token_12345"
)

# 1. LIST - fetch all endangered species in savanna
print("1. Species list (filtered, sorted, paginated)")
result = client.list_species(
    habitat="savanna",
    endangered=True,
    page=1,
    per_page=10,
    sort="-population"  # Descending by population
)

print(f"   Found: {result['pagination']['total']} species")
for species in result['data']:
    print(f"   - {species['common_name']}: {species['population']} individuals")

# 2. GET - fetch a specific species
print("\n2. Fetching a specific species")
lion = client.get_species("lion")
print(f"   {lion['common_name']} ({lion['scientific_name']})")
print(f"   Population: {lion['population']}")
print(f"   Endangered: {'Yes' if lion['endangered'] else 'No'}")

# 3. POST - create new species
print("\n3. Creating a new species")
new_species = {
    "scientific_name": "Acinonyx jubatus",
    "common_name": "Cheetah",
    "population": 7100,
    "habitat": "savanna",
    "endangered": True
}

created = client.create_species(new_species)
print(f"   Created: {created['common_name']} (ID: {created['id']})")

# 4. PATCH - update only population
print("\n4. Partial update (PATCH)")
updated = client.partial_update_species("lion", {"population": 125})
print(f"   Updated population: {updated['population']}")

# 5. Nested observations - list
print("\n5. List of lion observations")
observations = client.list_observations(
    "lion",
    start_date="2024-01-01",
    end_date="2024-01-31"
)

print(f"   Found {len(observations['data'])} observations in January")
for obs in observations['data']:
    print(f"   - {obs['date']}: {obs['count']} individuals in {obs['location']}")

# 6. Nested observations - create
print("\n6. Adding a new observation")
new_obs = {
    "date": datetime.now().isoformat(),
    "location": "Serengeti North",
    "count": 15,
    "notes": "Pride with 3 cubs"
}

obs_created = client.create_observation("lion", new_obs)
print(f"   Created observation ID: {obs_created['id']}")

# 7. Statistics
print("\n7. Fetching statistics")
stats = client.get_statistics(group_by="habitat")
print("   Statistics by habitat:")
for habitat_stat in stats['data']:
    print(f"   - {habitat_stat['habitat']}: {habitat_stat['species_count']} species")

# 8. DELETE - delete species
print("\n8. Deleting a species")
if client.delete_species("test-species"):
    print("   Species deleted")

client.close()
print("\nDemonstration complete")

REST API Best Practices

1. Use plural nouns

Good: /api/species, /api/observations Bad: /api/specie, /api/observation

2. Always version your API

Good: /api/v1/species Bad: /api/species (what when you change the format?)

3. Resource hierarchy no deeper than 2 levels

Good: /species/lion/observations Bad: /continents/africa/countries/kenya/parks/serengeti/species/lion/observations

4. Use snake_case or camelCase - consistently

Good: {"scientific_name": "..."} or {"scientificName": "..."} Bad: {"scientific_name": "...", "commonName": "..."} - mixed!

5. Filters, sorting, pagination via query params

Good: GET /species?habitat=savanna&sort=-population&page=2 Bad: POST /species/filter with body

6. Return appropriate status codes

  • 201 Created (not 200) for POST
  • 204 No Content for DELETE
  • 404 Not Found when resource doesn't exist
  • 400 Bad Request for validation errors

7. Use the Location header for new resources

POST /api/species
Response: 201 Created
Location: /api/species/cheetah

8. Rate limiting - communicate limits

Response Headers:
X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 999
X-RateLimit-Reset: 1640000000

Summary

In this lesson you learned:

  • What REST API is and what it's for
  • REST principles: resources, HTTP methods, stateless, hierarchy
  • RESTful endpoints: /api/v1/resources/:id
  • CRUD operations: GET, POST, PUT, PATCH, DELETE
  • Query parameters: filtering, sorting, pagination
  • JSON response structure (data, pagination, error)
  • Status codes in the context of REST
  • Authentication: API Key, Bearer Token
  • REST API best practices

Checkpoint

Before moving on:

  • You understand the difference between POST /species and GET /species/{id}
  • You can design a RESTful endpoint
  • You know how to use query params for filtering
  • You understand nested resources (/species/{id}/observations)
  • You know REST API best practices

Safari Analogy: REST API is like universal communication standards between research stations worldwide. Thanks to them, a biologist from Poland knows that GET /api/species/lion will return lion data - regardless of who built the API!

In the next lesson, Darwin will teach you web scraping - how to collect data from websites that don't have an API!

Docs entries used in this lesson

Spotted a mistake in this lesson?

Check yourself

Answer the questions from this lesson. Pick an answer to see right away whether it is correct.

  1. 1. REST stands for:

  2. 2. Which HTTP method corresponds to the UPDATE operation in CRUD?

These are 2 of 4 questions for this lesson. Solve the rest in the game.

Hands-on tasks in the game

  • Code editor

    Send a POST with data animal = {'name': 'Tygrys'} to URL '/api/animals'

  • Code editor

    Send a GET with the header Authorization: Bearer token123

  • Vertical ordering

    Arrange the stages of an HTTP request:

  • Horizontal ordering

    Arrange the elements in the correct order:

  1. QuestionREST
  2. QuestionCRUD Operations
  3. QuestionPOST Request
  4. Coding taskPOST Request
  5. Coding taskHeaders in a Request
  6. Put in orderHTTP Request Lifecycle
  7. Put in orderJSON loads
  8. QuestionCRUD - The Letter C

Useful articles