Processing JSON Data and Managing Complex Data Structures

In previous exercises we learned to fetch data from API servers using Fetch and to create advanced HTTP clients for our Jurassic Park. Now it is time to learn how to efficiently process, transform, and manage the complex JSON data structures we receive from our systems.

JSON (JavaScript Object Notation) is the standard data exchange format — lightweight, human-readable, and easy for machines to process. In Jurassic Park we use JSON to represent everything — from dinosaur data to security system statuses and visitor records.

JSON Basics in JavaScript

JSON in JavaScript is represented by two main structure types:

• Objects (JavaScript objects / hashmaps)
• Arrays (lists of elements)

Example JSON structure representing a dinosaur in our park:

1const trex = {
2  "id": "dino-001",
3  "name": "Rexy",
4  "species": "Tyrannosaurus Rex",
5  "age": 25,
6  "stats": {
7    "weight": 8000,
8    "length": 12.3,
9    "height": 3.6
10  },
11  "diet": ["meat", "carrion"],
12  "threatLevel": 5,
13  "location": {
14    "enclosure": "T-Rex Kingdom",
15    "coordinates": {
16      "x": 34.5,
17      "y": 67.2
18    }
19  },
20  "behaviors": [
21    {
22      "type": "hunting",
23      "frequency": "high",
24      "times": ["morning", "evening"]
25    },
26    {
27      "type": "resting",
28      "frequency": "medium",
29      "times": ["midday"]
30    }
31  ],
32  "addedOn": "2023-06-15T08:30:00Z",
33  "active": true
34}

Parsing and Serializing JSON

JavaScript offers two fundamental methods for working with JSON:

1. JSON.parse()

   — converts a JSON string to a JavaScript object

2. JSON.stringify()

   — converts a JavaScript object to a JSON string

1// Serialize object to JSON string
2const trexString = JSON.stringify(trex);
3console.log("Serialized T-Rex:", trexString);
4
5// Parse JSON string back to object
6const parsedTrex = JSON.parse(trexString);
7console.log("Parsed T-Rex:", parsedTrex);

Additional Parameters of JSON.stringify() and JSON.parse()

JSON.stringify()

1// Format JSON with indentation for readability
2const prettyTrex = JSON.stringify(trex, null, 2);
3console.log("Formatted T-Rex:");
4console.log(prettyTrex);
5
6// Selectively serialize only chosen fields
7const simpleTrex = JSON.stringify(trex, ['id', 'name', 'species', 'age']);
8console.log("Simplified T-Rex:", simpleTrex);
9
10// Custom replacer function
11const customTrex = JSON.stringify(trex, (key, value) => {
12  // Hide exact coordinates for security reasons
13  if (key === 'coordinates') return "HIDDEN";
14  // Convert threat level codes to text description
15  if (key === 'threatLevel') {
16    const descriptions = ["harmless", "low", "moderate", "high", "extreme"];
17    return descriptions[value - 1] || value;
18  }
19  return value;
20});
21console.log("T-Rex with custom serialization:", customTrex);

Techniques for Processing Complex JSON Structures

Let's assume we received a complex JSON structure containing data about all dinosaurs in the park. How can we efficiently process such data?

1. Filtering Data

Filtering is one of the most common operations when working with data collections:

1// Assume we have an array of dinosaurs
2const dinosaurs = await api.get('/dinosaurs');
3
4// Filter only predators
5const predators = dinosaurs.filter(dino => dino.diet.includes('meat'));
6
7// Filter dinosaurs in a specific enclosure
8const dinosInEnclosure = dinosaurs.filter(dino =>
9  dino.location.enclosure === 'T-Rex Kingdom'
10);
11
12// Filter dangerous dinosaurs (threat level > 3)
13const dangerousDinos = dinosaurs.filter(dino => dino.threatLevel > 3);
14
15// Complex filtering with multiple conditions
16const dangerousPredatorsInZoneA = dinosaurs.filter(dino =>
17  dino.threatLevel > 3 &&
18  dino.diet.includes('meat') &&
19  dino.location.enclosure === 'Zone A'
20);

2. Mapping and Transforming Data

We often need to transform data from one format to another, for example to display in a user interface:

1// Create simplified dinosaur list for UI
2const simpleDinos = dinosaurs.map(dino => ({
3  id: dino.id,
4  name: dino.name,
5  species: dino.species,
6  threatLevel: dino.threatLevel
7}));
8
9// Add new fields based on existing data
10const dinosWithExtras = dinosaurs.map(dino => ({
11  ...dino, // copy all existing fields
12  ageYears: dino.age, // rename field
13  threatDescription: ['harmless', 'low', 'moderate', 'high', 'extreme'][dino.threatLevel - 1],
14  weightTons: dino.stats.weight / 1000,
15  needsGuard: dino.threatLevel > 3
16}));
17
18// Transform to specific format for a chart
19const chartData = dinosaurs.map(dino => ({
20  x: dino.stats.length,
21  y: dino.stats.weight,
22  r: dino.threatLevel * 5, // point size proportional to threat
23  label: dino.name,
24  color: dino.diet.includes('meat') ? 'red' : 'green'
25}));

3. Aggregation and Data Reduction

Reduction allows aggregating data to obtain summaries and statistics:

1// Calculate average weight of all dinosaurs
2const avgWeight = dinosaurs.reduce((sum, dino) => sum + dino.stats.weight, 0) / dinosaurs.length;
3
4// Find the longest dinosaur
5const longest = dinosaurs.reduce((longest, dino) =>
6  dino.stats.length > longest.stats.length ? dino : longest
7, dinosaurs[0]);
8
9// Group dinosaurs by species
10const dinosBySpecies = dinosaurs.reduce((groups, dino) => {
11  // If the group for this species does not exist, create it
12  if (!groups[dino.species]) {
13    groups[dino.species] = [];
14  }
15  // Add dinosaur to the appropriate group
16  groups[dino.species].push(dino);
17  return groups;
18}, {});
19
20// Count dinosaurs by threat level
21const countByThreat = dinosaurs.reduce((counter, dino) => {
22  counter[dino.threatLevel] = (counter[dino.threatLevel] || 0) + 1;
23  return counter;
24}, {});

4. Chaining Operations

Array methods can be chained, allowing concise and readable code:

1// Find the 3 most dangerous dinosaurs in Zone A
2const top3DangerousInZoneA = dinosaurs
3  .filter(dino => dino.location.enclosure === 'Zone A')
4  .sort((a, b) => b.threatLevel - a.threatLevel) // sort descending
5  .slice(0, 3) // take only first 3
6  .map(dino => ({
7    name: dino.name,
8    species: dino.species,
9    threatLevel: dino.threatLevel
10  }));
11
12// Calculate average weight per species
13const avgWeightBySpecies = Object.entries(
14  dinosaurs.reduce((groups, dino) => {
15    if (!groups[dino.species]) {
16      groups[dino.species] = { sum: 0, count: 0 };
17    }
18    groups[dino.species].sum += dino.stats.weight;
19    groups[dino.species].count += 1;
20    return groups;
21  }, {})
22).map(([species, { sum, count }]) => ({
23  species,
24  avgWeight: sum / count
25}));

5. Deep Finding and Updating Data

In complex structures we often need to find or update deeply nested data:

1// Find dinosaur based on deeply nested condition
2const dinoNearCoordinates = dinosaurs.find(dino => {
3  const { x, y } = dino.location.coordinates;
4  // Find dinosaur within radius 5 units from point (30, 70)
5  return Math.sqrt(Math.pow(x - 30, 2) + Math.pow(y - 70, 2)) < 5;
6});
7
8// Update deeply nested property (immutability)
9function updateDinoLocation(dino, newX, newY) {
10  return {
11    ...dino,
12    location: {
13      ...dino.location,
14      coordinates: {
15        ...dino.location.coordinates,
16        x: newX,
17        y: newY
18      }
19    }
20  };
21}
22
23// Usage
24const updatedDino = updateDinoLocation(trex, 35.0, 68.0);

6. Recursive Data Processing

Some data structures are hierarchical or have unknown nesting depth. Recursive processing is useful in such cases:

1// Example hierarchical dinosaur category structure
2const dinosaurCategories = {
3  "name": "Dinosaurs",
4  "subcategories": [
5    {
6      "name": "Theropods",
7      "subcategories": [
8        {
9          "name": "Tyrannosauroids",
10          "dinosaurs": [
11            { "id": "dino-001", "name": "Rexy" },
12            { "id": "dino-002", "name": "Tara" }
13          ]
14        },
15        {
16          "name": "Deinonychosaurs",
17          "dinosaurs": [
18            { "id": "dino-003", "name": "Blue" },
19            { "id": "dino-004", "name": "Delta" }
20          ]
21        }
22      ]
23    },
24    {
25      "name": "Sauropods",
26      "dinosaurs": [
27        { "id": "dino-005", "name": "Long" },
28        { "id": "dino-006", "name": "Tall" }
29      ]
30    }
31  ]
32};
33
34// Recursive function to find dinosaur by ID in the hierarchy
35function findDinosaurById(node, id) {
36  // Check if the current node has a dinosaurs array
37  if (node.dinosaurs) {
38    const found = node.dinosaurs.find(dino => dino.id === id);
39    if (found) return found;
40  }
41
42  // If the node has subcategories, search them recursively
43  if (node.subcategories) {
44    for (const subcategory of node.subcategories) {
45      const found = findDinosaurById(subcategory, id);
46      if (found) return found;
47    }
48  }
49
50  return null;
51}
52
53// Recursive function to flatten the hierarchy into a list of all dinosaurs
54function getAllDinosaurs(node, list = []) {
55  if (node.dinosaurs) {
56    list.push(...node.dinosaurs);
57  }
58
59  if (node.subcategories) {
60    for (const subcategory of node.subcategories) {
61      getAllDinosaurs(subcategory, list);
62    }
63  }
64
65  return list;
66}
67
68// Usage
69const blue = findDinosaurById(dinosaurCategories, "dino-003");
70console.log("Found dinosaur:", blue);
71
72const allDinos = getAllDinosaurs(dinosaurCategories);
73console.log("All dinosaurs:", allDinos);

Practical Example: Dinosaur Monitoring System

Now let's combine all the discussed techniques in one practical example. Let's say we are building a dashboard for the Jurassic Park manager that shows the status of all dinosaurs in the park.

1// Function to fetch data from API
2async function fetchDinosaurData() {
3  try {
4    const response = await fetch('https://api.jurassic-park.com/dinosaurs');
5    if (!response.ok) throw new Error(`HTTP Error: ${response.status}`);
6    return await response.json();
7  } catch (error) {
8    console.error('Error fetching data:', error);
9    return { dinosaurs: [] };
10  }
11}
12
13// Function to process and prepare data for display
14function processDataForDisplay(data) {
15  const { dinosaurs } = data;
16
17  // Aggregate statistics
18  const stats = {
19    count: dinosaurs.length,
20    speciesCount: new Set(dinosaurs.map(d => d.species)).size,
21    avgWeight: dinosaurs.reduce((sum, d) => sum + d.stats.weight, 0) / dinosaurs.length,
22    maxThreat: Math.max(...dinosaurs.map(d => d.threatLevel))
23  };
24
25  // Group by enclosure
26  const dinosByEnclosure = dinosaurs.reduce((groups, dino) => {
27    const enclosure = dino.location.enclosure;
28    if (!groups[enclosure]) groups[enclosure] = [];
29    groups[enclosure].push(dino);
30    return groups;
31  }, {});
32
33  // Count by threat level
34  const countByThreat = [1, 2, 3, 4, 5].map(level => ({
35    level,
36    count: dinosaurs.filter(d => d.threatLevel === level).length,
37    percent: (dinosaurs.filter(d => d.threatLevel === level).length / dinosaurs.length) * 100
38  }));
39
40  // Identify potential threats
41  const threats = dinosaurs
42    .filter(d => d.threatLevel >= 4)
43    .map(d => ({
44      id: d.id,
45      name: d.name,
46      species: d.species,
47      enclosure: d.location.enclosure,
48      threatLevel: d.threatLevel,
49      lastBehavior: d.behaviors[0]?.type || 'unknown',
50      coordinates: d.location.coordinates
51    }))
52    .sort((a, b) => b.threatLevel - a.threatLevel);
53
54  // Prepare data for weight chart
55  const weightChartData = dinosaurs
56    .filter(d => d.stats.weight > 0)
57    .map(d => ({
58      id: d.id,
59      name: d.name,
60      species: d.species,
61      weight: d.stats.weight,
62      predator: d.diet.includes('meat'),
63      pointSize: Math.sqrt(d.stats.weight) / 5
64    }));
65
66  // Recent activities (sorted by ID)
67  const recentActivities = dinosaurs
68    .filter(d => d.behaviors && d.behaviors.length > 0)
69    .flatMap(d => d.behaviors.map(b => ({
70      dinoId: d.id,
71      dinoName: d.name,
72      type: b.type,
73      frequency: b.frequency,
74      times: b.times
75    })))
76    .sort((a, b) => a.dinoId.localeCompare(b.dinoId));
77
78  return {
79    stats,
80    dinosByEnclosure,
81    countByThreat,
82    threats,
83    weightChartData,
84    recentActivities,
85    raw: dinosaurs // Original data if needed
86  };
87}
88
89// Function to render data to the user interface (example)
90function renderDashboard(processedData) {
91  console.log("Processed data for dashboard:", processedData);
92
93  // Example: render threats table
94  const threatsTable = document.getElementById('threats-table');
95  if (threatsTable) {
96    threatsTable.innerHTML = processedData.threats
97      .map(t => `
98        <tr>
99          <td>${t.name}</td>
100          <td>${t.species}</td>
101          <td>${t.enclosure}</td>
102          <td class="threat-${t.threatLevel}">${t.threatLevel}</td>
103          <td>${t.lastBehavior}</td>
104          <td>
105            <button onclick="showOnMap(${t.coordinates.x}, ${t.coordinates.y})">
106              Show on map
107            </button>
108          </td>
109        </tr>
110      `)
111      .join('');
112  }
113}
114
115// Main update function
116async function updateDashboard() {
117  const data = await fetchDinosaurData();
118  const processedData = processDataForDisplay(data);
119  renderDashboard(processedData);
120}
121
122// Run update every 5 minutes
123updateDashboard();
124setInterval(updateDashboard, 5 * 60 * 1000);

Summary

In this exercise we learned:

1. JSON serialization and deserialization basics — how to convert data between text format and JavaScript objects

2. Advanced data processing techniques:

   • Filtering data using

     .filter()

   • Transforming data using

     .map()

   • Aggregating data using

     .reduce()

   • Chaining operations
   • Working with deeply nested structures
   • Recursive processing of hierarchical data

3. Practical application of these techniques in the context of a dinosaur park management system

The ability to efficiently process and analyze JSON data is crucial for every modern JavaScript project — from simple websites to complex applications like the Jurassic Park Management System. Thanks to the techniques you have learned, you will be able to transform raw data into useful information that helps keep the park running smoothly and safely... at least until a raptor finds a gap in the system!