JavaScript and TypeScript course ยท Module 4: Asynchronous JavaScript
Processing JSON Data and Managing Complex Data Structures
In this lesson4
In previous exercises we learned to fetch data from API servers using Fetch and to create advanced HTTP clients for our Jurassic Park. Now it is time to learn how to efficiently process, transform, and manage the complex JSON data structures we receive from our systems.
JSON (JavaScript Object Notation) is the standard data exchange format - lightweight, human-readable, and easy for machines to process. In Jurassic Park we use JSON to represent everything - from dinosaur data to security system statuses and visitor records.
JSON Basics in JavaScript
JSON in JavaScript is represented by two main structure types:
- Objects (JavaScript objects / hashmaps)
- Arrays (lists of elements)
Example JSON structure representing a dinosaur in our park:
const trex = {
"id": "dino-001",
"name": "Rexy",
"species": "Tyrannosaurus Rex",
"age": 25,
"stats": {
"weight": 8000,
"length": 12.3,
"height": 3.6
},
"diet": ["meat", "carrion"],
"threatLevel": 5,
"location": {
"enclosure": "T-Rex Kingdom",
"coordinates": {
"x": 34.5,
"y": 67.2
}
},
"behaviors": [
{
"type": "hunting",
"frequency": "high",
"times": ["morning", "evening"]
},
{
"type": "resting",
"frequency": "medium",
"times": ["midday"]
}
],
"addedOn": "2023-06-15T08:30:00Z",
"active": true
}Parsing and Serializing JSON
JavaScript offers two fundamental methods for working with JSON:
JSON.parse()- converts a JSON string to a JavaScript objectJSON.stringify()- converts a JavaScript object to a JSON string
// Serialize object to JSON string
const trexString = JSON.stringify(trex);
console.log("Serialized T-Rex:", trexString);
// Parse JSON string back to object
const parsedTrex = JSON.parse(trexString);
console.log("Parsed T-Rex:", parsedTrex);Additional Parameters of JSON.stringify() and JSON.parse()
JSON.stringify() has additional parameters that can be useful:
// Format JSON with indentation for readability
const prettyTrex = JSON.stringify(trex, null, 2);
console.log("Formatted T-Rex:");
console.log(prettyTrex);
// Selectively serialize only chosen fields
const simpleTrex = JSON.stringify(trex, ['id', 'name', 'species', 'age']);
console.log("Simplified T-Rex:", simpleTrex);
// Custom replacer function
const customTrex = JSON.stringify(trex, (key, value) => {
// Hide exact coordinates for security reasons
if (key === 'coordinates') return "HIDDEN";
// Convert threat level codes to text description
if (key === 'threatLevel') {
const descriptions = ["harmless", "low", "moderate", "high", "extreme"];
return descriptions[value - 1] || value;
}
return value;
});
console.log("T-Rex with custom serialization:", customTrex);Techniques for Processing Complex JSON Structures
Let's assume we received a complex JSON structure containing data about all dinosaurs in the park. How can we efficiently process such data?
1. Filtering Data
Filtering is one of the most common operations when working with data collections:
// Assume we have an array of dinosaurs
const dinosaurs = await api.get('/dinosaurs');
// Filter only predators
const predators = dinosaurs.filter(dino => dino.diet.includes('meat'));
// Filter dinosaurs in a specific enclosure
const dinosInEnclosure = dinosaurs.filter(dino =>
dino.location.enclosure === 'T-Rex Kingdom'
);
// Filter dangerous dinosaurs (threat level > 3)
const dangerousDinos = dinosaurs.filter(dino => dino.threatLevel > 3);
// Complex filtering with multiple conditions
const dangerousPredatorsInZoneA = dinosaurs.filter(dino =>
dino.threatLevel > 3 &&
dino.diet.includes('meat') &&
dino.location.enclosure === 'Zone A'
);2. Mapping and Transforming Data
We often need to transform data from one format to another, for example to display in a user interface:
// Create simplified dinosaur list for UI
const simpleDinos = dinosaurs.map(dino => ({
id: dino.id,
name: dino.name,
species: dino.species,
threatLevel: dino.threatLevel
}));
// Add new fields based on existing data
const dinosWithExtras = dinosaurs.map(dino => ({
...dino, // copy all existing fields
ageYears: dino.age, // rename field
threatDescription: ['harmless', 'low', 'moderate', 'high', 'extreme'][dino.threatLevel - 1],
weightTons: dino.stats.weight / 1000,
needsGuard: dino.threatLevel > 3
}));
// Transform to specific format for a chart
const chartData = dinosaurs.map(dino => ({
x: dino.stats.length,
y: dino.stats.weight,
r: dino.threatLevel * 5, // point size proportional to threat
label: dino.name,
color: dino.diet.includes('meat') ? 'red' : 'green'
}));3. Aggregation and Data Reduction
Reduction allows aggregating data to obtain summaries and statistics:
// Calculate average weight of all dinosaurs
const avgWeight = dinosaurs.reduce((sum, dino) => sum + dino.stats.weight, 0) / dinosaurs.length;
// Find the longest dinosaur
const longest = dinosaurs.reduce((longest, dino) =>
dino.stats.length > longest.stats.length ? dino : longest
, dinosaurs[0]);
// Group dinosaurs by species
const dinosBySpecies = dinosaurs.reduce((groups, dino) => {
// If the group for this species does not exist, create it
if (!groups[dino.species]) {
groups[dino.species] = [];
}
// Add dinosaur to the appropriate group
groups[dino.species].push(dino);
return groups;
}, {});
// Count dinosaurs by threat level
const countByThreat = dinosaurs.reduce((counter, dino) => {
counter[dino.threatLevel] = (counter[dino.threatLevel] || 0) + 1;
return counter;
}, {});4. Chaining Operations
Array methods can be chained, allowing concise and readable code:
// Find the 3 most dangerous dinosaurs in Zone A
const top3DangerousInZoneA = dinosaurs
.filter(dino => dino.location.enclosure === 'Zone A')
.sort((a, b) => b.threatLevel - a.threatLevel) // sort descending
.slice(0, 3) // take only first 3
.map(dino => ({
name: dino.name,
species: dino.species,
threatLevel: dino.threatLevel
}));
// Calculate average weight per species
const avgWeightBySpecies = Object.entries(
dinosaurs.reduce((groups, dino) => {
if (!groups[dino.species]) {
groups[dino.species] = { sum: 0, count: 0 };
}
groups[dino.species].sum += dino.stats.weight;
groups[dino.species].count += 1;
return groups;
}, {})
).map(([species, { sum, count }]) => ({
species,
avgWeight: sum / count
}));5. Deep Finding and Updating Data
In complex structures we often need to find or update deeply nested data:
// Find dinosaur based on deeply nested condition
const dinoNearCoordinates = dinosaurs.find(dino => {
const { x, y } = dino.location.coordinates;
// Find dinosaur within radius 5 units from point (30, 70)
return Math.sqrt(Math.pow(x - 30, 2) + Math.pow(y - 70, 2)) < 5;
});
// Update deeply nested property (immutability)
function updateDinoLocation(dino, newX, newY) {
return {
...dino,
location: {
...dino.location,
coordinates: {
...dino.location.coordinates,
x: newX,
y: newY
}
}
};
}
// Usage
const updatedDino = updateDinoLocation(trex, 35.0, 68.0);6. Recursive Data Processing
Some data structures are hierarchical or have unknown nesting depth. Recursive processing is useful in such cases:
// Example hierarchical dinosaur category structure
const dinosaurCategories = {
"name": "Dinosaurs",
"subcategories": [
{
"name": "Theropods",
"subcategories": [
{
"name": "Tyrannosauroids",
"dinosaurs": [
{ "id": "dino-001", "name": "Rexy" },
{ "id": "dino-002", "name": "Tara" }
]
},
{
"name": "Deinonychosaurs",
"dinosaurs": [
{ "id": "dino-003", "name": "Blue" },
{ "id": "dino-004", "name": "Delta" }
]
}
]
},
{
"name": "Sauropods",
"dinosaurs": [
{ "id": "dino-005", "name": "Long" },
{ "id": "dino-006", "name": "Tall" }
]
}
]
};
// Recursive function to find dinosaur by ID in the hierarchy
function findDinosaurById(node, id) {
// Check if the current node has a dinosaurs array
if (node.dinosaurs) {
const found = node.dinosaurs.find(dino => dino.id === id);
if (found) return found;
}
// If the node has subcategories, search them recursively
if (node.subcategories) {
for (const subcategory of node.subcategories) {
const found = findDinosaurById(subcategory, id);
if (found) return found;
}
}
return null;
}
// Recursive function to flatten the hierarchy into a list of all dinosaurs
function getAllDinosaurs(node, list = []) {
if (node.dinosaurs) {
list.push(...node.dinosaurs);
}
if (node.subcategories) {
for (const subcategory of node.subcategories) {
getAllDinosaurs(subcategory, list);
}
}
return list;
}
// Usage
const blue = findDinosaurById(dinosaurCategories, "dino-003");
console.log("Found dinosaur:", blue);
const allDinos = getAllDinosaurs(dinosaurCategories);
console.log("All dinosaurs:", allDinos);Practical Example: Dinosaur Monitoring System
Now let's combine all the discussed techniques in one practical example. Let's say we are building a dashboard for the Jurassic Park manager that shows the status of all dinosaurs in the park.
// Function to fetch data from API
async function fetchDinosaurData() {
try {
const response = await fetch('https://api.jurassic-park.com/dinosaurs');
if (!response.ok) throw new Error(`HTTP Error: ${response.status}`);
return await response.json();
} catch (error) {
console.error('Error fetching data:', error);
return { dinosaurs: [] };
}
}
// Function to process and prepare data for display
function processDataForDisplay(data) {
const { dinosaurs } = data;
// Aggregate statistics
const stats = {
count: dinosaurs.length,
speciesCount: new Set(dinosaurs.map(d => d.species)).size,
avgWeight: dinosaurs.reduce((sum, d) => sum + d.stats.weight, 0) / dinosaurs.length,
maxThreat: Math.max(...dinosaurs.map(d => d.threatLevel))
};
// Group by enclosure
const dinosByEnclosure = dinosaurs.reduce((groups, dino) => {
const enclosure = dino.location.enclosure;
if (!groups[enclosure]) groups[enclosure] = [];
groups[enclosure].push(dino);
return groups;
}, {});
// Count by threat level
const countByThreat = [1, 2, 3, 4, 5].map(level => ({
level,
count: dinosaurs.filter(d => d.threatLevel === level).length,
percent: (dinosaurs.filter(d => d.threatLevel === level).length / dinosaurs.length) * 100
}));
// Identify potential threats
const threats = dinosaurs
.filter(d => d.threatLevel >= 4)
.map(d => ({
id: d.id,
name: d.name,
species: d.species,
enclosure: d.location.enclosure,
threatLevel: d.threatLevel,
lastBehavior: d.behaviors[0]?.type || 'unknown',
coordinates: d.location.coordinates
}))
.sort((a, b) => b.threatLevel - a.threatLevel);
// Prepare data for weight chart
const weightChartData = dinosaurs
.filter(d => d.stats.weight > 0)
.map(d => ({
id: d.id,
name: d.name,
species: d.species,
weight: d.stats.weight,
predator: d.diet.includes('meat'),
pointSize: Math.sqrt(d.stats.weight) / 5
}));
// Recent activities (sorted by ID)
const recentActivities = dinosaurs
.filter(d => d.behaviors && d.behaviors.length > 0)
.flatMap(d => d.behaviors.map(b => ({
dinoId: d.id,
dinoName: d.name,
type: b.type,
frequency: b.frequency,
times: b.times
})))
.sort((a, b) => a.dinoId.localeCompare(b.dinoId));
return {
stats,
dinosByEnclosure,
countByThreat,
threats,
weightChartData,
recentActivities,
raw: dinosaurs // Original data if needed
};
}
// Function to render data to the user interface (example)
function renderDashboard(processedData) {
console.log("Processed data for dashboard:", processedData);
// Example: render threats table
const threatsTable = document.getElementById('threats-table');
if (threatsTable) {
threatsTable.innerHTML = processedData.threats
.map(t => `
<tr>
<td>${t.name}</td>
<td>${t.species}</td>
<td>${t.enclosure}</td>
<td class="threat-${t.threatLevel}">${t.threatLevel}</td>
<td>${t.lastBehavior}</td>
<td>
<button onclick="showOnMap(${t.coordinates.x}, ${t.coordinates.y})">
Show on map
</button>
</td>
</tr>
`)
.join('');
}
}
// Main update function
async function updateDashboard() {
const data = await fetchDinosaurData();
const processedData = processDataForDisplay(data);
renderDashboard(processedData);
}
// Run update every 5 minutes
updateDashboard();
setInterval(updateDashboard, 5 * 60 * 1000);Summary
In this exercise we learned:
JSON serialization and deserialization basics - how to convert data between text format and JavaScript objects
Advanced data processing techniques:
- Filtering data using
.filter() - Transforming data using
.map() - Aggregating data using
.reduce() - Chaining operations
- Working with deeply nested structures
- Recursive processing of hierarchical data
- Filtering data using
Practical application of these techniques in the context of a dinosaur park management system
The ability to efficiently process and analyze JSON data is crucial for every modern JavaScript project - from simple websites to complex applications like the Jurassic Park Management System. Thanks to the techniques you have learned, you will be able to transform raw data into useful information that helps keep the park running smoothly and safely... at least until a raptor finds a gap in the system!
Code for this lesson: index.js
// Processing JSON data - Jurassic Park
// Practice: filtering, mapping, reducing, recursion
// === Data of the dinosaurs in the park ===
const dinosaurs = [
{
id: "dino-001",
name: "Rexy",
species: "Tyrannosaurus Rex",
age: 25,
stats: { weight: 8000, length: 12.3, height: 3.6 },
diet: ["meat", "carrion"],
threat: 5,
location: { paddock: "T-Rex Kingdom", coordinates: { x: 34.5, y: 67.2 } },
behaviors: [
{ type: "hunting", frequency: "high", times: ["morning", "evening"] }
],
active: true
},
{
id: "dino-002",
name: "Blue",
species: "Velociraptor",
age: 8,
stats: { weight: 150, length: 3.0, height: 1.8 },
diet: ["meat"],
threat: 4,
location: { paddock: "Raptor Paddock", coordinates: { x: 50.1, y: 30.4 } },
behaviors: [
{ type: "hunting", frequency: "high", times: ["morning", "night"] },
{ type: "training", frequency: "medium", times: ["noon"] }
],
active: true
},
{
id: "dino-003",
name: "Brachio",
species: "Brachiosaurus",
age: 40,
stats: { weight: 56000, length: 25.0, height: 13.0 },
diet: ["plants"],
threat: 1,
location: { paddock: "Herbivore Valley", coordinates: { x: 20.0, y: 45.0 } },
behaviors: [
{ type: "grazing", frequency: "high", times: ["morning", "noon", "evening"] }
],
active: true
},
{
id: "dino-004",
name: "Spike",
species: "Triceratops",
age: 15,
stats: { weight: 9000, length: 9.0, height: 3.0 },
diet: ["plants"],
threat: 2,
location: { paddock: "Herbivore Valley", coordinates: { x: 22.5, y: 42.1 } },
behaviors: [
{ type: "grazing", frequency: "high", times: ["morning", "noon"] },
{ type: "defense", frequency: "low", times: ["night"] }
],
active: true
},
{
id: "dino-005",
name: "Delta",
species: "Velociraptor",
age: 6,
stats: { weight: 130, length: 2.8, height: 1.6 },
diet: ["meat"],
threat: 4,
location: { paddock: "Raptor Paddock", coordinates: { x: 51.0, y: 31.0 } },
behaviors: [
{ type: "hunting", frequency: "high", times: ["night"] }
],
active: true
}
];
// === TODO: Exercises to do ===
// 1. JSON serialization and deserialization
const jsonString = JSON.stringify(dinosaurs, null, 2);
console.log("=== JSON serialization ===");
console.log("Number of characters:", jsonString.length);
const parsedDino = JSON.parse(jsonString);
console.log("Parsed dinosaurs:", parsedDino.length);
// 2. TODO: Filtering - find all the predators
const predators = dinosaurs.filter(d => d.diet.includes("meat"));
console.log("\n=== Predators ===");
predators.forEach(d => console.log(d.name, "-", d.species));
// 3. TODO: Mapping - create a simplified list
const simplifiedList = dinosaurs.map(d => ({
name: d.name,
species: d.species,
weight_tons: d.stats.weight / 1000,
dangerous: d.threat >= 4
}));
console.log("\n=== Simplified list ===");
console.log(JSON.stringify(simplifiedList, null, 2));
// 4. TODO: Reduction - calculate the average weight
const averageWeight = dinosaurs.reduce(
(sum, d) => sum + d.stats.weight, 0
) / dinosaurs.length;
console.log("\n=== Average weight ===");
console.log(averageWeight.toFixed(1), "kg");
// 5. TODO: Grouping by paddock
const byPaddock = dinosaurs.reduce((groups, d) => {
const paddock = d.location.paddock;
if (!groups[paddock]) groups[paddock] = [];
groups[paddock].push(d.name);
return groups;
}, {});
console.log("\n=== Dinosaurs in paddocks ===");
Object.entries(byPaddock).forEach(([paddock, names]) =>
console.log(paddock + ":", names.join(", "))
);
// 6. TODO: Method chain - top 3 heaviest
const top3 = dinosaurs
.sort((a, b) => b.stats.weight - a.stats.weight)
.slice(0, 3)
.map(d => d.name + " (" + (d.stats.weight / 1000) + "t)");
console.log("\n=== Top 3 heaviest ===");
top3.forEach((info, i) => console.log((i + 1) + ".", info));
// 7. Add your own data processing!
// E.g. find the dinosaur with the longest body, count predators vs herbivores, etc.Docs entries used in this lesson
Spotted a mistake in this lesson?
Check yourself
Answer the questions from this lesson. Pick an answer to see right away whether it is correct.
1. What does the JSON.parse() method do?
2. What does the third argument in JSON.stringify(obj, null, 2) do?
Hands-on tasks in the game
- Click in order
Arrange the elements of fetching and parsing JSON from an API:
- Horizontal ordering
Arrange the elements of creating a Promise:
- Horizontal ordering
Arrange the elements of handling a Promise with then:
- Vertical ordering
Try-catch