NestJS course ยท Module 8: Caching and Performance
Performance Optimization - increasing fort speed
In this lesson6
In the evening, when the cohorts file their reports, the fort slows down: a response that takes 80 ms in the morning drags on for seconds after dark. Consul Caesar.js wants it faster, but nobody knows what slows it down - the database, the CPU or memory. So the first rule of a legion engineer is: measure first, optimize second.
What is performance optimization?
Imagine the application as a legion camp:
- the legionaries' strength (CPU) - must be enough for every task,
- the storehouses (memory) - must be well managed,
- the tabularium (database) - must be in order,
- the cohorts (processes) - must cooperate smoothly,
- roads and messengers (I/O operations) - must be passable.
Optimization is the art of tuning all these elements in harmony.
Four signals
Before you change anything, decide what you measure. Google's SRE book lists four golden signals, in this order:
- latency - response time, given as the p95 and p99 percentiles,
- traffic - throughput, for example requests per second,
- errors - the share of failed requests,
- saturation - resource usage, that is CPU and memory.
A p95 of 200 ms means that 95% of requests finish faster than 200 ms. An average hides the unlucky ones in the tail: a user waiting 10 seconds disappears in it among a thousand fast responses.
Four load trials
You read the signals during load tests, from the lightest to the heaviest:
- smoke test - minimal load, checks that the system works at all,
- load test - the expected, typical traffic,
- stress test - load above the expected level, looking for the breaking point,
- spike test - a sudden, short and huge jump in traffic.
That is how the k6 tool documentation divides them; you will meet the actual request firing in the profiling lesson.
Profiling - diagnosing problems
Inside the application time is measured by the Node.js perf_hooks module: performance.mark() puts a mark on the timeline, performance.measure() computes the gap between two marks, and PerformanceObserver receives every new measurement. We build a profiling service on top of it:
// performance-profiler.service.ts
import { Injectable } from '@nestjs/common';
import { performance, PerformanceObserver } from 'node:perf_hooks';
@Injectable()
export class PerformanceProfilerService {
private measurements = new Map<string, number[]>();
private observer: PerformanceObserver;
constructor() {
this.setupPerformanceObserver();
}
private setupPerformanceObserver(): void {
this.observer = new PerformanceObserver((list) => {
list.getEntries().forEach((entry) => {
if (entry.entryType === 'measure') {
this.recordMeasurement(entry.name, entry.duration);
}
});
});
this.observer.observe({ entryTypes: ['measure'] });
}
private recordMeasurement(name: string, duration: number): void {
if (!this.measurements.has(name)) {
this.measurements.set(name, []);
}
const measurements = this.measurements.get(name)!;
measurements.push(duration);
// Keep only the last 100 measurements
if (measurements.length > 100) {
measurements.shift();
}
// Log slow operations
if (duration > 1000) {
console.warn(`Slow operation: ${name} - ${duration.toFixed(2)}ms`);
}
}The observer passes measurements to recordMeasurement(), which keeps the last 100 results of each operation and warns about those longer than a second.
The most convenient way to measure is a decorator - a static method that wraps the original method with a measurement:
// Decorator measuring execution time
static Measure(operationName?: string) {
return function (target: any, propertyName: string, descriptor: PropertyDescriptor) {
const originalMethod = descriptor.value;
const measureName = operationName || `${target.constructor.name}.${propertyName}`;
let callId = 0;
descriptor.value = async function (...args: any[]) {
const id = ++callId; // separate marks for concurrent calls
const startMark = `${measureName}-start-${id}`;
const endMark = `${measureName}-end-${id}`;
performance.mark(startMark);
try {
const result = await originalMethod.apply(this, args);
performance.mark(endMark);
performance.measure(measureName, startMark, endMark);
return result;
} catch (error) {
performance.mark(endMark);
performance.measure(`${measureName}-error`, startMark, endMark);
throw error;
} finally {
// Marks stay in the global timeline until we remove them
performance.clearMarks(startMark);
performance.clearMarks(endMark);
performance.clearMeasures(measureName);
performance.clearMeasures(`${measureName}-error`);
}
};
};
}The first version built mark names from Date.now(), so two calls in the same millisecond overwrote each other's marks; the callId counter fixes that. The finally block cleans up, because marks without clearMarks() would pile up in a server running for weeks like a memory leak. The observer has already received the measurement, so nothing is lost.
A manual measurement is handy when you time only part of a method:
startMeasurement(name: string): void {
performance.mark(`${name}-start`);
}
endMeasurement(name: string): number {
const endMark = `${name}-end`;
performance.mark(endMark);
// measure() returns the fresh measurement - no searching the history
const measurement = performance.measure(name, `${name}-start`, endMark);
performance.clearMarks(`${name}-start`);
performance.clearMarks(endMark);
performance.clearMeasures(name);
return measurement.duration;
}Before, the service looked the measurement up with getEntriesByName(name)[0], which returns the oldest measurement with that name - from the second call on the result was the same every time.
From the collected numbers we compute statistics:
getStatistics(operationName: string) {
const measurements = this.measurements.get(operationName) || [];
if (measurements.length === 0) return null;
const sorted = [...measurements].sort((a, b) => a - b);
const avg = measurements.reduce((a, b) => a + b, 0) / measurements.length;
return {
count: measurements.length,
average: avg.toFixed(2),
median: sorted[Math.floor(sorted.length / 2)].toFixed(2),
min: sorted[0].toFixed(2),
max: sorted[sorted.length - 1].toFixed(2),
p95: sorted[Math.ceil(sorted.length * 0.95) - 1].toFixed(2),
};
}
getAllStatistics() {
const stats = {};
this.measurements.forEach((_, name) => {
stats[name] = this.getStatistics(name);
});
return stats;
}
}We compute p95 with the nearest-rank method: with 100 measurements it is the 95th result after sorting.
In a service the decorator sits above a method, and the manual measurement surrounds the chosen fragment:
// Usage in a service
@Injectable()
export class TributeService {
constructor(private profiler: PerformanceProfilerService) {}
@PerformanceProfilerService.Measure('tribute-search')
async findTributes(criteria: any) {
// Complex tribute search
await this.complexDatabaseQuery(criteria);
return [];
}
async manualMeasurement() {
this.profiler.startMeasurement('manual-operation');
// Some operation
await this.doSomething();
const duration = this.profiler.endMeasurement('manual-operation');
console.log(`Operation took: ${duration}ms`);
}
private async complexDatabaseQuery(criteria: any) {
await new Promise((r) => setTimeout(r, 20)); // query stub
}
private async doSomething() {
await new Promise((r) => setTimeout(r, 30)); // work stub
}
}The decorator does not change the body of findTributes - the method still only searches, and the measurement happens alongside.
Database Query Optimization
The most common culprit is the database. The classic N+1 problem looks innocent:
// query-optimizer.service.ts
@Injectable()
export class QueryOptimizerService {
constructor(
@InjectRepository(Tribute) private tributeRepo: Repository<Tribute>,
) {}
// Instead of N+1 queries
async getBadTributesWithOwners(): Promise<any[]> {
const tributes = await this.tributeRepo.find();
// This generates N extra queries!
const tributesWithOwners = [];
for (const tribute of tributes) {
const owner = await this.getOwner(tribute.ownerId);
tributesWithOwners.push({ ...tribute, owner });
}
return tributesWithOwners;
}One query for the list and one more for every owner: in a test on PostgreSQL, 120 tributes cost 121 queries.
The solution is a JOIN, which TypeORM builds from the relations option:
// Optimal version with a JOIN
async getOptimizedTributesWithOwners(): Promise<any[]> {
// One query with a JOIN
return await this.tributeRepo.find({
relations: { owner: true },
select: {
id: true,
name: true,
value: true,
owner: {
id: true,
name: true,
rank: true,
},
},
});
}The same result came back in one query, and select fetched only the columns we need. TypeORM 1.x removed the array syntax relations: ['owner'] - the object form is now required.
We process large sets in batches instead of loading everything into memory:
// Batch processing for large sets
async processTributesInBatches(batchSize: number = 100): Promise<void> {
let offset = 0;
let batch;
do {
batch = await this.tributeRepo.find({
skip: offset,
take: batchSize,
order: { id: 'ASC' },
});
if (batch.length > 0) {
await this.processTributeBatch(batch);
offset += batchSize;
// Briefly hand the event loop over to other requests
await this.delay(10);
}
} while (batch.length === batchSize);
}The pause hands the event loop over to other requests; the first version claimed it "gives the garbage collector time", but GC runs whenever it decides to. skip grows with every batch, and the database still walks through the skipped rows, so with millions of records a cursor is better:
// Cursor pagination for very large sets
async getTributesWithCursor(cursor?: number, limit: number = 50) {
const qb = this.tributeRepo.createQueryBuilder('tribute');
if (cursor) {
qb.where('tribute.id > :cursor', { cursor });
}
const tributes = await qb
.orderBy('tribute.id', 'ASC')
.limit(limit + 1) // +1 to check whether there are more
.getMany();
const hasNext = tributes.length > limit;
const items = hasNext ? tributes.slice(0, -1) : tributes;
const nextCursor = hasNext ? tributes[tributes.length - 2].id : null;
return {
items,
nextCursor,
hasNext,
};
}The cursor is the id of the last returned row, and WHERE id > :cursor uses the primary key index, so the hundredth page is as fast as the first. One extra row tells you whether a next page exists, without a separate COUNT.
Filters also decide the speed:
// Optimization with indexes
async searchTributesOptimized(filters: any) {
const qb = this.tributeRepo.createQueryBuilder('tribute');
// Use indexed columns in WHERE
if (filters.type) {
qb.andWhere('tribute.type = :type', { type: filters.type });
}
if (filters.minValue) {
qb.andWhere('tribute.value >= :minValue', { minValue: filters.minValue });
}
// Use the index on created_at
if (filters.dateRange) {
qb.andWhere('tribute.createdAt BETWEEN :start AND :end', {
start: filters.dateRange.start,
end: filters.dateRange.end,
});
}
return await qb
.orderBy('tribute.value', 'DESC') // Index on value
.limit(filters.limit || 100)
.getMany();
}
private async delay(ms: number): Promise<void> {
return new Promise(resolve => setTimeout(resolve, ms));
}
private async processTributeBatch(tributes: Tribute[]): Promise<void> {
// Processing a batch
console.log(`Processing ${tributes.length} tributes...`);
}
}Conditions on indexed columns let the database jump straight to the right rows. limit() is safe here because the query has no JOINs; with them use take().
Layers of speed
The fastest request is the one that never reaches the fort. From the layer closest to the user:
- browser cache - controlled by
Cache-Controlheaders, - CDN, that is edge cache - copies of content on servers close to users, which shortens the route and the latency,
- application cache - Redis from the previous lessons,
- query cache on the database side.
I recommend one order of work: measure, fix the biggest problem, measure again - without the second measurement you do not know whether you helped. In the next lessons we will deal with indexes, spreading traffic and memory.
Remember: a fast cohort does not run faster, it simply carries no dead weight - weigh it first, then drop it.
Code for this lesson: src/performance-optimization.ts
// Performance Optimization - Increasing the Cohort's Speed
import { Injectable, NestInterceptor, ExecutionContext, CallHandler } from '@nestjs/common';
import { Observable } from 'rxjs';
import { tap } from 'rxjs/operators';
// 1. Interceptor measuring response time
@Injectable()
export class PerformanceInterceptor implements NestInterceptor {
intercept(context: ExecutionContext, next: CallHandler): Observable<any> {
const start = Date.now();
const req = context.switchToHttp().getRequest();
return next.handle().pipe(
tap(() => {
const duration = Date.now() - start;
const method = req.method;
const url = req.url;
console.log(`[${method}] ${url} - ${duration}ms`);
if (duration > 1000) {
console.warn(`SLOW REQUEST: ${url} took ${duration}ms`);
}
}),
);
}
}
// 2. Lazy loading of modules
// @Module({
// imports: [
// LazyModuleLoader, // NestJS lazy loading
// ],
// })
// class AppModule {}
//
// // In the service:
// const { ProvinceModule } = await import('./province.module');
// const moduleRef = await this.lazyModuleLoader.load(
// () => ProvinceModule
// );
// const service = moduleRef.get(ProvinceService);
// 3. Pagination - don't load everything at once
@Injectable()
class PaginationService {
async findPaginated<T>(
query: any,
page: number = 1,
limit: number = 20
): Promise<{
data: T[];
total: number;
page: number;
lastPage: number;
}> {
const skip = (page - 1) * limit;
// const [data, total] = await repo.findAndCount({
// skip, take: limit,
// order: { createdAt: 'DESC' },
// });
const total = 100; // simulation
const data = [] as T[];
return {
data,
total,
page,
lastPage: Math.ceil(total / limit),
};
}
}
// 4. Connection pooling
const databaseConfig = {
type: 'postgres',
host: 'localhost',
port: 5432,
// Pool configuration
extra: {
max: 20, // Max connections in the pool
min: 5, // Min connections
idle: 10000, // Idle timeout (ms)
acquire: 30000, // Connection acquisition timeout
},
};
// 5. Response compression
// import * as compression from 'compression';
// app.use(compression());
// Reduces response size by 60-80%!
Docs entries used in this lesson
Spotted a mistake in this lesson?
Check yourself
Answer the questions from this lesson. Pick an answer to see right away whether it is correct.
1. Stress testing in the context of performance means:
2. Edge caching in CDN means:
Hands-on tasks in the game
- Vertical ordering
Arrange load testing stages from lightest:
- Click in order
Arrange the key performance metrics in the order of Google SRE's four golden signals: