Iterators and Generators

A dinosaur genome has billions of base pairs. If the Jurassic Park lab loaded the whole sequence at once, the computers would crash before anyone read the first gene. So the scientists analyze DNA fragment by fragment, on demand. Iterators and generators in JavaScript work the same way: they let you process data element by element, lazily and economically. "It is the lab's assembly line," says Dr. Rex. "You get the next sample exactly when you need it."

The Iterable Protocol

Any object that has a [Symbol.iterator]() method is iterable and can go into a for...of loop. Arrays and strings have it from birth:

1// Arrays, strings, Map, Set - are iterable by nature
2for (const char of 'ATCG') {
3  console.log(char); // A, T, C, G
4}
5
6for (const item of [1, 2, 3]) {
7  console.log(item); // 1, 2, 3
8}

Under the hood for...of calls [Symbol.iterator]() and gets an iterator, an object with a next() method. You can call it by hand:

1const it = ['T-Rex', 'Raptor'][Symbol.iterator]();
2console.log(it.next()); // { value: 'T-Rex', done: false }
3console.log(it.next()); // { value: 'Raptor', done: false }
4console.log(it.next()); // { value: undefined, done: true }

Every call returns an object with the fields value and done. The loop ends when done is true, and the value of that last result is skipped.

Creating a Custom Iterator

A plain object is not iterable. To walk around a paddock with a loop, we add a [Symbol.iterator]() method that returns an object with next():

1const dinoEnclosure = {
2  dinosaurs: ['Rex', 'Blue', 'Delta', 'Echo'],
3
4  [Symbol.iterator]() {
5    let index = 0;
6    const dinos = this.dinosaurs;
7
8    return {
9      next() {
10        if (index < dinos.length) {
11          return { value: dinos[index++], done: false };
12        }
13        return { value: undefined, done: true };
14      }
15    };
16  }
17};
18
19// Now we can use for...of!
20for (const dino of dinoEnclosure) {
21  console.log('Dinosaur:', dino);
22}
23
24// And the spread operator
25const allDinos = [...dinoEnclosure];
26console.log(allDinos); // ['Rex', 'Blue', 'Delta', 'Echo']

The index counter lives in a closure, so every new loop starts from zero. Spread works right away, because it uses the same protocol as for...of.

for...of vs for...in

These two loops have similar names but completely different jobs. Compare them on an object and on an array:

1const park = { name: 'Jurassic', size: 'large', open: false };
2const species = ['T-Rex', 'Velociraptor', 'Triceratops'];
3
4// for...in - iterates over KEYS (of objects and arrays)
5for (const key in park) {
6  console.log(key); // 'name', 'size', 'open'
7}
8
9for (const index in species) {
10  console.log(index); // '0', '1', '2' (strings!)
11}
12
13// for...of - iterates over VALUES (of iterables)
14for (const animal of species) {
15  console.log(animal); // 'T-Rex', 'Velociraptor', 'Triceratops'
16}
17
18// for...of does NOT work on plain objects!
19// for (const val of park) {} // TypeError!
20// But it works on Object.entries():
21for (const [key, val] of Object.entries(park)) {
22  console.log(key, val);
23}

for...in returns keys as strings and also visits properties inherited from the prototype. My advice: walk arrays with for...of, and objects with for...of over Object.entries().

Generators (function*)

Writing next() by hand is tedious. A generator does the bookkeeping for you: it is a function marked with an asterisk, function*, that can pause at yield and resume later:

1function* dinoGenerator() {
2  console.log('Generating first dinosaur...');
3  yield 'T-Rex';
4
5  console.log('Generating second...');
6  yield 'Velociraptor';
7
8  console.log('Generating third...');
9  yield 'Triceratops';
10
11  console.log('Generation complete!');
12}
13
14const gen = dinoGenerator();
15
16console.log(gen.next()); // { value: 'T-Rex', done: false }
17console.log(gen.next()); // { value: 'Velociraptor', done: false }
18console.log(gen.next()); // { value: 'Triceratops', done: false }
19console.log(gen.next()); // { value: undefined, done: true }

Calling dinoGenerator() prints nothing; it only returns a generator object. The code starts at the first next() and runs to the nearest yield, so each message appears right before its result. A generator can also finish with return:

1function* dinos() {
2  yield 'T-Rex';
3  yield 'Raptor';
4  return 'done';
5}
6
7const herd = dinos();
8console.log(herd.next()); // { value: 'T-Rex', done: false }
9console.log(herd.next()); // { value: 'Raptor', done: false }
10console.log(herd.next()); // { value: 'done', done: true }
11console.log([...dinos()]); // ['T-Rex', 'Raptor'] - without 'done'

The value from return arrives together with done: true, which is why for...of and spread skip it.

Infinite Generator

A generator may never finish, because it computes the next value only on request:

1function* idGenerator(prefix) {
2  let id = 1;
3  while (true) {
4    yield prefix + '-' + String(id).padStart(3, '0');
5    id++;
6  }
7}
8
9const dinoIds = idGenerator('DINO');
10console.log(dinoIds.next().value); // 'DINO-001'
11console.log(dinoIds.next().value); // 'DINO-002'
12console.log(dinoIds.next().value); // 'DINO-003'
13// Never ends - generates on demand!

The while (true) loop does not freeze the program, because execution stops at every yield. Never unpack such a generator with spread, though: [...dinoIds] would try to collect infinitely many values.

yield* - Delegation to Another Generator

yield* passes along, one by one, every value of another generator or iterable, so small generators can be combined into larger ones:

1function* carnivores() {
2  yield 'T-Rex';
3  yield 'Velociraptor';
4}
5
6function* herbivores() {
7  yield 'Triceratops';
8  yield 'Brachiosaurus';
9}
10
11function* allDinosaurs() {
12  yield* carnivores();
13  yield* herbivores();
14}
15
16for (const dino of allDinosaurs()) {
17  console.log(dino);
18}
19// T-Rex, Velociraptor, Triceratops, Brachiosaurus

Since yield* also works on arrays, you can shorten the paddock's custom iterator to one line: *[Symbol.iterator]() { yield* this.dinosaurs; }. That is how I recommend writing it.

Async Generators

Data from a server arrives in pages, and each page needs await. An async generator combines async with function*, and its results are received by a for await...of loop:

1async function* fetchDinoPages(apiUrl) {
2  let page = 1;
3  let hasMore = true;
4
5  while (hasMore) {
6    const response = await fetch(apiUrl + '?page=' + page);
7    const data = await response.json();
8
9    yield data.results;
10
11    hasMore = data.hasNextPage;
12    page++;
13  }
14}
15
16// Usage with for await...of
17async function loadAllDinos() {
18  for await (const batch of fetchDinoPages('/api/dinosaurs')) {
19    console.log('Loaded batch:', batch.length, 'dinosaurs');
20  }
21}

The next page loads only when the loop asks for the next batch. If nobody processes the results, no extra request goes out to the server.

Practical Applications

The first classic is lazy evaluation: the Fibonacci sequence is infinite, and we take only as much of it as we need:

1// 1. Lazy evaluation - process on demand
2function* fibonacci() {
3  let a = 0, b = 1;
4  while (true) {
5    yield a;
6    [a, b] = [b, a + b];
7  }
8}
9
10// Take only the first 10
11const fib = fibonacci();
12const first10 = [];
13for (let i = 0; i < 10; i++) {
14  first10.push(fib.next().value);
15}
16console.log(first10); // [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]

The line [a, b] = [b, a + b] is the destructuring swap you know from module 2. The second classic is pagination, serving data in portions:

1// 2. Pagination - load pages on demand
2function* paginate(items, pageSize) {
3  for (let i = 0; i < items.length; i += pageSize) {
4    yield items.slice(i, i + pageSize);
5  }
6}
7
8const allSpecies = ['A','B','C','D','E','F','G','H'];
9const pages = paginate(allSpecies, 3);
10console.log(pages.next().value); // ['A', 'B', 'C']
11console.log(pages.next().value); // ['D', 'E', 'F']
12console.log(pages.next().value); // ['G', 'H']

The last page has only two elements, because slice stops at the end of the array. The same pattern, a loop with a step and yield, is all you need to write your own range(start, end, step).

Practice

The sandbox contains a custom paddock iterator and generators to practice with. Call next() by hand and predict when done: true will appear.

1// === ITERATORS AND GENERATORS ===
2
3// 1. Custom iterator
4const enclosure = {
5  dinosaurs: ['Rex', 'Blue', 'Delta', 'Echo'],
6  [Symbol.iterator]() {
7    let i = 0;
8    const dinos = this.dinosaurs;
9    return {
10      next() {
11        return i < dinos.length
12          ? { value: dinos[i++], done: false }
13          : { done: true };
14      }
15    };
16  }
17};
18
19console.log('=== Iterator ===');
20for (const dino of enclosure) {
21  console.log('Dino:', dino);
22}
23console.log('Spread:', [...enclosure]);
24
25// 2. for...of vs for...in
26console.log('\n=== for...of vs for...in ===');
27const species = ['T-Rex', 'Raptor', 'Triceratops'];
28console.log('for...in (keys):');
29for (const i in species) console.log(' ', i, typeof i);
30console.log('for...of (values):');
31for (const s of species) console.log(' ', s);
32
33// 3. Generator
34function* dinoGen() {
35  yield 'T-Rex';
36  yield 'Velociraptor';
37  yield 'Triceratops';
38}
39console.log('\n=== Generator ===');
40const gen = dinoGen();
41console.log(gen.next()); // { value: 'T-Rex', done: false }
42console.log(gen.next()); // { value: 'Velociraptor', done: false }
43console.log(gen.next()); // { value: 'Triceratops', done: false }
44console.log(gen.next()); // { done: true }
45
46// 4. Infinite generator
47function* idGen(prefix) {
48  let id = 1;
49  while (true) {
50    yield prefix + '-' + String(id++).padStart(3, '0');
51  }
52}
53console.log('\n=== ID Generator ===');
54const ids = idGen('DINO');
55console.log(ids.next().value); // DINO-001
56console.log(ids.next().value); // DINO-002
57console.log(ids.next().value); // DINO-003
58
59// 5. yield* - delegation
60function* carnivores() { yield 'T-Rex'; yield 'Raptor'; }
61function* herbivores() { yield 'Triceratops'; yield 'Brachio'; }
62function* allDinos() { yield* carnivores(); yield* herbivores(); }
63console.log('\n=== yield* ===');
64console.log([...allDinos()]);
65
66// 6. Pagination with a generator
67function* paginate(items, size) {
68  for (let i = 0; i < items.length; i += size) {
69    yield items.slice(i, i + size);
70  }
71}
72console.log('\n=== Pagination ===');
73const pages = paginate(['A','B','C','D','E','F','G'], 3);
74console.log('Page 1:', pages.next().value);
75console.log('Page 2:', pages.next().value);
76console.log('Page 3:', pages.next().value);

Remember: a generator is the lab's conveyor belt, which delivers the next sample only when you reach out for it.