Vibe coding with Claude Code course · Module 14: Pinterest clone
A Spyglass for Treasure: Visual Search and a Shop Shelf
In this lesson10
Polly the Parrot has a photo of a sunset over the harbour on her phone and wants to find something in the same mood in Treasure Map. Nobody will type "that shade of orange" into a search box. What is needed is a spyglass: you drop in a photo and get similar pins. Some of the pins are also things to buy, for example a chest or a map, so under the results a shelf with a price and a link to the shop will come in handy. Visual search can be built in several ways, though, and they differ in cost and in where the photo ends up. That is why, before a single line of code exists, you send out a scout.
What you will learn
- send a subagent to scout and get back a decision note in which uncertain numbers are marked;
- build the simplest approach locally, with a clear place for a service in the future;
- check an uploaded image on the server by its first bytes and its size;
- show the price in the product's own currency and a safe link to the shop.
Scouting before the voyage
You know the research subagent from lesson 5_4: it works in its own context and comes back with only a report, so your conversation does not drown in notes. This time the question is about design:
Do not write code. Use a subagent to compare three ways of searching
for similar photos in a Next.js app: (1) an external image analysis
service, (2) embeddings and a vector database, (3) matching by the
dominant colour and the tags of the pins. For each: what leaves the
app, the cost, the drawbacks. Every number with a link to the vendor's
page, and mark whatever you cannot confirm as "to be checked". Save the
result as a table in docs/visual-search.md.An embedding is a list of numbers into which a model turns an image or a text. Similar images get similar lists, and a vector database quickly finds the lists closest to a given one. Open docs/visual-search.md in Cursor. The two most important columns of the table looked like this for us:
| Approach | What leaves the app | Cost |
|---|---|---|
| 1. Image analysis service | the whole photo goes to the vendor | price per 1000 requests: to be checked |
| 2. Embeddings and a vector database | the photo or its vector, depending on the model | to be checked |
| 3. Colour and tags | nothing | free |You read the note like the report in lesson 5_4, in three steps. First you check that every number has a link to the vendor's page, not to a blog. Then that the uncertain ones say "to be checked". Last, what follows from it for you. A typical agent mistake is a price list and limits stated in a confident tone without a source, for example an exact sum per thousand requests. You strike such a number out or ask for a link. The decision for today is approach 3: zero cost and no data thrown overboard. It has a known flaw, though: it catches the mood, not the object, so it will treat two completely different blue photos as similar.
Request for Claude Code
Request for Claude Code:
/plan Build approach 3 from docs/visual-search.md. A /search page with
a CameraSearch component: a file picker and drag and drop, a preview
of the photo. A Route Handler POST /api/visual-search: only JPEG, PNG
and WebP up to 5 MB, the type checked by the first bytes, like the MP4
in lesson 5_4. The dominant colour is computed by the sharp library in
lib/imageFeatures.ts; mark there the place for an external service.
Do not save or log the photo. Result: the pins with the closest colour
and their most frequent tags, and above them a shelf of products with
the price in the product's currency. Tests for lib/visualSearch.ts and
lib/imageCheck.ts first. Finish with tests, lint, npx tsc --noEmit
and npm run build.sharp is a library for processing images in Node.js. Claude will add it to the project with the command npm install sharp.
What Claude wrote: the check on the server
The accept attribute on a file field is only a hint for the picker window, as you know from lesson 5_4. The real check stands in lib/imageCheck.ts:
// lib/imageCheck.ts
export function detectImageKind(head: Uint8Array): ImageKind | null {
const ascii = (from: number, to: number) => String.fromCharCode(...head.slice(from, to));
if (head[0] === 0xff && head[1] === 0xd8 && head[2] === 0xff) return "jpeg";
if (ascii(1, 4) === "PNG" && head[0] === 0x89) return "png";
if (ascii(0, 4) === "RIFF" && ascii(8, 12) === "WEBP") return "webp";
return null;
}Each of the three formats begins with a fixed signature. JPEG has the bytes FF D8 FF, PNG the byte 0x89 and the text PNG, and WebP the text RIFF and, four bytes further, WEBP. The function looks only at the first twelve bytes and does not check whether the image can be opened. It checks only that the file is not pretending to be an image. The route app/api/visual-search/route.ts puts the checks in order:
// app/api/visual-search/route.ts
if (file.size > MAX_IMAGE_BYTES) {
return Response.json({ error: "The photo can be 5 MB at most." }, { status: 413 });
}
const bytes = Buffer.from(await file.arrayBuffer());
if (!detectImageKind(bytes.subarray(0, 12))) {
return Response.json({ error: "We only accept JPEG, PNG and WebP." }, { status: 415 });
}The size goes first, because file.size is ready at once, while arrayBuffer() makes a copy of the whole file. request.formData() has already read the whole request, though, so for big files the Content-Length header check from lesson 5_4 comes in handy. Higher up stand request.formData().catch(() => null) and the instanceof File check from lesson 5_4, and a missing file gives 400. Five megabytes is a rule of our clone, not a standard. For us a text file sent as cat.jpg with the type image/jpeg got 415, a 6 MB file got 413, and a photo of a bay got a 200 response.
What Claude wrote: colour, tags and the seam
Open lib/imageFeatures.ts:
// lib/imageFeatures.ts
// SEAM: this is where we will plug in an external image analysis service.
// Before the photo leaves the app, the user must agree to it (see docs/visual-search.md).
export async function extractFeatures(bytes: Buffer): Promise<ImageFeatures> {
const { dominant } = await sharp(bytes).stats();
const color = "#" + [dominant.r, dominant.g, dominant.b].map((v) => v.toString(16).padStart(2, "0")).join("");
return { color };
}A seam is one place where you will one day swap the implementation and the rest of the app will not notice. The route knows only extractFeatures and its result ImageFeatures. stats() from sharp returns statistics of the image, among them the dominant colour as three numbers from 0 to 255, which the function turns into the #rrggbb notation. Next, lib/visualSearch.ts:
// lib/visualSearch.ts
export function findSimilarPins(color: string, pins: readonly Pin[], limit = 12): Pin[] {
return [...pins]
.sort((a, b) => colorDistance(color, a.dominantColor) - colorDistance(color, b.dominantColor))
.slice(0, limit);
}colorDistance counts the distance between colours like the distance between points in a red, green and blue space. It is a simple measure, not the human eye, but it is enough for a mood. [...pins] copies the list before sorting, as in lesson 12_1. Next to it stands relatedTags, which counts the tags of the found pins and returns the five most frequent. The name is honest: these are keywords related to the results, not things recognised in the photo. For us the sunset photo gave the colour #f8a858, "Sunset over the Harbour" in first place, and among the keywords "harbour", "sunset" and "sky".
Privacy: the photo stays in port
A phone photo is often personal data: faces, the inside of a flat, a document on a table. Our clone has three simple rules. The photo lives only in the memory of one request, so we do not write it to disk. It does not get into the logs. It does not leave the app. When you one day plug in the service from approach 1 or 2, the seam requires the user's consent before the photo is sent. The text under the drop zone says it plainly: "The photo is not saved or sent anywhere else".
What Claude wrote: the drop zone
The old code promised "drag and drop" but handled only the file field. Open components/CameraSearch.tsx:
// components/CameraSearch.tsx
onDragOver={(e) => {
e.preventDefault();
setDragging(true);
}}
onDrop={(e) => {
e.preventDefault();
setDragging(false);
const dropped = e.dataTransfer.files[0];
if (dropped) search(dropped);
}}MDN explains both preventDefault calls. The drop event arrives only if the element cancelled dragover. Without the cancel, the browser may itself open or download the dropped file instead. dataTransfer.files is the list of dropped files. search sends the file in FormData, the same as the file picker does. The preview is created from URL.createObjectURL in an effect with cleanup, following the pattern from lesson 9_3. For us in Chrome a dropped PNG file gave the same results as one chosen from the window, and a text file showed the message "We only accept JPEG, PNG and WebP."
What Claude wrote: the shop shelf
A pin that can be bought has an extra field product?: Product. The question mark marks an optional field, and the product itself is described by a type:
// lib/types.ts
export interface Product {
price: number;
currency: string; // ISO 4217 code, e.g. "PLN", "EUR"
inStock: boolean;
rating: number; // 0-5
merchant: string;
url: string;
}The old code wrote the price as $ and a number, so a chest for 149 zloty pretended to be goods in dollars. Claude's version, lib/price.ts:
// lib/price.ts
export function formatPrice(product: Pick<Product, "price" | "currency">, locale = "en-US"): string {
return new Intl.NumberFormat(locale, { style: "currency", currency: product.currency }).format(product.price);
}According to MDN, with style: "currency" the currency field is required and has no default value. It accepts ISO 4217 codes such as "USD" or "EUR". For us the output was "PLN 149.00", "€24.50", "$39.99" and "PLN 1,299.00": with the en-US locale thousands are grouped with a comma, and a currency without its own symbol in that locale appears as a code. The rating is product.rating.toFixed(1), a string with one digit after the point, for example "4.7". The link to the shop:
// components/ProductShelf.tsx
<a href={product.url} target="_blank" rel="noopener noreferrer">
To the shop
</a>MDN describes noopener like this: the new tab does not get window.opener, so a foreign page cannot steer ours. noreferrer additionally does not send the shop a Referer header with the address of the page the learner came from. When inStock is false, in place of the link stands the text "Out of stock".
Captain's review
- The check on the server. You send a fake image without a form:
curl -F 'image=@note.txt;type=image/jpeg' localhost:3000/api/visual-searchshould return an error with the code 415. - Nothing overboard. Search
lib/andapp/api/for the wordsfetch(,writeFileandconsole.log. There is none on the path of the photo. A typical agent mistake is a ready call to a service in the code "for the future", without consent and without an entry in the note. - Currency. Search
components/for$andPLN. The price should pass only throughformatPrice. - Dropping beside the zone. Drop a photo a little way from the frame. The browser will open the file instead of the page, because only the zone handles the
dropevent. MDN advises cancelling it on the whole window too, but only for files.
Next step
When the review passes, ask for a commit "Visual search and the shop shelf". Then a small step: "A file dropped beside the zone must not open in the tab; cancel drop on window only for files". The clone already has a grid, chests and a spyglass, but it keeps everything in memory on globalThis: restart the server and the boards vanish. In the next lessons of this location Claude will save the pins and boards in a database and tidy up the API routes, and in the next location the voyage sails on. The editor below holds the spyglass in miniature: pick a colour, see the closest pins and the prices in each product's currency.
Remember: a spyglass shows only what lies under the glass, and before you send someone else's picture overboard, you ask the owner for consent.
Code for this lesson: App.tsx
// A spyglass in miniature: the closest colours, related keywords and prices in the product's currency
import { useState } from 'react';
interface Product {
price: number;
currency: string; // ISO 4217 code
inStock: boolean;
rating: number;
merchant: string;
url: string;
}
interface Pin {
id: string;
title: string;
dominantColor: string;
tags: string[];
product?: Product;
}
// The prices and shops are made up for our clone
const PINS: Pin[] = [
{ id: 'p1', title: 'Bay at Dawn', dominantColor: '#289888', tags: ['sea', 'dawn', 'bay'] },
{ id: 'p2', title: 'Sunset over the Harbour', dominantColor: '#f8a858', tags: ['sunset', 'harbour', 'sky'] },
{ id: 'p3', title: 'Ocean Depths', dominantColor: '#487898', tags: ['sea', 'depths', 'diving'] },
{ id: 'p4', title: 'Gold in the Chest', dominantColor: '#f88808', tags: ['treasure', 'gold', 'chest'], product: { price: 149, currency: 'PLN', inStock: true, rating: 4.66, merchant: 'Anchor Stall', url: 'https://example.com/skrzynia' } },
{ id: 'p5', title: 'Old Map', dominantColor: '#986848', tags: ['map', 'treasure', 'paper'], product: { price: 24.5, currency: 'EUR', inStock: true, rating: 4.2, merchant: 'Map & Co', url: 'https://example.com/mapa' } },
{ id: 'p6', title: 'Flag in the Harbour', dominantColor: '#d8a898', tags: ['harbour', 'flag', 'ship'], product: { price: 39.99, currency: 'USD', inStock: false, rating: 3.85, merchant: 'Flag Shop', url: 'https://example.com/bandera' } },
{ id: 'p7', title: 'Wooden Deck', dominantColor: '#785838', tags: ['ship', 'wood', 'deck'], product: { price: 1299, currency: 'PLN', inStock: true, rating: 4.9, merchant: 'Shipwright Jan', url: 'https://example.com/poklad' } },
{ id: 'p8', title: 'Night Watch', dominantColor: '#080858', tags: ['night', 'sky', 'ship'] },
];
function hexToRgb(hex: string): [number, number, number] {
const n = parseInt(hex.slice(1), 16);
return [(n >> 16) & 255, (n >> 8) & 255, n & 255];
}
function colorDistance(a: string, b: string) {
const [r1, g1, b1] = hexToRgb(a);
const [r2, g2, b2] = hexToRgb(b);
return Math.hypot(r1 - r2, g1 - g2, b1 - b2);
}
function findSimilarPins(color: string, pins: readonly Pin[], limit: number) {
return [...pins].sort((a, b) => colorDistance(color, a.dominantColor) - colorDistance(color, b.dominantColor)).slice(0, limit);
}
function relatedTags(pins: readonly Pin[], limit: number) {
const counts = new Map<string, number>();
for (const pin of pins) for (const tag of pin.tags) counts.set(tag, (counts.get(tag) ?? 0) + 1);
return [...counts].sort((a, b) => b[1] - a[1]).slice(0, limit).map(([tag]) => tag);
}
function formatPrice(product: Product) {
return new Intl.NumberFormat('en-US', { style: 'currency', currency: product.currency }).format(product.price);
}
const PRESETS = ['#f5a55a', '#2a9a86', '#7a5a3a', '#101060'];
export default function App() {
const [color, setColor] = useState('#f5a55a');
const [agentPrices, setAgentPrices] = useState(false);
const results = findSimilarPins(color, PINS, 4);
const tags = relatedTags(results, 5);
const products = results.filter((pin) => pin.product);
return (
<div className="min-h-screen space-y-4 bg-amber-50 p-4 text-zinc-900">
<h1 className="text-xl font-bold">A Spyglass for Treasure</h1>
<div className="flex flex-wrap items-center gap-2">
<label className="flex items-center gap-2">
Photo colour:
<input type="color" value={color} onChange={(e) => setColor(e.target.value)} />
</label>
{PRESETS.map((preset) => (
<button key={preset} type="button" onClick={() => setColor(preset)} aria-label={'Colour ' + preset} className="size-8 rounded-full border border-zinc-400" style={{ backgroundColor: preset }} />
))}
</div>
<ol className="space-y-1">
{results.map((pin) => (
<li key={pin.id} className="flex items-center gap-2">
<span className="inline-block size-5 rounded" style={{ backgroundColor: pin.dominantColor }} />
{pin.title} <span className="text-xs text-zinc-500">distance {Math.round(colorDistance(color, pin.dominantColor))}</span>
</li>
))}
</ol>
<p className="text-sm">Related keywords: {tags.join(', ')}</p>
<section aria-labelledby="shelf" className="rounded-xl bg-white p-3">
<h2 id="shelf" className="font-semibold">For sale</h2>
{products.length === 0 && <p className="text-sm text-zinc-500">There are no products among the results.</p>}
<ul className="mt-2 space-y-2">
{products.map(({ id, title, product }) =>
product ? (
<li key={id} className="flex flex-wrap items-center gap-2">
<strong>{agentPrices ? '$' + product.price : formatPrice(product)}</strong>
{title} · {product.merchant} · ★ {product.rating.toFixed(1)}
{product.inStock ? (
<a href={product.url} target="_blank" rel="noopener noreferrer" className="rounded-full bg-red-600 px-3 py-1 text-sm text-white">
To the shop
</a>
) : (
<span className="text-sm text-zinc-500">Out of stock</span>
)}
</li>
) : null,
)}
</ul>
</section>
<label className="flex items-center gap-2 text-sm">
<input type="checkbox" checked={agentPrices} onChange={(e) => setAgentPrices(e.target.checked)} />
The agent's version: a hard-coded $ sign instead of the product's currency
</label>
</div>
);
}Spotted a mistake in this lesson?
Check yourself
Answer the questions from this lesson. Pick an answer to see right away whether it is correct.
1. Which approach to visual search did the captain choose to start with, and why?
2. What should the subagent write next to a number it did not confirm on the vendor's page?
These are 2 of 9 questions for this lesson. Solve the rest in the game.
Hands-on tasks in the game
- Vertical ordering
Put in order the steps of the route POST /api/visual-search:
- Horizontal ordering
Put in order the line of the Product interface with the currency code:
- Click in order
Click the elements to build the line that reads the dropped file in onDrop:
- Code editor
Create a board page
- Vertical ordering
Put in order the work on choosing the spyglass approach:
- Click in order
Click the elements to build a product rating with one digit after the point:
- Horizontal ordering
Put in order the expression from formatPrice that shows the price in the product's currency:
- Code editor
Create a visual search interface
- Vertical ordering
Put in order the way from photo to shop in Treasure Map:
Tasks after this lesson
- QuestionQuiz: Choosing the spyglass approach
- QuestionQuiz: Uncertain numbers in the note
- Put in orderSorting: Processing the photo on the server
- QuestionQuiz: The seam for an external service
- Put in orderSorting: The currency field in Product
- Click in orderBuild: The dropped file
- Coding taskPractice: Board page
- QuestionQuiz: Related keywords
- Put in orderSorting: From scouting to a decision
- QuestionQuiz: Out of stock
- Click in orderBuild: A rating with one digit
- Put in orderSorting: A price in the product's currency
- QuestionQuiz: A safe link to the shop
- Coding taskPractice: Camera search
- QuestionQuiz: The drop zone
- Put in orderSorting: The spyglass from photo to shop
- QuestionQuiz: The privacy of uploaded photos
- QuestionQuiz: Checking the file type