Event photo galleries
All Cloudflare
Decision-model moderation
Staging live · waitlist open

candids

Shared photo albums for weddings and parties. Guests scan a QR code and upload from the phone browser with no app or account. Photos go straight to R2, and each one is checked by a decision-model moderation pipeline before anyone else can see it.

candidsapp.com Benchmark repo

'26 10 21

AUROC for TypeSafe Jev on 2,037 held-out UnsafeBench images: 0.866
Unsafe images caught at a 5% false-flag budget (Claude Haiku: 31.4%): 50.7%
Cost spread across four systems, 4.0¢ to $1.00 per 1,000 images: 25×
Median Jev score on gore positives after rewording one question: 0.13 → 0.77
Ordinary event photos hidden on the draft holdout, against a 1% budget: 0.78%
Worker and SPA tests on real D1, KV and R2 bindings: 865

01Context

Open upload links need moderation

Anyone with the QR code can post to an album that the couple, their families and the other guests are viewing. The moderation step has to keep up with a venue of phones on one network, cost a small fraction of the gallery price, and hold a photo when its scores are uncertain.

Cost

Galleries sell once for $15 to $99. Scoring 16 policy questions per photo cost $0.31 for the 2,519-image eval set, about 12¢ per 1,000 photos, so a 2,000-photo wedding costs roughly 25¢ to moderate.

Latency

Guests expect their photo to show up while they are still at the table. On the eval set, Jev answered all 16 questions with a p50 of 180 ms and a p95 of 384 ms.

Failure direction

A public media URL cannot be revoked once someone has seen it. Nothing is published before it is checked, and when both models are unavailable a photo waits in the checking state and retries.

02System

Architecture

The whole system runs on Cloudflare. A React SPA is served as Workers static assets. A Hono API Worker uses D1, R2, KV, Queues, Images, Rate Limiting and Email Sending, and a queue consumer runs moderation. Uploads move left to right across the top of the diagram. Moderation returns along the bottom and ends in one of four outcomes.

Hover or tap any component to trace its connections, or pick a flow to step through a request end to end.

03Uploads

Upload pipeline

Photo bytes go from the phone to R2 directly. The Worker issues the presigned URL and verifies the object afterwards.

01
Gate in policy order
Upload window, then the HMAC passcode cookie, then sign-in for registered galleries, then an active invite, ownership or moderator grant for invite-only galleries. Each check lives in its own policy file.
02
Work on the phone
The browser empties the EXIF GPS block and drops XMP in place, so the original stays byte-identical apart from location. It renders the 480px thumb and 1920px display copy with createImageBitmap and canvas, and rejects oversize files before any upload.
03
Presign
The Worker writes a pending-upload row, mints a time-ordered photo id and returns a 5-minute presigned PUT that signs Content-Type, Cache-Control and the declared size. The route is limited to 240 requests a minute per IP, user and event.
04
PUT straight to R2
The phone uploads directly to the bucket and retries 429s, 5xx and network errors five times with jittered exponential backoff, since every guest at a venue can share one IP.
05
Confirm
R2 HEAD and a magic-byte sniff for JPEG, PNG, WebP and GIF, plan file size, photo count, per-event storage and the workspace pool, then real dimensions from the header. Caps are re-checked against upload order so concurrent confirms cannot pass a limit. Any rejection deletes every key for the photo.
06
Moderate, then deliver
The photo is stored in the checking state and queued for moderation. Once published, public galleries serve it from the edge-cached media domain and private galleries through signed URLs that one KV write can revoke.

04Model selection

Choosing a model

Two decision models, a small general-purpose LLM and a commercial moderation API were run on all 10,146 UnsafeBench images with the same prompt, tuned for none of them. Each model's threshold was chosen on the 8,109-image train split. All results below are on the 2,037-image test split, with 95% bootstrap intervals.

Ranking quality

AUROC on the test split. 0.5 is a coin flip.

Primary model
Fallback
Benchmarked, not used

TypeSafe Jev 1.14OpenRouter Decisions API0.866

Claude Haiku 5.5Anthropic API0.859

Cloudflare clef-flashWorkers AI0.856

Amazon RekognitionDetectModerationLabels0.698

Recall at a 5% false-flag budget

Share of unsafe test images caught when only 5% of safe images may be flagged.

Primary model
Fallback
Benchmarked, not used

TypeSafe Jev 1.1450.7%

Cloudflare clef-flash46.5%

Claude Haiku 5.531.4%

Amazon Rekognition30.2%

Median latency

Per-request p50 over public APIs from one workstation, all 10,146 images. Lower is better.

Primary model
Fallback
Benchmarked, not used

Amazon Rekognition148 ms

TypeSafe Jev 1.14162 ms

Cloudflare clef-flash471 ms

Claude Haiku 5.51,234 ms

Cost per 1,000 images

Provider-reported token usage at list price. Lower is better.

Primary model
Fallback
Benchmarked, not used

Cloudflare clef-flash4.0¢

TypeSafe Jev 1.147.7¢

Claude Haiku 5.518.2¢

Amazon Rekognition$1.00

F1 at each model's train-split threshold. Own-verdict F1 uses each system's out-of-the-box answer with no tuning. Every per-image output is published in the repo, and a rescore script re-checks each number against your own copy of the dataset.
Model	AUROC	AP	F1	Own-verdict F1	ECE	p95	$ / 1k
TypeSafe Jev 1.14	0.866	0.813	0.721	0.710	0.093	313 ms	$0.077
Claude Haiku 5.5	0.859	0.774	0.736	0.691	0.096	2,021 ms	$0.182
Cloudflare clef-flash	0.856	0.802	0.715	0.615	0.110	1,010 ms	$0.040
Amazon Rekognition	0.698	0.621	0.569	0.443	0.262	299 ms	$1.000

05Domain eval

Tuning thresholds on event photos

UnsafeBench is 40% unsafe, half AI-generated, and has almost no weddings or parties. Ordinary event photos are where false positives come from, so the questions and thresholds were tuned on a separate 2,519-image eval set built for candids.

01
Collect
1,529 Wikimedia Commons images with licence and author recorded: 667 event photos, 694 hard negatives chosen from the policy’s “fine at events” lists (costumes, fake blood, pool parties, tattoos, Hindu swastikas) and 168 positives. Plus 876 UnsafeBench candidates and 114 synthetic images such as fake IDs, scam flyers and look-alike QR table tents. Split 70/30 into 1,770 tune and 749 holdout.
02
Pre-label with Claude Haiku 5.5
Haiku answers yes, no or unsure on all 16 policy categories from a prompt generated from the policy text, and never sees the source labels. 2,519 images cost $1.41 with no refusals. 24 images it flagged as a possible minor or non-consensual image were removed without anyone viewing them.
03
Prioritise human review
Disagreements with the source labels, unsure answers and non-photos put 1,716 images at the front of a local labelling app. Images stay blurred until clicked, sessions get break reminders, there is a blind mode, and Cohen’s kappa is computed per category once a second labeller joins.
04
Score in production shape
One Jev call per image with all 16 questions as yes/no and the event type in state, exactly as the moderation consumer calls it. $0.31 for the set, p50 180 ms, p95 384 ms.
05
Budget thresholds on tune, report on holdout
Greedy budget filling lowers whichever threshold catches the most new positives per extra benign photo flagged, weighted by severity, until the budgets are spent: at most 1% of ordinary photos hidden and 3% of plain event photos held for the host on the wedding preset.

Question wording moves scores

Draft 1 asked about “real gore” inside the graphic-violence question. Draft 2 asks plainly and moves the costume exception into the false criterion. A third variant asked whether most guests would find the photo disturbing. Thresholds are picked only after the questions are frozen and versioned, because wording alone moved the median score this far.

0.13Draft 1 median score on gore positives

0.77Draft 2, same images, same model

Tune split: 22 gore positives, 1,604 negatives, 125 costume and fake-blood hard negatives.
Graphic-violence wording	AUROC	Median gore score	Recall at 0.3	Costume photos flagged
Draft 1 · “real gore” in the question	0.990	0.13	27.3%	0%
Draft 2 · plain question, exception in criteria (shipped)	0.999	0.75 – 0.77	100%	0%
Variant C · “would most guests find this disturbing”	0.996	0.87	100%	48% at 0.1 (rejected)

Holdout split. v1 held 3.1% of innocent family photos for review, which is why the minor-safety threshold moved to 0.6.
Iteration	Change	Ordinary photos hidden	Host queue, wedding preset	Held for Candids (P2)
v1	Budgets only, minor-safety threshold at 0.2	1.03%	2.27%	3.10%
v2	Sexual pinned at 0.3, violence at 0.1, minor safety raised to 0.6	0.52%	–	0.26%
v3	Graphic-violence question reworded (draft 2), violence back to 0.3	0.78%	2.73%	0.26%

Draft thresholds on the 749-image holdout. Labels are Claude Haiku pre-labels with human confirmation in progress, so these numbers measure agreement with that labeller. The same scripts re-run on human-only labels, and positive counts on the holdout are small.
Category	Positives	AUROC	Threshold	Recall	False flags on ordinary photos
P1 Sexual (explicit)	23	0.977	0.30	61%	0.24%
P3 Graphic violence	6	0.997	0.30	100%	0.24%
P4 Hate and extremism	11	0.999	0.60	91%	0.00%
P5 Self-harm	4	0.998	0.30	100%	0.46%
P7 Spam and scams	34	0.984	0.40	82%	0.24%
P8 Private documents	7	1.000	0.30	100%	0.00%
P9 Hard drugs	2	1.000	0.40	100%	0.00%
G1 Revealing (host rule)	113	0.997	0.40	98%	2.24%
G3 Smoking (host rule)	33	0.998	0.50	97%	0.49%
G6 Off-topic (host rule)	388	0.993	0.60	58%	0.35%

06Policy

Moderation policy

Each photo goes through a fixed decision order. The first rung that matches decides the outcome.

01
Known-material hash match
Runs before any model. The decision model is a second layer for new material.
Block
02
A platform category in its block band
No category has a block band at launch.
Block
03
Minor safety (P2) in its hold band
Always reviewed by a person.
Hold for Candids
04
Any other platform category in its hold band
The uploader or host can request a review, with an optional note.
Hidden until review
05
Private documents (P8)
“Add it” continues down the ladder. “Remove it”, or 24 hours with no answer, deletes the photo.
Ask the uploader
06
A gallery category the host set to “Hold for me”, or manual approval on
Revealing, alcohol, smoking, off-topic and similar categories are the host’s call.
Hold for host
07
Nothing above matched
Per-category scores, outcome, model id, question version and threshold version are stored on the photo.
Publish

Base rates

Unsafe photos are a small fraction of a percent of event uploads, so even a 0.5% false-positive rate blocks more good photos than bad ones. Reaching 99.5% precision at about 0.1% prevalence needs a false-positive rate near 1 in 400,000. Three thousand clean photos can only show a rate below 1 in 1,000, by the rule of three.

Moderation therefore rolls out in shadow mode, and no category auto-blocks at launch. A category can start blocking only after about 300 reviewed decisions above a score with no overturns.

Measuring errors

Review requests catch the false positives that people notice. A weekly random audit of about 20 unrequested hidden photos estimates the false-positive rate without that selection bias. Every reviewer decision becomes a labelled example for the next threshold pass.

Thresholds live in one versioned file that names the model, the question version and the eval-set version. The fallback model has its own file.

07Decisions

Decisions

ADR 01

Upload path

Chose: Presigned PUT from the phone to R2
Over: Proxying uploads through the API Worker

The Worker presigns and then verifies with HEAD and magic bytes, so it never pays CPU or bandwidth for photo bytes.

Trade-off The bucket CORS policy must change before any header the client signs, or uploads fail.

ADR 02

Moderation model

Chose: TypeSafe Jev 1.14, with clef-flash as fallback
Over: Claude Haiku 5.5 or Amazon Rekognition

Jev tied Haiku on AUROC but caught 50.7% of unsafe images at 5% FPR against 31.4%, at 162 ms p50 against 1,234 ms and 7.7¢ against 18.2¢ per 1,000. Rekognition scored 0.698 AUROC at $1.00 per 1,000.

Trade-off A decision model returns probabilities with no explanation. Reviewers see per-category scores, not reasons.

ADR 03

Default outcome

Chose: Hold and review, with budgets per category
Over: Auto-blocking above a threshold

At event-photo base rates, blocking would remove more good photos than bad ones, and the eval set is too small to prove otherwise.

Trade-off More human review early on, and a slower path for a category to earn blocking.

ADR 04

Thresholds

Chose: Per-model, per-category files versioned with the questions
Over: One global threshold

clef-flash scores run lower than Jev’s, and rewording one question moved Jev’s median gore score from 0.13 to 0.77.

Trade-off Any change to the model or the wording means re-running the eval set before release.

ADR 05

Media URLs

Chose: HMAC URLs with a day-stable expiry and a KV access version
Over: Short-lived per-request tokens

Stable URLs keep caches warm, and revoking access is a single KV write that applies within 60 seconds.

Trade-off Public-gallery URLs on the media domain cannot be revoked, which is why nothing is published before it is checked.

ADR 06

Tests

Chose: Integration tests against Miniflare D1, KV and R2
Over: Mocking Cloudflare bindings

The Worker tests call app.request() through vitest-pool-workers with remote bindings off and pinned vars, so a local .dev.vars cannot change results.

Trade-off The Worker suite takes about 52 seconds, so the pre-push hook runs tests and the pre-commit hook does not.

08Operations

Testing and delivery

A pnpm and Turborepo monorepo with a shared types package used by both apps. All suites below passed on 2026-10-23.

Worker tests on real Miniflare D1, KV and R2: 531
React tests, querying by role and label: 334
Benchmark tests for metrics, models and image prep: 24
Hand-written additive D1 migrations: 29

01
Hooks
lefthook runs Biome with fixes re-staged, ESLint for React hooks and a Turbo typecheck on commit, and the test suites on push. Typechecking uses tsgo, the native TypeScript 7 compiler.
02
Pull requests
Biome runs in parallel with affected-only Turbo typecheck, lint, test and build, filtered against the base branch and backed by a Turbo cache.
03
Staging deploy
Every check runs again, plus config guards that fail the deploy if a dashboard-managed var appears in wrangler vars. Remote D1 migrations apply before the code, then the API Worker, then the SPA, then a Discord notice. Deploys queue instead of cancelling each other.
04
Observe and roll back
Workers Logs with head sampling on the API, a structured logger, and Sentry wrapped around fetch, queue and cron with PII off. Every cron job reports its own failure. Rollback is wrangler rollback, and D1 time travel restores up to 30 days.

09Failure modes

Failure modes

What breaks	Guard	How I'd know
Jev is unavailable	clef-flash scores the same questions through env.AI against its own thresholds.	The scoring model is recorded per photo, and a fallback share above a few percent alerts.
Both models are down	The photo stays in the checking state and retries with backoff. It is never published unchecked.	Photos checking for over an hour appear on the Trust & Safety health page.
A question is reworded	Thresholds are tied to a question version and re-picked on the eval set before release.	Question and threshold versions are stored on every scored photo.
A real guest photo is hidden	Hold and hide instead of block, with review on request.	Overturn rates per category, plus the weekly random audit.
A venue shares one IP	Rate limits keyed by IP, user and event and sized for a venue; the client backs off with jitter.	429 responses in Workers Logs.
Concurrent confirms race past a plan cap	Caps are re-checked against upload order inside confirm.	Regression tests in the Worker suite.
A migration breaks running code	Hand-written additive migrations, applied before the code deploy, guarded by a migration-safety test.	The deploy stops at the migration step.

10Stack

Stack

Frontend

React 19
Vite 6
React Router 7
TanStack Query 5
Mantine 9
GSAP
qr-code-styling

API

Hono 4 on Workers
Better Auth 1.6
Drizzle ORM
aws4fetch presigning
Smart Placement

Data

D1 · 24 tables
R2 photos
R2 exports
KV

Async

Queues
Hourly cron · 8 jobs
Images binding
Email Sending

Moderation

Hash matching
TypeSafe Jev 1.14
Cloudflare clef-flash
OpenRouter Decisions API
Workers AI
TanStack AI decide()

Evals

UnsafeBench
Claude Haiku 5.5 pre-labels
Amazon Rekognition (benchmarked)
Bootstrap CIs
hyparquet
sharp

Tooling

pnpm + Turborepo
Biome
tsgo
lefthook
Vitest 4
vitest-pool-workers
GitHub Actions
Sentry

Engineering portfolio