modelsandbox A WORK IN CURIOSITY
← All experiments
website generationPROTOCOL 1.0.0

Make me a website: effort comparison.

GPT-6.1 Sol and Claude Opus 5.5 at Low, Medium, High, and Extra High. One fresh static website per setting.

8 recorded runs2 models24 screenshots44m 2.4s recorded agent time

01 / THE STARTING POINT

Here's what
we asked.

The original prompt is part of the evidence. Delivery rules and limitations are recorded with the method.

USER PROMPT
Machine,

Pls make website,

all responsive like,
w/ BIG pictures ooo,
use my fav fonts,
also fancy menus with whooosh on,
load fast pls

Thanks,
Human

PS no bugs :)

02 / THE NUMBERS

How long? How much?

INDIVIDUAL RUNS · NO AVERAGING
ClaudeCodex
Claude Opus 5.5Low
53.5s
Claude Opus 5.5Medium
2m 34.5s
Claude Opus 5.5High
4m 16.9s
Claude Opus 5.5Extra high
21m 38.7s
GPT-6.1 SolLow
48.7s
GPT-6.1 SolMedium
1m 39.1s
GPT-6.1 SolHigh
2m 2.7s
GPT-6.1 SolExtra high
10m 8.4s

Completed T3 provider-turn wall time; excludes parent orchestration gaps and post-run browser verification.

READ WITH CONTEXT One sample per setting; uncontrolled provider load and cache state. Different scopes chosen by the models; no matched feature budget.

View the full timing and usage breakdown +
Timing and usage for each recorded run
RunCompletionTool timeOther wall timeFirst toolInput tokensCached inputOutputReasoningLocal loadCode size
opus-low53.5s0.0s53.4s43.8s113,93081,3725,40031240ms10.2 KB
opus-medium2m 34.5s0.9s2m 33.5s1m 0.6s267,166221,35218,028992164ms33.2 KB
opus-high4m 16.9s2.8s4m 14.1s25.1s575,182517,91229,1914,623143ms48.6 KB
opus-xhigh21m 38.7s11.5s21m 27.2s1m 19.5s2,877,4782,693,807143,41084,513423ms119.0 KB
sol-low48.7s0.0s48.6s46.3s53,87736,6083,1780323ms10.0 KB
sol-medium1m 39.1s0.0s1m 39.1s3.5s114,870111,6166,68176183ms21.1 KB
sol-high2m 2.7s0.0s2m 2.7s4.0s117,712100,0968,178339236ms24.4 KB
sol-xhigh10m 8.4s4.5s10m 4.0s4.1s408,722387,71225,4614,174401ms64.7 KB

Input includes cached input. Output includes reported reasoning. Other wall time includes generation, reasoning, transport, and runtime overhead. Local load observations use an unthrottled browser with uncontrolled cache state; see run notes for browser changes. Missing counters remain unknown.

03 / WHAT THEY MADE

See for yourself.

Claude Opus 5.5

Low · Run 1
53.5s
COMPLETED8/8 requirement checks
Run data ↓
Settings, checks & notes +
Model
claude-opus-5-5
Runtime
T3 Code
Settings
{"effort":"low","fastMode":false}
  • ✓Rendersdeterministic
  • ✓Responsivedeterministic
  • ✓Large Picturedeterministic
  • ✓Typographydeterministic
  • ✓Menu Worksdeterministic
  • ✓Menu Motiondeterministic
  • ✓Script Checksdeterministic
  • ✓Anchor Targetsdeterministic

Recorded notes

  • Smaller one-file site with mobile circular-reveal menu; desktop has hover navigation. No synthesized audio or photo viewer.
  • No submission behavior was requested or tested beyond navigation/menu smoke checks.
  • Open-menu screenshot recaptured in local Chrome after the reveal settled; functional metrics retain the original shared-browser observations.
Download original website (.zip) ↓

Claude Opus 5.5

Medium · Run 1
2m 34.5s
COMPLETED8/8 requirement checks
Run data ↓
Settings, checks & notes +
Model
claude-opus-5-5
Runtime
T3 Code
Settings
{"effort":"medium","fastMode":false}
  • ✓Rendersdeterministic
  • ✓Responsivedeterministic
  • ✓Large Picturedeterministic
  • ✓Typographydeterministic
  • ✓Menu Worksdeterministic
  • ✓Menu Motiondeterministic
  • ✓Script Checksdeterministic
  • ✓Anchor Targetsdeterministic

Recorded notes

  • Includes desktop dropdown, animated circular menu, synthesized whoosh, gallery, and a demo contact form. Audio output was not independently listened to.
  • Desktop/phone screenshots were recaptured in local Chrome after the entrance animation settled; the heading and hero image both had opacity 1. Browser timings remain the original shared-browser measurements.
Download original website (.zip) ↓

Claude Opus 5.5

High · Run 1
4m 16.9s
COMPLETED8/8 requirement checks
Run data ↓
Settings, checks & notes +
Model
claude-opus-5-5
Runtime
T3 Code
Settings
{"effort":"high","fastMode":false}
  • ✓Rendersdeterministic
  • ✓Responsivedeterministic
  • ✓Large Picturedeterministic
  • ✓Typographydeterministic
  • ✓Menu Worksdeterministic
  • ✓Menu Motiondeterministic
  • ✓Script Checksdeterministic
  • ✓Anchor Targetsdeterministic

Recorded notes

  • Includes full-screen story panels, photo lightbox, remembered sound preference, and demo signup. Audio output was not independently listened to.
Download original website (.zip) ↓

Claude Opus 5.5

Extra high · Run 1
21m 38.7s
COMPLETED8/8 requirement checks
Run data ↓
Settings, checks & notes +
Model
claude-opus-5-5
Runtime
T3 Code
Settings
{"effort":"xhigh","fastMode":false}
  • ✓Rendersdeterministic
  • ✓Responsivedeterministic
  • ✓Large Picturedeterministic
  • ✓Typographydeterministic
  • ✓Menu Worksdeterministic
  • ✓Menu Motiondeterministic
  • ✓Script Checksdeterministic
  • ✓Anchor Targetsdeterministic

Recorded notes

  • Larger scope: photo viewer, trip cards, demo signup, desktop dropdown, and generated whoosh audio.
  • Images with no src are intentionally deferred; they are not broken-image failures. Audio output was not independently listened to.
  • Protocol deviation: this run also wrote temporary photo-research contact sheets to its Claude scratch directory outside the requested output folder. No browser was used by the contender.
Download original website (.zip) ↓

GPT-6.1 Sol

Low · Run 1
48.7s
COMPLETED8/8 requirement checks
Run data ↓
Settings, checks & notes +
Model
gpt-6.1-sol
Runtime
T3 Code
Settings
{"reasoningEffort":"low","serviceTier":"priority"}
  • ✓Rendersdeterministic
  • ✓Responsivedeterministic
  • ✓Large Picturedeterministic
  • ✓Typographydeterministic
  • ✓Menu Worksdeterministic
  • ✓Menu Motiondeterministic
  • ✓Script Checksdeterministic
  • ✓Anchor Targetsdeterministic

Recorded notes

  • Open-menu screenshot recaptured in local Chrome after the reveal settled; functional metrics retain the original shared-browser observations.
Download original website (.zip) ↓

GPT-6.1 Sol

Medium · Run 1
1m 39.1s
COMPLETED8/8 requirement checks
Run data ↓
Settings, checks & notes +
Model
gpt-6.1-sol
Runtime
T3 Code
Settings
{"reasoningEffort":"medium","serviceTier":"priority"}
  • ✓Rendersdeterministic
  • ✓Responsivedeterministic
  • ✓Large Picturedeterministic
  • ✓Typographydeterministic
  • ✓Menu Worksdeterministic
  • ✓Menu Motiondeterministic
  • ✓Script Checksdeterministic
  • ✓Anchor Targetsdeterministic

Recorded notes

  • Includes story dialogs and story filters; newsletter is a demo.
Download original website (.zip) ↓

GPT-6.1 Sol

High · Run 1
2m 2.7s
COMPLETED8/8 requirement checks
Run data ↓
Settings, checks & notes +
Model
gpt-6.1-sol
Runtime
T3 Code
Settings
{"reasoningEffort":"high","serviceTier":"priority"}
  • ✓Rendersdeterministic
  • ✓Responsivedeterministic
  • ✓Large Picturedeterministic
  • ✓Typographydeterministic
  • ✓Menu Worksdeterministic
  • ✓Menu Motiondeterministic
  • ✓Script Checksdeterministic
  • ✓Anchor Targetsdeterministic

Recorded notes

  • Includes destination filters, journey details, and an adventure planner dialog. External fonts and images need internet access.
  • Six apparent missing hash targets were clicked: all open the correct journey dialog. They are valid scripted actions.
Download original website (.zip) ↓

GPT-6.1 Sol

Extra high · Run 1
10m 8.4s
COMPLETED8/8 requirement checks
Run data ↓
Settings, checks & notes +
Model
gpt-6.1-sol
Runtime
T3 Code
Settings
{"reasoningEffort":"xhigh","serviceTier":"priority"}
  • ✓Rendersdeterministic
  • ✓Responsivedeterministic
  • ✓Large Picturedeterministic
  • ✓Typographydeterministic
  • ✓Menu Worksdeterministic
  • ✓Menu Motiondeterministic
  • ✓Script Checksdeterministic
  • ✓Anchor Targetsdeterministic

Recorded notes

  • Includes saved stories in local storage, filters, story dialogs, and locally downloaded photos.
  • Browser deviation: shared-browser screenshot capture timed out twice. Screenshots and browser measurements for this result use a separate local Chrome session; load timings are not directly comparable.
  • Verified saving a story increments the count and appears in the reading list; removing it restores the count. The coast filter shows two stories.
Download original website (.zip) ↓

Screenshots show the recorded outputs. Website previews run in isolated frames; browser-storage features may be limited. Download the original to inspect or run it yourself.

04 / THE METHOD

The fine print matters.

Experiment data ↓

Execution

Eight fresh T3 child sessions, sequential; Promptfoo grades captured browser artifacts without new inference.

Controls

  • Same static-site instructions except output directory.
  • Codex Priority tier held constant; Claude Fast explicitly off.
  • Fresh session and empty output directory for every setting.
  • Contenders could verify locally; browser use, delegation, publishing, package installs, and edits outside output directory were forbidden.
Full delivery instructions
Website generation benchmark. Work independently; do not delegate or invoke other models. You have a fresh empty output directory: {outputDirectory}
Create the website requested below. Since subject and favorite fonts are unspecified, choose them yourself without asking questions. Deliver a static site with index.html at that directory; HTML/CSS/JS and external image/font URLs are allowed. No package installs, build pipeline, servers, publishing, or edits outside that directory. Use filesystem/shell tools to write files. Make the result complete, then stop. Do not read other contenders, benchmark files, prior runs, or repository source. You may verify your own work locally but do not call preview tools or launch browsers. This harness fixes static-site delivery for all models; you are being timed from task start to final answer. In your final answer state completion and any assumptions briefly.

Exact user prompt transcribed from their image:
Machine,

Pls make website,

all responsive like,
w/ BIG pictures ooo,
use my fav fonts,
also fancy menus with whooosh on,
load fast pls

Thanks,
Human

PS no bugs :)

Limits of this experiment

  • One sample per setting; uncontrolled provider load and cache state.
  • Subject and favorite fonts unspecified; compliance smoke checks are not visual-design accuracy.
  • Different scopes chosen by the models; no matched feature budget.
  • Local browser load observations are not production mobile performance.
  • Controls were prompt instructions rather than filesystem restrictions; deviations are noted per run.
  • Generated websites may depend on live external images and fonts.
  • Raw T3 traces and machine-specific paths are not included in this dataset.

Scoring

Rubric 1.0.0. These are requirement smoke checks, not visual-design accuracy or proof of bug-free behavior.

What each check means
Renders
Body contains substantive page content; screenshots inspected separately.
Responsive
No page-wide horizontal overflow at 390, 768, and 1440 CSS pixels.
Large Picture
At least one loaded image is at least half the desktop viewport width and 240px high.
Typography
Computed fonts show a deliberate font choice; favorite-font preference is unspecified.
Menu Works
Observed menu opens and closes.
Menu Motion
Menu or descendant has nonzero transition/animation duration.
Script Checks
Inline/local JS parses and no errors were observed during tested interactions.
Anchor Targets
Nonempty internal hash links have section targets or browser-verified dialog actions.

Screenshot

Original image ↓

Website preview

ISOLATED PREVIEW