Benchmarks
SKY-001 vs GPT-6 Astra
On Skyfall’s own 3D CAD test, SKY-001 matched GPT-6 Astra’s accuracy (79.7% vs 79.1% IoU) in 3.9 s instead of 199 s, at $0.005 instead of $0.75 per sample. No one outside Skyfall has reproduced these numbers yet. [skyfall.ai]
Last checked 2026-10-21All figures: Skyfall’s benchmarkIoU = shape overlap with the reference
3D CAD parts
| Model | IoU ↑ | Time / sample ↓ | Cost / sample ↓ |
|---|---|---|---|
| SKY-001 | 79.7% | 3.9 s | $0.005 |
| GPT-6 Astra | 79.1% | 199 s | $0.75 |
| Claude Opus 5.5 | 69.4% | 110 s | $0.31 |
| Gemini 3.8 Flash | 54.7% | 112 s | $0.08 |
| Qwen 3.8 Max | 34.4% | 764 s | $0.3 |
| Kimi K3 | 9.5% | 439 s | $0.13 |
| Grok 4.7 | — | 1,250 s | $0.59 |
Grok 4.7 completed none of the samples, per the official chart.
BIM floor plans
| Model | IoU ↑ | Time / plan ↓ | Cost / plan ↓ |
|---|---|---|---|
| SKY-001 | 43.6% | 1.22 s | $0.00237 |
| GPT-6 Astra | 40.6% | 144.4 s | $0.45 |
| Claude Opus 5.5 | 34.4% | 113.1 s | $0.28 |
| Gemini 3.8 Flash | 42.3% | 345.4 s | $0.22 |
| Qwen 3.8 Max | 28.8% | 254.5 s | $0.11 |
| Kimi K3 | 33.7% | 114 s | $0.27 |
| Grok 4.7 | 28.2% | 592.3 s | $0.3 |
The official page prints the BIM times in two places that do not quite agree: the data table gives GPT-6 Astra 144.4 s and Gemini 3.8 Flash 345.4 s, the chart labels 144.6 s and 346.2 s (other rows differ by up to 0.8 s). We use the data table, which the headline 118× is computed from.
In the BIM run, Gemini 3.8 Flash (42.3%) scores higher IoU than GPT-6 Astra (40.6%); the “LLMs best” figure on the page refers to Gemini, not GPT-6 Astra.
Calculator
Time and cost at your volume
Scales Skyfall’s per-sample benchmark figures to a batch. These are the costs of Skyfall’s test runs, not a price: SKY-001 has no published pricing.
| Total time | Total cost | IoU | |
|---|---|---|---|
| SKY-001 | |||
faster · cheaper, by Skyfall’s numbers. Not independently verified.
Do the headline claims add up?
51× faster than GPT-6 Astra on 3D CAD
Company-reported
Matches the official table: 199 s ÷ 3.9 s = 51.0. [skyfall.ai]
150× lower cost than GPT-6 Astra on 3D CAD
Company-reported
Matches the official table: $0.75 ÷ $0.005 = 150. [skyfall.ai]
118× faster and 188× lower cost on BIM
Company-reported
Matches the official table: 144.4 s ÷ 1.22 s = 118.4 and $0.4465 ÷ $0.00237 = 188.4. [skyfall.ai]
Higher accuracy than every LLM tested
Company-reported
True in both official tables, but the margins are small: 79.7% vs 79.1% IoU on CAD, 43.6% vs 42.3% on BIM. Sample sizes and test sets are not published. [skyfall.ai]
“100 times the speed and 1/150th the cost of LLMs”
Company-reported
A rounded summary on the official page. The press coverage cites a 51–125× speed range and 150–210× cost range; the official tables show 51–118× and 150–188× against GPT-6 Astra. [skyfall.ai, Doomers]
Benchmarks reproduced by a third party
Not announced
No independent evaluation has been published as of the date checked. [Startup Fortune]
What the benchmark does not tell you
- How many samples each test used, or how they were chosen. Neither is published.
- How the LLMs were prompted and allowed to call CAD tools. A different harness could move their scores.
- Whether the per-sample cost includes the same things for every model. LLM costs are API prices; SKY-001 has no public price.
- Anything about tasks outside CAD parts and BIM plans. Skyfall’s other domains (PCBs, aerospace, proteins) are stated plans, not results.
These gaps are why every row carries the Company-reported stamp. We will move a claim to Independently verified when an independent evaluation is published.
Get the next update
One short email when something on this tracker actually changes: API, pricing, launch date or an independent benchmark. This is our own update list, not Skyfall’s waitlist — for that, use skyfall.ai.
Sources
- official Skyfall AI, SKY-001 product page, 2026-10-19.
- news Startup Fortune, Skyfall AI unveils SKY-001, a world model built only for engineering, 2026-10-20.
- launch agency Doomers (the agency that ran the launch), Skyfall AI SKY-001 launch breakdown, 2026-10-20.