Claude Fable 5.2 on iac-bench 2026-10

Every model evaluated on the 2026-10 fixtures side by side, with a sortable leaderboard, picks by use case and the cost of each: LLM rankings.

Summary

Claude Fable 5.2 has the highest overall score on the 2026-10 leaderboard, and it is the only model whose 95% rank interval is a single place (1–1). All 9 modules planned, and none failed a mandatory security item. Its weakest attempt (88.6) is above the weakest attempt of every other model, and its bootstrap win probability is at least 0.98 against each of them (0.985 against GPT-6.1 Sol, the closest).

It is the only model that covers every expertise item in every attempt: 7/7 on S3, 9/9 on Cloud SQL and 5/5 on App Service. Its cost is the counterweight. It was the most expensive model on every fixture, at $21.13 for all 9 attempts, about six times Claude Opus 5.5 ($3.58) and 1.5 times Claude Fable 5.1 ($13.97).

Scores

Rubric items not at full credit, counted over attempts that planned: G001: A4 2/3, A5 3/3, B1 1/3, H12 3/3, H15 3/3; G002: A8 2/3, B1 2/3, H4 1/3, H5 1/3, H11 3/3; G003: A7 2/3. Definitions are in Scoring criteria.

Strengths

• S3 access logging that delivers. It is the only model of 16 that grants the logging service permission to write (H13, 3/3). Each module creates a companion log bucket by default, encrypted with SSE-S3, and its policy admits logging.s3.amazonaws.com for this bucket and account only (excerpt below).
• Cloud SQL data outlives its resources.
  • deletion_policy = "ABANDON" is hardcoded on every google_sql_database (H11 at 0.75, 3/3). Claude Fable 5.1 and Claude Opus 5.5 lose H11 in 3/3.
  • Backups outlive the instance (retain_backups_on_delete, E1) in 3/3, in one sample for production only. This is the one Cloud SQL expertise item Claude Opus 5.5 misses.
• Strict guards. 11–27 validation blocks per module, plus preconditions in 7/9. On App Service, one capacity object rejects min_instances below 2 when zone_redundant is set.

Weaknesses

• Interface size:
  • S3: 13–14 inputs against a budget of 8 (A5 partial, 3/3).
  • Cloud SQL: 24–27 inputs against 25 (A8 partial, 2/3).
  • App Service: 15, 22 and 37 inputs across the three samples. The 37-input attempt scores zero on A7 and is its lowest App Service attempt (92.7).
• Scope on S3. A cloudfront_distribution_arns input adds CloudFront read and decrypt grants to the bucket and key policies in 2/3 attempts (A4).
• S3 provider and tags. No input sets the region (H15, 3/3), and tags are merged into each resource, which earns half credit on H12 (3/3).
• Provider constraints that admit a new major in 3/9 attempts (B1): unbounded >= 6.0 on S3, and < 8.0.0 spanning two majors on Cloud SQL. Claude Fable 5.1 loses B1 in 6/9.
• Judge deductions:
  • an environment regex, ^[a-z][a-z0-9]{1,7}$, that rejects "production" (Variables and validation 1, S3);
  • a main.tf of about 300 lines mixing instance, databases and IAM (File layout 1, Cloud SQL);
  • both container_registry_id and a free-text image.registry host, which can disagree (Data sources vs inputs 1, App Service).

Characteristic patterns

• It writes long modules: median 32,853–54,813 output tokens per attempt, two to three times Claude Fable 5.1.
• It writes a README in 9/9 attempts and splits files by concern (logging.tf, iam.tf, observability.tf), though main.tf passes 200 lines in 5/9.
• Its App Service interface shape varies by sample. One sample groups settings into typed objects with optional() defaults (capacity, network, observability, alerts) for 15 inputs and scores 100.0, the only perfect attempt in the field. In it, a terraform_data.release hash of the image and settings triggers azurerm_web_app_active_slot to swap the staging slot into production. Another sample flattens the same concerns into 37 variables.

Representative excerpt

G001/claude-fable-5-2-e1/logging.tf lines 106–117

  # Only the S3 logging service, and only on behalf of the main bucket, may
  # write here.
  statement {
    sid       = "AllowS3AccessLogDelivery"
    effect    = "Allow"
    actions   = ["s3:PutObject"]
    resources = ["${aws_s3_bucket.logs[0].arn}/${local.bucket_name}/*"]

    principals {
      type        = "Service"
      identifiers = ["logging.s3.amazonaws.com"]
    }

The companion log bucket's policy lets the S3 logging service write access logs, and two conditions that follow restrict it to this module's bucket (aws:SourceArn) and account (aws:SourceAccount). No other model in the field grants this permission in any attempt.

Notes on the judge

The judge is Claude Opus 5.5, from the same vendor as this model. Its judge-only score is 100.0 (rank 1), with a median judge grade of 100 on every fixture. Ranked on the mechanical score alone, Claude Fable 5.2 still ranks first (92.7, ahead of Claude Opus 5.5 at 86.4), so its first place does not depend on the judge. The same judge places Claude Fable 5.1 eighth on judge-only score (91.2). Every section scored 2 in 6 of 9 attempts. See Methodology §5.3.

---

Results · Methodology · Scoring criteria