Current evaluation result

Immutable evidence. This page is the result page generated inside the checksum-verified v0.3.4 publication bundle, at source revision e9ffdf26a2a216c4c1fcc0dc810b0b498e726abe. It is reproduced here without retyping a score; the evidence map records its manifest and report checksums.

Evaluation-only results

Partition: evaluation only. These tables do not include or pool development cases.

Evaluation scope and audit

• Freeze ID: real-project-v2
• Bounded claim: This independent slice supports only descriptive per-profile comparisons for the source-only sampled Java, Rust, and C++ repositories and the References and Definition operations recorded here. Report v2 separately. A combined v1/v2 view may preserve and aggregate only documented language/profile strata and their denominators; it must not pool the slices into language-wide, ecosystem-wide, cross-language, latency, memory, or cold-start claims.
• Audited corpus: 12 repositories and 36 cases

Per-profile denominators

Language	Reference profile	Registered adapter	Repositories	Cases	Population exclusions	Source-review replacements
java	eclipse-jdtls	adapters/lsp/eclipse-jdtls.json	4	12	27 (repository has fewer than 20 eligible source files × 3; repository lacks a GitHub-reported SPDX license × 10; repository lacks root build marker for java × 10; repository reported source size exceeds 157286400 bytes × 17; repository source tree is truncated by GitHub × 2)	none
rust	rust-analyzer	adapters/lsp/rust-analyzer.json	4	12	40 (repository has fewer than 20 eligible source files × 3; repository lacks a GitHub-reported SPDX license × 11; repository lacks root build marker for rust × 9; repository reported source size exceeds 157286400 bytes × 29; repository source tree is truncated by GitHub × 3)	none
cpp	apple-clangd-21	adapters/lsp/apple-clangd-21.json	4	12	36 (repository has fewer than 20 eligible source files × 1; repository lacks a GitHub-reported SPDX license × 13; repository lacks root build marker for cpp × 11; repository reported source size exceeds 157286400 bytes × 22; repository source tree is truncated by GitHub × 1)	none

Hash-bound artifact provenance

Artifact	File	SHA-256
Protocol	benchmarks/evaluation/real-project-v2/protocol.json	1c0ff8a5c0fc4f4a66b564dcdb9dd384e1b9f5ee28134a1dd34770b85a4ffbf1
Selection	benchmarks/evaluation/real-project-v2/selection.json	88a853512223c5c4859df2c9fe068e83364a3c3bb84defdd2d9f2c502aa58b2c
Independent review	benchmarks/evaluation/real-project-v2/review.json	a22a32fa5945fc3d716b866f3fb9b4b055fce9984534dd8e2417fae569b86e1d
Source lock	benchmarks/evaluation/real-project-v2/sources.json	cd0684b7210d63ae21b7b953699e53e1eaafd750ec8918bb4ae783dc9a377857
Reviewer openai-gpt-5.6-sol	benchmarks/evaluation/real-project-v2/reviews/openai-gpt-5.6-sol.json	956a043ea74e0e894d156c45f842e16b6d7cb3e9cdb0274ef57406c45e179afe
Reviewer anthropic-claude-fable-5	benchmarks/evaluation/real-project-v2/reviews/anthropic-claude-fable-5.json	47d53e24dccf31a7b8c06f746f0d338bca1bcb38d2f9ae3810ce6b56effccd99
Adjudication real-project-v2-human-adjudication	benchmarks/evaluation/real-project-v2/reviews/adjudication.json	726076c0759eb88a836185f9430dc1ff4f6fb7fbe67d9d7fdabe686423bc59db

Snapshot inputs

Candidate	Runner	Requested version	Profile	Environment	Report SHA-256
apple-clangd-21	clangd	21.0.0	adapters/lsp/apple-clangd-21.json	macos/“aarch64”/Native/HostSpecific	2e8167d21745617e6d507875bb190206d30954d5ee5f2ad873db4825ff514c3f
bifrost	bifrost	8ddf13625653caf927137a0de1966401055debad	—	macos/“aarch64”/Native/HostSpecific	0341a8f8aee26d5f8ccd88b48f1cd665efc2a49ed914418346ccd7227c62a03e
eclipse-jdtls	JDT Language Server (Standard)	1.61.0-202607142124	adapters/lsp/eclipse-jdtls.json	macos/“aarch64”/Native/HostSpecific	b0b41090ff1d6f1f4393761148c181800926bb09eb213ca91e2d3bc17a45b2ca
rust-analyzer	rust-analyzer	2026-07-13	adapters/lsp/rust-analyzer.json	macos/“aarch64”/Native/HostSpecific	b039a8be300568355668808de8a8161ae1e2735a770395aeb47d8e55753be2a4

Required-destination comparison

Reference profile	Shared	Bifrost found	Reference found
Apple clangd 21.0.0	12	8/12 (66.7%)	1/12 (8.3%)
Eclipse JDT LS 1.61.0-202607142124	12	12/12 (100.0%)	6/12 (50.0%)
rust-analyzer 2026-07-13	12	11/12 (91.7%)	12/12 (100.0%)

Location-level precision and recall

TP, FP, and FN are reported without true negatives. Strict precision counts every extra result; policy-adjusted precision excludes authored and policy-allowed extras.

Reference profile	Analyzer	Cases	TP	FP	FN	Destination recall	Exact-token recall	Strict precision	Policy-adjusted precision	Exact-set case rate	Extras/success
Apple clangd 21.0.0	Bifrost	12	143	0	9	94.1%	94.1%	100.0%	100.0%	66.7%	0.00
Apple clangd 21.0.0	Apple clangd	12	60	4	92	39.5%	39.5%	93.8%	93.8%	8.3%	0.00
Eclipse JDT LS 1.61.0-202607142124	Bifrost	12	75	0	0	100.0%	100.0%	100.0%	100.0%	100.0%	0.00
Eclipse JDT LS 1.61.0-202607142124	Eclipse JDT LS	12	18	0	57	24.0%	24.0%	85.7%	100.0%	33.3%	0.25
rust-analyzer 2026-07-13	Bifrost	12	73	0	1	98.6%	98.6%	100.0%	100.0%	91.7%	0.00
rust-analyzer 2026-07-13	rust-analyzer	12	74	24	0	100.0%	100.0%	71.2%	75.5%	58.3%	1.25

Strict contract conformance

Reference profile	Shared	Both exact	Bifrost only	Reference only	Neither
Apple clangd 21.0.0	12	1	7	0	4
Eclipse JDT LS 1.61.0-202607142124	12	6	6	0	0
rust-analyzer 2026-07-13	12	7	4	0	1