# Data quality and publication review

This release passes the listed crash-grain and coordinate checks. It is a qualified official-source snapshot, not a complete combined citywide crash census. The original user research ZIP remains unavailable and uninspected.

## Release gates

- canonical_ids_present_and_unique: **PASS**.
- canonical_rows_equal_reconciled_core: **PASS**.
- annual_crash_counts_reconcile: **PASS**.
- all_years_in_2020_2024: **PASS**.
- all_published_coordinates_present: **PASS**.
- all_published_points_inside_official_boundary: **PASS**.
- all_published_map_eligible: **PASS**.
- city_ksi_arithmetic_reconciles: **PASS**.
- city_injury_counts_nonnegative: **PASS**.
- city_dates_hours_pass: **PASS**.
- highway_same_route_within_50m: **PASS**.
- intersection_ids_unique: **PASS**.
- intersection_counts_reconcile: **PASS**.
- intersection_outliers_excluded: **PASS**.
- segment_ids_unique: **PASS**.
- segment_counts_do_not_exceed_crashes: **PASS**.
- fresh_city_parties_present: **PASS**.
- no_unreviewed_chp_repeated_reports: **PASS**.
- no_unreviewed_chp_coincident_reports: **PASS**.
- within_chp_aliases_excluded_from_canonical: **PASS**.
- within_chp_representatives_present: **PASS**.

## Record-count reconciliation

- Fresh City archive: 24,359 unique 2020–2024 crashes; no duplicate CRASHNAME or TCRNUMBER.
- City core inside current official boundary: 24,133; 226 City-source rows outside, including 203 within 30m of the boundary. They are retained in processed exceptions and excluded from the main map.
- Exact City/CHP duplicate crosswalk: 8,921 CHP records represented by City rows, never added again.
- Reviewed CHP highway additions: 13,844; combined core: 37,977. This is a conservative mapped subset, not full highway coverage.
- Source intersection-ID matches missing: 553; at/related records excluded by geography/intersection QA: 452.

## Dates, severity, locations and street names

- City missing coordinates: 0; implausible coordinate envelope: 0; invalid dates: 0; KSI arithmetic disagreements: 0.
- Missing core fatality counts: 0; missing severe counts: 0. Unknowns remain blank and coverage columns accompany every metric.
- City unmatched or absent nearest-segment assignments: 57; highway records without a strict 30m same-route segment: 783.
- Street-pair display normalization never substitutes for geographic intersection identity. Name variants and unmatched reference IDs are recorded in the full tables.
- Official geometry is preferred to source latitude/longitude attributes. Some legacy SJPD CHP coordinates appear encoded differently; a diagnostic conversion was never applied to publication data.
- Current 2026 municipal boundary is used retrospectively. Historical annexations and roadway geometry changes remain a limitation.

## Cross-source discrepancies

- Exact-match outcome disagreements (any compared outcome): 1,841; coordinate differences over200m: 2,399; time differences over30minutes: 62. City records take precedence, with both sources auditable in the crosswalk.
- The City rolling layer currently covers2021–2025 while the dated explanatory webpage still names2020–2024. The fresh archive and rolling layer have different IDs, injury values and geometry. They are not unioned.
- City Action Plan annual aggregates differ from the fresh archive. Exact differences are in research/city/plan_vs_fresh512_reconciliation.csv; do not overwrite event records with a published aggregate.
- TIMS is an earlier overlapping CHP/SWITRS snapshot; differences are unresolved and recorded in research/chp/tims_chp_crosscheck.csv. No TIMS crashes are added.
- FARS uses fatal crashes, different geographic/agency definitions and some missing coordinates. City-code and boundary counts are both retained. No FARS crashes are added.
- OTS Total Fatal and Injury is a person count; it excludes CHP-taken reports on shared-jurisdiction roads from city rankings. Do not compare it as a crash count.

## Parties, people and factors

- Fresh City party rows: 50234; unique party IDs: 50234. Party records are not crashes or injured people.
- City crashes whose party person metrics fully validate: 14006; crashes with no party rows: 27; party rows missing at least one severity field: 16610.
- Party mode conflicts: 19; unclassified party modes: 2897. Conflicts cannot establish a positive mode unless another unambiguous party does so.
- Actual pedestrian/bicyclist/motorcyclist/occupant KSI people are published only where party/person coverage and injuries reconcile. Generic KSI people in mode-involved crashes are separately labeled.
- City older speeding flags contradict positive violation descriptions. The analytical speeding flag unions explicit positive evidence. Hit-and-run negative flags are unknown because historical coverage is inconsistent. Annual factor patterns are not automatically comparable.

## Exposure and rank stability

- Exposure rates cover only eligible 2023 census-year station intervals (October2022–September2023). Every numerator crash is in the canonical core; withheld intervals and uncertainty are documented in exposure_analysis.md.
- The exposure intervals are not a citywide or five-year risk ranking. Exact Poisson intervals condition on fixed estimated exposure and do not include all traffic-volume uncertainty.
- Intersection-radius and quarter/half-mile segment files support sensitivity analysis. Road-segment bins can be short end fragments. Highway geometry versions can split the same physical corridor across era-specific bins.
- Ranking position is descriptive. Ties and small counts do not establish statistical differences; reported factors do not prove causation.

## Reproducibility and verification

- Seven scientific-integrity tests check name normalization, mixed unknown flags, null-safe aggregation, boolean roundtrip and known absence versus unknown road-user involvement. Five additional tests verify within-CHP aliases and eight verify occupant-involvement logic. Final release gates above test actual input/output invariants.
- All source URLs, timestamps, successful downloads and failed requests are preserved in manifests and source_notes.md. Lossless gzip files preserve received bytes; checksums identify the raw snapshots.
- The page uses the same canonical crash IDs and metrics; map_qa.json records payload and intersection count reconciliation after export.
- The site receives a production build and TypeScript check. Actual browser interaction and visual verification are recorded separately in browser_qa.json, including the build checked; these data checks alone do not establish a browser pass.

## Claims requiring withholding or qualification

- Withhold claims of full combined San José coverage, final2024 data, five-year citywide exposure risk, complete road-user person counts, complete early hit-and-run ascertainment, or official City endorsement of custom rankings.
- Qualify all hotspot statements with the source network,2020–2024 period, dated snapshot and location definition. Keep highway subsets and eligible exposure intervals visibly separate.
- The missing user ZIP prevents certifying its catalog, collector tests, workbook, download index or release status. It does not negate the independently verified official data in this folder.

## Within-CHP event identity and corrected release

- Exact repeated source aliases removed: 1. CHP:1705550 is represented by CHP:1705549; the crash and full party/person payloads match except storage identities. Their outcomes are never summed.
- Reviewed distinct coincident report groups retained: 1. The 2024 reports contain different principal vehicles and previous-collision coding. Shared time/coordinates alone do not establish duplicate events or an impact sequence.
- Decisions and original payload hashes are in data/processed/dedup/within_source_qa.json and the corresponding alias/review crosswalks. Five regression tests cover identity, multiplicity and stale review invalidation.
- City party-person metrics validate for 14006 source crashes before the boundary filter and 13865 retained City core crashes. These are different populations.
- Export is permitted only when the current canonical SHA-256 matches the release QA record. Browser/runtime verification evidence is maintained separately in output/browser_qa.json; do not infer a live-browser pass from the data checks.