San Jose Crash Map Methodology and Data Limitations

Release scope: 2020–2024. Official snapshots acquired 6 September 2026. This is independent research, not an official City ranking. City-reported records cover SJPD city streets and county expressways. CHP highway additions are a conservative, coordinate-verified subset. Neither the combined sample nor the highway sample is a complete census of crashes in San José.

Source package and acquisition

The user-described ZIP was not available in the workspace or Downloads and no path or download URL was supplied. Consequently, its README, registers, workbook, collector, tests, offline results and release status remain uninspected. Original official data were independently acquired. This limitation does not convert unknown package contents into verified data.

City data: fresh ArcGIS Hub exports of the official OPN/512 crash archive and OPN/513 vehicle/party archive, filtered to 2020–2024. Raw received bytes are preserved, sometimes losslessly gzip-compressed. Fresh exports, old cached downloads, query bodies, timestamps and checksums remain distinct. The rolling OPN/553 layer currently covers 2021–2025, despite June 2026 City website prose naming 2020–2024. Its rows, severity values and geometry differ from OPN/512. No union of those overlapping layers is taken. The common 2020–2024 period was retained for CHP, FARS and Caltrans comparison.

CHP data: all five statewide annual crash, party and person-related tables were actually downloaded from California's official CKAN DataStore export API. Original raw-file endpoints returned403; failures are logged. The DataStore export is a separate official serialization, not claimed byte-identical to the blocked uploaded file. Its source resource metadata says the DataStore contains all source records. Every table was parsed and primary keys checked. Modern Collision Id is not assumed to equal legacy SWITRS CASE_ID.

TIMS is a cross-check of overlapping CHP/SWITRS records, never an additional crash source. Its June 2026 snapshot differs from the September CCRS snapshot. FARS is only a fatal-crash validation source. OTS has different agency coverage and a mix of units; its Total Fatal and Injury row counts people, not crashes. None is appended to the canonical table.

The 2024 CHP/TIMS data remain provisional. The 2024 FARS file is the first annual file, not the final file. City data are a dated dynamic snapshot, not certified final. All agency update dates are retained where available.

Grain and schema

One canonical row is one crash. City CRASHNAME and CHP Collision Id are source identities. Crash count is count of unique canonical crash_id, never party count or person-table row count. Parties and people remain separate processed tables. CHP witness-only records do not become injured people. City vehicle-party injuries can describe several people; vehicle-row count is not injured-person count.

Numeric nulls remain null. All-null sums remain unknown. Sums with partial coverage represent observed reported outcomes and include corresponding known-crash counts. True zeros require explicit source zeros or complete reconciled detail. Flag counts include only explicit/derived positive records; flag-known and flag-unknown denominators accompany results. A mixed string such as Driver, Unknown does not prove no pedestrian. A positive term can still establish involvement when another term is unknown.

Fatal crashes have at least one fatality. Severe-injury crashes have at least one severe injury and can also be fatal. KSI crashes have at least one person killed or severely injured. KSI people equal fatalities plus severe injuries only when both components are known. Minor/moderate injury crashes may also have severe/fatal people; lesser-injury-only excludes KSI. These categories are not automatically additive.

City explicit injury totals take precedence for matched City/CHP crashes. CHP severe injury totals require person-table injury reconciliation with reported crash totals. Disagreements are retained and not silently corrected. The source fields and source crosswalk permit auditing both records.

Cross-source deduplication and network inclusion

The primary duplicate rule matches unique normalized nine-digit San José report numbers, restricting CHP to SJPD NCIC 4313 and requiring crash dates within one day. Example: City 23-159-0769 and CHP 2023-231590769 normalize to the same report key. Nonunique keys, incompatible dates and unresolved matches are quarantined. Exact-match time, geometry and outcome differences are logged; no fuzzy record merge is performed. City values are retained and linked CHP identities recorded.

CHP highway additions require an explicit route identifier in the primary road, a supporting freeway/highway flag, a known crash time, a point within the official City boundary, and a point no farther than 50 m from the same official Caltrans route geometry valid on the crash date. IsHighwayRelated alone does not qualify a surface street. A freeway mentioned as a cross street does not prove the crash occurred on the freeway. County expressways remain in the City-reported network.

Any prospective highway addition with a City crash within 200 m and 30 minutes is held out as a possible overlap. Even a same-minute/50 m candidate is not automatically merged. Missing-coordinate San José-coded CHP records and unmatched nonhighway records remain in exclusion tables; they are not assumed outside San José or zero. Matched City records with conflicting freeway classification are withheld from the core unless resolved. No inferred conversion of suspect CHP coordinates is applied.

Geographic QA

Coordinates use the City's official exported geometry, not an assumption that LATITUDE/LONGITUDE attributes are identical. Fresh X/Y requested in EPSG4326 become longitude/latitude; the original attributes remain preserved. Earlier cached X/Y in EPSG2227 are explicitly transformed when that archived input is used. Source-record coordinate disagreements are not erased.

The official City Limits polygon is OPN/372, last updated 6 May 2026. It is valid without repair. EPSG26910 (NAD83/UTM10N, meters) supports local distances and lengths. The current boundary is applied retrospectively; historical annexations are not reconstructed. The main sample uses exact polygon covers, including points on the boundary. Outside points within 30 m are identified separately and excluded rather than silently buffered into the city.

REPORTEDLOCATIONFOUND is an undocumented Yes/No flag. All source-designated intersection crashes in the inspected snapshot can carry No. It is not interpreted as a certificate of accuracy or as missing coordinates. Official intersection geometry independently checks assignments. The map is released only after valid geometry, plausible coordinates, boundary exceptions, unique crash IDs and intersection outliers are checked.

Intersections

Street normalization standardizes common suffixes (Road/Rd, Street/St, Avenue/Av/Ave and others), punctuation, spacing and cardinal directions. Street order is ignored for a display pair. Geographic grouping primarily uses official INTERSECTIONNUMBERINT matched to the City's INTNUM intersection GIS key, which avoids merging distant crossings that happen to share names. Multiple labels per ID are retained as variants; a stable modal normalized pair is displayed.

The primary table uses City-designated Intersection or Related crashes, an exact official intersection-ID match, distance no greater than 75 m from that official point, and City-boundary inclusion. Non-related crashes are not assigned to an intersection simply because Street A and Street B name it. Unmatched/outlier rows are quarantined from intersection rankings. Independent radius analyses at 15.24/30.48/45.72/76.2 m (50/100/150/250ft) test proximity sensitivity, using unique crashes per intersection; a crash can appear in multiple radius buffers, so those tables are not additive.

Road segments and named roads

City centerlines are clipped to the official boundary, dissolved by normalized street name, line-merged where connected, and split into bins no longer than 402.336 m (0.25 mile). Connected components and end fragments have their actual lengths recorded. These are not guaranteed equal-length bins. An 804.672 m (0.5 mile) alternative is provided. Segment indices are deterministic within this dated geometry; they are not permanent agency segment IDs.

A City crash is assigned to its nearest centerline bin only within 30 m. At/related intersection records are flagged as ambiguous road-of-travel assignments. Named-road proximity totals are nearest-road summaries, not proof of travel direction, road ownership or causation. They must not be described as exposure-adjusted danger. Intersections, divided roads, frontage roads and grade-separated crossings can affect results.

Highway segments use only the matched same-route Caltrans alignment valid at the crash date, rather than the nearest unrelated City street. Caltrans path third coordinates are postmile measures, not elevations. Route geometries are clipped and split into 0.25/0.5 mile bins; inclusion in a segment requires 30 m proximity. Route-alignment version and R/L labels are preserved and not assumed to mean compass travel direction.

Priority Safety Corridors and severity

Official PSC geometry is retained separately from crash-derived hotspots. Independent proximity uses the union of PSC buffers, with 15/30/50 m sensitivity. Main tables use 30 m. Multiple nearby PSC names are retained; corridor totals can overlap and must not be summed. Primary PSC rankings use City-reported records, consistent with Vision Zero's source coverage. Highway proximity to an overcrossing PSC does not establish that the freeway crash occurred on that corridor.

Separate rank columns cover total crashes, KSI crashes, fatalities, severe injuries, pedestrian/bicycle-involved KSI crashes, speeding-positive KSI crashes and a custom severity score. Rank ties are minimum shared ranks; display order then uses total crashes and name. Small differences are not claims of statistical significance.

Custom severity score =3×(fatalities+severe injuries)+minor injuries+moderate injuries. It is person-outcome weighted, not3×KSI crash count. The City's2025 Vision Zero Action Plan p26 documents3×fatal/severe and1×minor/moderate person weighting for its 2017–2021 PSC analysis, while retaining older corridors. Applying similar weights to this different period/geography is a custom analysis, not an official City score or replication of its PSC process. Priority corridors were never substituted for independently derived intersection rankings.

Road users, time and factors

Pedestrian/bicycle/motorcycle involvement is different from the injured person's mode. A pedestrian-involved KSI crash can severely injure a vehicle occupant. Mode-specific KSI people require verified party/person attribution, expected party coverage and outcome reconciliation. City party injury fields are incomplete in earlier years, so person-mode totals cannot automatically cover all 2020–2024 crashes. Motorcycles/mopeds and powered scooters have explicit group definitions; scooters are not silently counted as bicycles. Vehicle occupant injured people are distinct from vehicle parties; uninjured occupant totals are unavailable.

City CRASHDATETIME is local wall time even though its serialized suffix is+00; paired CRASHDATETIMEUTC is separately preserved. CHP local timestamps use explicit annual format parsing. Unknown times are not midnight. Ambiguous/nonexistent Pacific daylight-saving UTC conversions remain null. Annual/month/day/weekend use local dates. Rush hour is weekday 07:00–09:59 and 16:00–18:59. Darkness uses the recorded lighting field; clock-night 18:00–05:59 is separate. No sunrise/sunset calculation is implied.

Speeding, unsafe-turn/movement, red-signal, DUI/intoxication and hit-and-run flags use official source coding. Definitions differ between systems and are separated by network in tables. CHP turn/movement codes also include lane changes, signals and starting/backing; its group is not precisely equivalent to the City's Unsafe Turn Movement label. Conditions and factors can overlap. Reported factors are not independent causal findings or legal conclusions.

Exposure, validation and reproduction

Exposure analysis is restricted to fully validated station-to-station mainline intervals, with exact route/postmile matching, official network lengths, appropriate AADT back/ahead handling, year matching and stable numerator geocoding. Back and ahead volumes describe adjacent legs and are never summed. Ramp and mainline volumes are not interchangeable. See exposure_analysis.md for eligible intervals, rate formulas, exclusions and census-year alignment. Raw frequency and exposure-adjusted risk are different analyses.

FARS is checked using both reported City code and coordinates inside the same current boundary. Missing FARS coordinates and different road/agency coverage prevent treating unmatched annual totals as errors. Source differences are reported, not forced to agree. OTS and TIMS are independently cited qualified cross-checks, never additional canonical events.

Reproduce in the order listed in the project README. All raw checksums and exact official URLs are in acquisition manifests; source notes explain versions and failed endpoints. Analytical scripts write the canonical dataset, ranking/sensitivity files, per-field known counts, publication tables and claim register. Scientific-integrity tests cover normalization, partial unknown values and all-null sums; final release validation additionally checks actual data uniqueness, reconciled counts, geometry, required files and map payload consistency.

Additional source-field coverage checks

The fresh City archive's SpeedingFlag has only 53 positive rows in 2020 and 129 in 2021, while its own violation descriptions identify 1,113 and 1,158 speeding-positive rows, respectively. The analysis therefore preserves the original flag, description evidence and party evidence, and counts a speeding-positive crash when any explicit source evidence establishes it. Negative flags are not used to override positive text. This is an analyst-defined evidence union, not the raw City flag metric.

The City's hit-and-run flag has 152 positives in 2020 and 68 in 2021, compared with 1,478 in 2022. The party file has no complete dedicated hit-and-run field; Leave Accident Scene provides only sparse additional positive evidence. Negative hit-and-run flags are treated as unknown for the analytical involvement flag, and positive counts are lower-bound recorded evidence. This population discontinuity is a reporting-quality warning, not proof of a real increase.

Historical Caltrans geometry versions (notably SR 85) remain separate segment IDs. Route-level totals combine valid-date records correctly, but a physical segment can be split across geometry versions; segment rankings are explicitly version-specific. Exposure intervals separately use validated postmile/geometry calibration.

Completion-audit corrections and explicit coverage

The 2020–2024 window is the selected common cross-source period. The City's current rolling GIS layer covers 2021–2025, so this report does not call 2020–2024 the latest City dashboard period.

Intersection-radius sensitivity is centered on the 3,704 intersections eligible for the primary ranking. It does not independently screen every one of the 22,668 reference GIS intersections. Counts can overlap between radius buffers.

Every segment now includes `any_crash_within_30m_psc`: at least one assigned crash lies within 30 meters of official PSC geometry. This describes geometric proximity, not official corridor membership or grade-separated roadway equivalence.

Within-CHP duplicate screening goes beyond unique Collision Id values. Repeated agency/report identities and coincident timestamp/coordinate groups are checked against the complete crash, party and person payloads. An exact duplicate 2022 source alias is represented once; a reviewed 2024 pair with different reports and principal vehicles and previous-collision coding remains two official reports. The review does not infer their impact sequence. See `research/chp/reconciliation_methodology.md`, `within_source_alias_crosswalk.csv` and `within_source_coincident_review.csv` for the complete rule and evidence.

Vehicle-occupant crash involvement uses explicit unambiguous motor-vehicle driver/passenger evidence. City Driver party categories establish positive presence; a parked car by itself does not. CHP explicitly linked non-witness driver/passenger records can also establish presence. Known complete nonoccupant party sets can establish false; other cases remain unknown. Motorcycle riders and cyclists retain their separate modes. This crash-involvement flag never substitutes for an occupant count or a mode-specific injury count.

The map export now requires a passing release record whose canonical SHA-256 matches the current CSV. Changing a canonical file invalidates its prior export approval until the data gates run again.