Data Sources & Methodology
InvisibleSignal uses CDC surveillance, Census population estimates, and NIH funding data. Every dataset listed here has an ingest script in data/scripts/. The 4-weight risk index uses ONLY CDC NNDSS (twice).
Federal Data Sources
4 sources with ingest scriptsAll data is sourced from U.S. federal agency open data programs. Each source has a corresponding ingest script and a role in the product.
Weekly state-level case counts for Lyme disease, Babesiosis, Anaplasmosis, and Ehrlichiosis. The gold standard for tracking nationally notifiable tickborne illness incidence across all U.S. states and territories.
How InvisibleSignal uses this data:
State-level incidence rate calculations, multi-year trend analysis, and state fallback when a county is not county-coded by CDC.
County-coded Lyme disease cases (confirmed and probable) for 688 counties. Reporting completeness varies by state: Maine >99% complete, Texas 0% county-coded. 152,018 cases reconciled exactly against CDC's own published total.
How InvisibleSignal uses this data:
Geographic risk index (0.30 + 0.30 of the 4-weight model — same CDC rate used twice, as historical incidence and current exposure geography). County-resolution maps. Reporting-completeness transparency.
Annual population estimates by county. Required to compute per-capita incidence rates from CDC's absolute case counts.
How InvisibleSignal uses this data:
Per-capita incidence calculations (cases per 100,000 population).
Comprehensive database of NIH-funded research projects, including funding amounts, principal investigators, institutions, and disease category classifications. Used for SPECTRA funding-gap analysis, NOT as a risk-model input.
How InvisibleSignal uses this data:
Funding gap analysis (Lyme vs. comparable diseases), research investment trends, and institutional research mapping. NOT used in the risk index.
Risk Index Methodology
The risk index is a transparent weighted sum with no learned parameters. The weights are the entire model:
Historical incidence
CDC county-coded Lyme surveillance, falling back to state NNDSS
Current exposure geography
Same resolved rate — county where CDC county-coded, else state
Seasonal vector activity
Month-of-year Ixodes activity curve
Symptom overlap
Fraction of Lyme's catalogued symptoms reported. Unvalidated.
Geographic incidence is weighted twice (0.30 + 0.30) using the SAME resolved CDC rate — once as historical incidence, once as current exposure geography. This is not two independent sources.
Validated in data/scripts/validate_geographic_risk.py. Ranked on 2022 only, scored against 2023 only (no 2023 information in the ranking).
Spearman ρ
0.882
Counties
424
Pearson 0.872. Tier exact agreement 0.769. Tier within-one 0.991. Counties from states where CDC county-coded <90% of cases are excluded (their rate is a floor of unknown depth).
The temporal holdout also produces AUC 0.987 for high-incidence classification, but we do not headline that number: prevalence in this sample is 98.1%, and specificity rests on 8 negative counties. Spearman ρ = 0.882 is the honest headline.
The output is a 0–100 exposure risk index for a place and time of year. It is not a probability that the user has Lyme disease.
- The FHIR export carries it as
RiskAssessment.prediction.probabilityDecimalscoped explicitly to "Lyme disease exposure risk for the reported region and season" - Symptom overlap is exported as a
%quantity with numerator and denominator, never as a probability - Geographic incidence is normalised against a fixed ceiling of 300 per 100,000, not the observed maximum (Dukes County MA sits near 1,425)
An earlier version advertised an XGBoost model with AUC 0.984, precision 0.96, recall 0.94, trained on tens of thousands of samples. Those numbers have been withdrawn. Here is why:
- The target variable was a function of the features. The label was a thresholded weighted sum of the inputs plus σ=0.03 noise. The AUC measured how well gradient boosting approximates arithmetic.
- The advertised sample count was 250 observations enumerated 216 times. 50 states × 5 years = 250 real state-year observations, expanded across 12 months × 6 symptom-burden levels × 3 exposure levels. An 80/20 split leaked near-identical rows.
- The model was never actually served. No model artifact exists in this repository. Production has always run the deterministic heuristic.
See docs/MODEL_CARD.md for the full analysis.
Privacy & Compliance
InvisibleSignal uses only aggregate public data. No individual patient records, diagnoses, or personal health information is accessed, stored, or processed. All data sources are publicly available aggregate statistics from federal open data programs.
Risk assessments and symptom overlap fractions are exported as HL7 FHIR R4 resources (RiskAssessment and Observation) for import into clinician EHR systems. Export is user-initiated and follows USCDI+ standards.
Epistemic Status
Established
- 152,018 CDC Lyme cases reconciled exactly against CDC's own total
- Spearman ρ = 0.882 out-of-sample validation on 424 counties
- Reporting completeness by state (Maine >99%, Texas 0%)
- 4 published weights as a deterministic index
- HL7 FHIR R4 export for EHR integration
Preliminary / Unvalidated
- Month-of-year Ixodes activity curve (0.20 weight)
- Symptom overlap fraction (0.20 weight)
- NIH funding-gap analysis (no validation against policy outcomes)
Not Yet Conducted
- Zero primary patient interviews
- Zero chart reviews or diagnostic accuracy studies
- Zero clinical trials or prospective cohort studies
- No independent validation by a second research team