NAI Study Methods

This is a prospective external validation study of a proprietary nocturnist alignment algorithm (NAI)— a frozen, night-anchored scoring function for adapted nocturnist emergency-department schedules. NAI is a sibling of the CAI™ (Circadian Alignment Index): same shape, same 0–100 scale, but with a night-anchored baseline appropriate for physicians whose biological baseline has shifted to night work. The two indices are never averaged or coerced; A/B comparisons are matched within score type.

As with the CAI study, this study does not disclose the internal implementation or optimization methodology of the NAI. It tests whether the frozen NAI agrees with the preferences of an independent sample of practicing nocturnist emergency physicians on data it has never seen.

Board-certified or board-eligible emergency physicians who self-identify as adapted nocturnists (predominantly night-shift practice), who complete informed consent and a short demographic questionnaire. No protected health information is collected. Participation is voluntary and uncompensated.

Each participant completes a session consisting of 11 forced-choice comparisons between two candidate monthly nocturnist schedules and selects the schedule they would personally prefer to work. Pairs are drawn from an approved nocturnist stock pool and matched so that the NAI ranks one schedule higher than the other. Participants are not shown the algorithm's prediction, ranking, or score during the task.

Primary endpoint: agreement between physician preference and the NAI's preferred schedule on forced-choice comparisons.

Secondary endpoint: inter-physician agreement on the same comparisons, reported as a validity checkpoint.

Repeated participation: physicians may complete more than one session. Sessions contribute independent comparison-level observations; clustering by participant is handled in the analysis.

Agreement is reported using Gwet's AC1 as the primary statistic, with Cohen's κ reported alongside for comparability. All estimates are reported with 95% confidence intervals (bootstrap CIs clustered by participant).

The study is organized in the same three sequential stages as the CAI study:

• Stage 1 — Development (n = 250 complete sessions). Held-out development set. Pre-registered decision rule governs whether the NAI may be modified after this stage. If the rule is not triggered, Stage 1 may be pooled with Stage 2 under the pre-specified pooling rule.
• Stage 2 — Validation (n = 1,250 complete sessions). Independent test of the frozen NAI. Publishable target n = 1,500 pooled if the pooling rule is met.
• Stage 3 — Benchmark. Descriptive evaluation at scale on a frozen NAI. No tuning, reweighting, or renormalization.

Reliability weighting, sensitivity analyses, and the pre-registered resolution claims for each stage mirror the CAI study governance. The specific thresholds are held in the analysis plan on file and are not published here, along with the proprietary implementation of the index itself.

The NAI is version-tagged. Every session records the NAI version in effect at the time responses were collected, and the raw response record is preserved. The version being tested is frozen — it is not iterated, reweighted, or renormalized while validation data are being collected.

Validation freeze: The NAI version used in this validation is locked before the first validation response is collected. If a feature change, reweighting, or normalization update is made after validation begins, the current validation cycle is invalidated and a new validation must be conducted with the modified NAI.

The CAI study (rotator schedules, h8 and h12) and this NAI study (adapted nocturnist schedules) are independent. Data are not pooled across indices. A physician may participate in both if they qualify, but each session contributes to exactly one study.

Conducted in accordance with the TRIPOD-AI reporting checklist. No identifiable patient data is used. Participant data is limited to demographic and preference responses, stored in an access-controlled administrative environment. Investigator-initiated, privately funded, designed to be IRB-approval-eligible.