# 02 — Added Methods & Results: Draft Text, Table/Figure Specs, and Author Instructions

**Manuscript:** *Random Forest Classification for Early Detection of Developmental Delay Risk in Children Based on Parenting, Stimulation, and Perinatal Risk Factors* (Pertiwi, Aprizal, Setiawan — ITB Palcomtech Palembang)
**Target venue:** Jurnal ELTIKOM (Politeknik Negeri Banjarmasin) — technical/IT journal, methodological novelty weighted heavily.
**Purpose of this document:** supply insert-ready Methods prose, Results prose templates, exact table/figure specifications, and a single executable instruction for each of the seven additions agreed in `novelty-upgrade-plan.md`.
**Status:** internal working draft. Not for submission as-is.

---

## 0. How to use this document

1. Every number that already exists in `manuscript-v1.txt` is reproduced here **verbatim** (e.g. RF accuracy 0.852 ± 0.082; pooled out-of-fold 16 TP / 7 FN / 2 FP / 32 TN; permutation importance screen time 0.223 ± 0.083; unmatched RF F1 0.515; SES-augmented accuracy 0.983; secondary KPSP at-risk recall 0.15; KPSP 267 appropriate / 22 doubtful / 1 possible deviation; Cronbach α 0.789 and 0.10 / −0.18 / 0.13; community 30.8 ± 16.2 months, 49.3% boys; clinic 62.8 ± 26.2 months, 70.0% boys; matched mean ages 53.1 vs 46.4 months).
2. **Every quantitative result not in `manuscript-v1.txt` is written as a placeholder of the form `[[TO BE COMPUTED: description]]`.** No metric, p-value, count, threshold, coefficient or figure content in this document has been invented. Items marked `[[VERIFY CITE]]` need bibliographic confirmation before submission.
3. Each of the seven sections ends with an **AUTHOR ACTION (one line)** naming the library, the function, and the expected output.

### 0.1 One cross-cutting caveat that must be handled before any of this is written up

The primary analysis is a **matched case–control sample (17 cases / 34 controls, 1:2 age matching)**. Matched case–control sampling fixes the case fraction by design (here 1/3) and therefore does **not** carry the population prevalence of clinically ascertained developmental delay. Consequences:

- Predicted probabilities from the primary model are **not population risks**. All probability-based quantities added below (Brier score, calibration curve, calibration intercept, net benefit, SHAP values expressed on the probability scale) must either (a) be reported **explicitly conditional on the matched sampling frame**, or (b) be computed **after an explicit prior/prevalence correction** to a stated target prevalence, with the assumed prevalence and its source disclosed.
- The SES-augmented sensitivity model (accuracy 0.983) and the SMOTE-based analyses shift priors further; imbalance correction is known to distort calibration (van den Goorbergh et al., *J Am Med Inform Assoc* 2022;29:1525–1534) and must be followed by recalibration or prior correction before any probability or risk stratum is reported.
- This caveat is itself a **methodological contribution** for a technical journal: it forces the manuscript to state *whose* risk the model estimates. It should appear once in Methods (§2.1 below) and be carried into the Limitations section.

---

## 1. TRIPOD+AI reporting alignment (Addition 1)

### 1.1 (i) Methods prose to insert

> **Reporting standard.**
> The study and this report were structured in accordance with the TRIPOD+AI statement for prediction models developed with regression or machine learning methods (Collins et al., 2024). The completed 27-item TRIPOD+AI checklist, indicating the location of each item in this manuscript, is provided as Supplementary Table S1; the abstract is reported against the 13-item TRIPOD+AI for Abstracts checklist (Supplementary Table S2). The model is positioned as a **diagnostic (cross-sectional) prediction model for the presence of clinically ascertained developmental delay**, developed — not validated — in the present dataset; external validation is explicitly out of scope and declared as such. The model output is a **predicted probability** of case status, with a prespecified classification threshold and a three-level risk stratification, defined in §2.1; the probability estimates refer to the sampling frame described in §2.3 and are not interpreted as population risks.

### 1.2 (ii) Results prose template

> **TRIPOD+AI adherence.**
> Of the 27 TRIPOD+AI items, `[[TO BE COMPUTED: count of items fully reported]]` were fully reported in v1, `[[TO BE COMPUTED: count]]` were partially reported, and `[[TO BE COMPUTED: count]]` were not reported; after the additions in this revision, `[[TO BE COMPUTED: count]]` items are fully reported and `[[TO BE COMPUTED: count]]` remain partially reported, with the residual gaps enumerated in Supplementary Table S1 (column *Residual gap*). The items that remain only partially reported are `[[TO BE COMPUTED: list residual partial items after revision]]`.

### 1.3 (iii) What the manuscript currently misses — the working checklist

The table below is the substantive deliverable for this addition. Status was assessed by reading `manuscript-v1.txt` in full. Item numbers follow the TRIPOD+AI 27-item checklist as published (Collins et al., *BMJ* 2024;385:e078378); Results and Discussion item numbers are referred to by section/topic because numbering should be re-verified against the published checklist before submission. `[[VERIFY ITEM NUMBERS]]`

| TRIPOD+AI item | Requirement (abridged) | Status in v1 | Section of v1 affected | Action in this revision |
|---|---|---|---|---|
| 1 Title | Identify the study as developing/evaluating a prediction model; name target population and outcome | **Missing** — "Random Forest Classification" frames it as a classifier, not a prediction-model development study | Title | Retitle (Tier 1, item T1.1 of `04-author-todo.md`) |
| 2 Abstract | Report against TRIPOD+AI for Abstracts (13 items) | **Partial** — no model output type, no calibration, no sample-size/uncertainty statement, no code availability | Abstract | Rewrite per §1.4 |
| 3a Background | Healthcare context, rationale, references to existing models | **Partial** — no existing developmental-delay prediction model is cited or benchmarked | Introduction | Gap table (Tier 1) |
| 3b Target population / intended purpose / intended users | State clinical decision supported, point in care pathway, users | **Partial** — triage use stated in Discussion; users named in Abstract only | Introduction | Make explicit in Introduction |
| 3c Health inequalities between sociodemographic groups | Describe known inequalities in the target population | **Missing** | Introduction | Add 2–3 sentences with citations |
| 4 Objectives | State explicitly development and/or validation | **Partial** — "develops and evaluates" is not framed as model development | Introduction | Rewrite objective statement |
| 5a Data sources, rationale, representativeness | Source + suitability + representativeness per dataset | Present | Methods II.A | Minor edit only |
| 6a Dates, setting, centres | Accrual dates, setting, number/location of centres | Present | Methods II.A | None |
| 6b Eligibility criteria | Inclusion/exclusion | Present | Methods II.A | None |
| 6c Treatments received and how handled | Report treatments, and handling | **Missing** — cases were in active therapy and items were framed retrospectively "before the child started therapy"; this is a treatment-related design feature and is unreported as such | Methods II.A/II.B | Add one paragraph |
| 7 Data preparation | Preprocessing, quality checks, **and whether similar across sociodemographic groups** | **Partial** — preprocessing fully reported; subgroup consistency not assessed | Methods II.D | Add subgroup-consistency sentence |
| 8a Outcome definition and time horizon | Definition, how/when assessed, reference standard, consistency across groups | **Partial** — outcome defined; no time horizon; no assessor/reference-standard detail for the clinical arm | Methods II.C | Add reference-standard sentence |
| 8b Outcome assessor qualifications | Qualifications of assessors for subjective outcomes | **Missing** — the clinic diagnosis source and the enumerators who scored KPSP are described only functionally | Methods II.B/II.C | Add assessment paragraph |
| 8c Blinding of outcome assessment | Actions to blind outcome to predictors | **Partial** — "no label leakage" is asserted for KPSP features but no blinding statement for case-status ascertainment | Methods II.B/II.C | Add explicit statement |
| 9a Predictor selection rationale | How initial predictors were chosen | **Partial** | Methods II.C | Expand one sentence |
| 9b Predictor definitions | All predictors defined with units, categories, **and measurement time** | **Partial** — Table 1 lists variables at cluster level only, no units/coding/measurement time | Methods II.C, Table 1 | Add a full predictor dictionary (Supplementary Table S3) |
| 9c Predictor assessor qualifications | Qualifications/training if subjective | **Missing** | Methods II.B | Add one sentence |
| 10 Study size | How size was arrived at; whether sufficient; sample-size calculation | **Missing** — no EPV/Riley calculation, no justification for 30 cases | Methods II.A | Add §2.2-style sample-size paragraph |
| 11 Missing data | Handling + reasons for omitting data | Present (median/mode imputation within training folds) | Methods II.D | None |
| 12a Data partitioning | How data used/partitioned; no leakage | Present | Methods II.E | Minor edit |
| 12b Predictor handling | Functional form, rescaling, transformations | Present | Methods II.D | Minor edit |
| 12c Model type, rationale, tuning, internal validation, stability | Rationale, hyperparameter tuning, internal validation replay, stability checks | **Partial** — no tuning procedure described, no stability/bootstrap check, no explicit "all steps replayed in CV" statement | Methods II.E | Expand; add stability check |
| 12d Clustering / heterogeneity | Handling of clustering (here: two recruitment channels / sites) | **Missing** | Methods II.E | Superseded by Addition 5 (§5) |
| 12e Measures and plots | Discrimination **and calibration** (+ clinical utility); plots specified | **Partial → Missing** | Methods II.E | Additions 2 and 3 (§2, §3) |
| 12f Model updating / recalibration | Updating arising from evaluation | **Missing** | Methods II.E | Addition 2 (§2.1 prior correction) |
| 12g How predictions were calculated (code/object) | Code, object, API for a non-equation model | **Missing** | Methods II.E | Open-science items below |
| 13 Class imbalance | Why/how + **subsequent recalibration of predictions** | **Partial** — SMOTE reported, no recalibration after imbalance correction | Methods II.E | Addition 2 (§2.1) |
| 14 Fairness | Approaches to address fairness | **Missing** — sex is used as a predictor; no subgroup performance analysis | Methods/Results | Add subgroup performance (Addition 7 table) |
| 15 Model output and risk groups | Output type; thresholds and their rationale | **Missing** — only class labels reported, no probabilities, no thresholds, no risk strata | Methods II.E / Results III.B | Additions 2 and 3 (§2.3, §3) |
| 16 Development vs evaluation differences | Differences between datasets | **Partial** — noted in Limitations, not as an item | Discussion III.F | Fold into channel analysis (§5) |
| 17 Ethics approval | Name the IRB; consent or waiver | **Partial** — written informed consent stated; **IRB/ethics committee not named**, no approval number | Methods II.A | Add one sentence |
| 18 Funding | Funding source and role | **Missing** | Front matter / Acknowledgements | Add |
| Protocol & registration | Protocol accessibility; registry | **Missing** | Methods | Add statement (retrospective registration optional) |
| Data availability | Availability of data or reasons not to share | **Missing** | End matter | Add |
| Code availability | Software libraries, **versions**, licences; analytic code DOI/link | **Partial** — "scikit-learn and imbalanced-learn" without versions; no code link | Methods II.E | Add versions + repository link |
| Results — participant flow | Flow of participants, with/without outcome; diagram helpful | **Partial** — numbers narrated, no flow diagram | Results III.A | Add Figure S1 (flow diagram) |
| Results — participant characteristics | Characteristics table, overall and per data source; missingness | **Missing as a table** — v1 has no characteristics table (Table 1 is a feature-cluster list, not sample characteristics) | Results III.A | Add Table 2 (characteristics by arm) |
| Results — model specification | Full model object/code to allow predictions in new individuals | **Missing** | Results | Add code/model object availability |
| Results — performance with CIs | Performance measures with confidence intervals | **Partial** — mean ± SD across folds reported, but no confidence intervals | Results III.B, Table 2 (v1) | Add 95% CIs |
| Results — subgroup performance | Performance in key subgroups (sex, age group, channel) | **Missing** | Results | Addition 7 (§7) |
| Results — prediction errors | Analysis of prediction errors | **Missing** | Results | Add misclassification review |
| Discussion — usability / implementation | How/when to use; software requirements; handling poor-quality input data; user expertise | **Partial** | Discussion III.F / Conclusion | Add implementation paragraph |

### 1.4 Abstract additions (TRIPOD+AI for Abstracts)

Append to the existing abstract, in this order: (a) explicit statement that this is **model development**; (b) the model output (**predicted probability** with threshold `[[TO BE COMPUTED: prespecified decision threshold]]`); (c) discrimination **and** calibration (Brier score `[[TO BE COMPUTED: Brier score]]`, calibration slope `[[TO BE COMPUTED: slope]]`); (d) net benefit at the clinically relevant threshold `[[TO BE COMPUTED: net benefit]]`; (e) software versions and code availability URL; (f) that external validation is future work.

**AUTHOR ACTION:** open the TRIPOD+AI checklist PDF, complete the 27 rows against the revised manuscript, and record for each row the page/line where the item is reported — expected output: `Supplementary_Table_S1_TRIPODAI.xlsx` with columns *Item | Reported (Yes/Partial/No) | Location | Residual gap* and zero unresolved "No" rows.
*(Reference: Collins GS, Moons KGM, Dhiman P, et al. TRIPOD+AI statement. BMJ 2024;385:e078378.)*

---

## 2. Calibration (Addition 2)

### 2.1 (i) Methods prose to insert

> **Probability estimation, prior correction, and calibration.**
> In the primary age-matched case–control analysis the Random Forest was refit with a probability output (`predict_proba`) on a 25% stratified holdout, with all preprocessing replayed within folds. Because matched case–control sampling fixes the case fraction by design (1:2; 33.3% cases) and imbalance corrections such as SMOTE further distort the prior (van den Goorbergh et al., 2022), raw predicted probabilities were **not** interpreted as population risks. Two probability scales are reported throughout:
> (a) the **sampling-frame scale**, i.e. the model's probabilities as estimated under the matched design, used for discrimination and for the internal calibration diagnostics; and
> (b) the **target-population scale**, obtained by prior correction — the linear predictor is shifted by logit(π) − logit(π₀), where π₀ = `[[TO BE COMPUTED: prevalence of clinically ascertained developmental delay in the target population, with source]]` and π = 0.333 is the matched-sampling case fraction (Pozzolo et al., 2015). Prior correction is a prevalence adjustment only and is not assumed to guarantee calibration; post-correction calibration was therefore re-assessed rather than assumed.
> Calibration was evaluated following the calibration hierarchy of Van Calster et al. (2019) and Steyerberg et al. (2010):
> (i) **mean calibration / calibration-in-the-large** — the observed:expected (O:E) ratio and the calibration intercept, estimated from a logistic regression of observed outcome on the logit of the predicted probability with the slope fixed at 1;
> (ii) **weak calibration** — the **calibration slope** from the same model with the slope free, where 1 indicates ideal calibration and <1 indicates overfitting or over-extreme predictions; the Hosmer–Lemeshow test was deliberately not used (Van Calster et al., 2019);
> (iii) **moderate calibration** — the U-shaped **calibration curve** with lowess smoothing and a bootstrap 95% band, plotted against the ideal 45° line, with the number of children per probability decile reported;
> (iv) **overall performance** — the **Brier score** (Brier, 1950), reported with a bootstrap 95% confidence interval (2,000 resamples), and its decomposition into reliability and resolution.
> Finally, three **risk strata** were defined on the target-population probability scale — low (< `[[TO BE COMPUTED: lower threshold]]`), medium (`[[TO BE COMPUTED: lower threshold]]`–`[[TO BE COMPUTED: upper threshold]]`), and high (> `[[TO BE COMPUTED: upper threshold]]`) — with the thresholds chosen `[[TO BE COMPUTED: on what basis — e.g. to yield a prespecified sensitivity for referral]]`; observed case rates, O:E ratios and exact binomial confidence intervals were computed per stratum. All calibration analyses were repeated for the logistic regression baseline and for the SES-augmented sensitivity model, since adding socioeconomic features changed accuracy to 0.983 and therefore may also change calibration.

### 2.2 (ii) Results prose template

> **Calibration.**
> On the sampling-frame scale, the Random Forest achieved a Brier score of `[[TO BE COMPUTED: Brier, RF, sampling frame]]` (95% CI `[[TO BE COMPUTED]]`–`[[TO BE COMPUTED]]`), compared with `[[TO BE COMPUTED: Brier, logistic regression]]` for logistic regression and `[[TO BE COMPUTED: Brier, decision tree]]` for the decision tree. The mean predicted probability of case status was `[[TO BE COMPUTED: mean predicted probability]]` against an observed case fraction of 0.333; the calibration intercept was `[[TO BE COMPUTED: intercept (95% CI)]]` and the calibration slope `[[TO BE COMPUTED: slope (95% CI)]]`, indicating `[[TO BE COMPUTED: interpretation — over/under-extreme predictions]]`. The calibration curve (Figure 1) showed `[[TO BE COMPUTED: pattern — e.g. overprediction above/below a probability of X]]`, with the largest absolute deviation of the smoothed curve from the ideal line at `[[TO BE COMPUTED: probability value]]` (`[[TO BE COMPUTED: deviation]]`).
> After prior correction to a target prevalence of `[[TO BE COMPUTED: π₀]]`, the calibration intercept was `[[TO BE COMPUTED]]` and the O:E ratio `[[TO BE COMPUTED]]`. Risk stratification on the corrected scale produced `[[TO BE COMPUTED: n]]` children in the low stratum (observed case rate `[[TO BE COMPUTED]]`, O:E `[[TO BE COMPUTED]]`), `[[TO BE COMPUTED: n]]` in the medium stratum (observed case rate `[[TO BE COMPUTED]]`), and `[[TO BE COMPUTED: n]]` in the high stratum (observed case rate `[[TO BE COMPUTED]]`). The high stratum therefore captured `[[TO BE COMPUTED: proportion of all cases]]` of cases, at the cost of referring `[[TO BE COMPUTED: proportion of children]]` of the sample (number needed to screen to detect one case: `[[TO BE COMPUTED]]`).
> The SES-augmented model, whose accuracy rose to 0.983, had a Brier score of `[[TO BE COMPUTED: Brier, SES-augmented]]` after `[[TO BE COMPUTED: correction applied]]`, i.e. `[[TO BE COMPUTED: better/worse]]` calibration than the primary model — `[[TO BE COMPUTED: interpretation of whether the accuracy gain reflects genuine risk discrimination or channel/SES separation]]`. The logistic regression baseline was `[[TO BE COMPUTED: better/equally/worse]]` calibrated than Random Forest (Brier `[[TO BE COMPUTED]]`, slope `[[TO BE COMPUTED]]`, intercept `[[TO BE COMPUTED]]`).

### 2.3 (iii) Table / figure specification

**Figure 1. Calibration of the Random Forest model.**
- Panel A: calibration curve on the **sampling-frame** probability scale. x-axis: mean predicted probability of case status per decile (or smoothed lowess curve); y-axis: observed proportion of cases. Elements: 45° reference line; lowess smoothed curve with bootstrap 95% band; decile points sized by n; rug plot of predicted probabilities; inset reporting Brier score, calibration slope and intercept with 95% CIs.
- Panel B: same curve after **prior correction to target prevalence `[[TO BE COMPUTED]]`**, with the reference line at that prevalence.
- Panel C: distribution of predicted probabilities by observed class (density or histogram, cases vs controls overlaid) to show the discrimination/calibration contrast.
- Caption must state: n = 51 in the primary matched analysis (17 cases, 34 controls); probabilities are not population risks before the correction in Panel B; number of bootstrap resamples.

**Table 4 (new). Calibration and overall predictive performance.**
Columns: Model (RF / logistic regression / decision tree / SES-augmented RF) | Scale (sampling frame / prior-corrected) | Brier score (95% CI) | Calibration slope (95% CI) | Calibration intercept (95% CI) | O:E ratio | Mean predicted probability | Observed case fraction.

**Table 5 (new). Risk strata on the prior-corrected scale.**
Columns: Stratum | Predicted-probability range | n | Cases | Observed case rate (95% CI) | Expected cases | O:E (95% CI) | Sensitivity for case detection | Proportion of all cases | Children referred per case detected.

**AUTHOR ACTION:** use `sklearn.calibration.calibration_curve(..., strategy='quantile')`, `sklearn.metrics.brier_score_loss` and `statsmodels.api.Logit` (regressing the observed outcome on the logit of the out-of-fold predicted probability, once with slope free and once with `offset`, i.e. slope fixed at 1) after applying prior correction with the Pozzolo et al. (2015) factor — expected output: `fig1_calibration.pdf` (3 panels) plus a `calibration_metrics.csv` containing Brier with bootstrap 95% CI, slope, intercept, O:E per model and per scale.

---

## 3. Decision-curve / net-benefit analysis (Addition 3)

### 3.1 (i) Methods prose to insert

> **Decision-curve (net-benefit) analysis.**
> To evaluate clinical utility rather than statistical accuracy, we performed decision-curve analysis (Vickers & Elkin, 2006; Vickers et al., 2016). Net benefit was computed as
> NB(pₜ) = (TP(pₜ)/N) − (FP(pₜ)/N) · pₜ/(1 − pₜ),
> where pₜ is the threshold probability at which a child is referred for full KPSP assessment, TP and FP are counts among children whose predicted risk exceeds pₜ, and N is the number of children. Net benefit was evaluated over pₜ ∈ [`[[TO BE COMPUTED: lower bound — e.g. 0.05]]`, `[[TO BE COMPUTED: upper bound — e.g. 0.50]]`] on the **prior-corrected** probability scale, because decision thresholds are only interpretable on a population-risk scale.
> Three comparators were plotted: (i) **screen everyone** (refer all children), (ii) **screen no one**, and (iii) **KPSP-only**. Two KPSP-only strategies were evaluated and reported separately: **KPSP-only (empirical)**, restricted to the community arm where KPSP was actually administered (n = 290; 267 appropriate, 22 doubtful, 1 possible deviation), and **KPSP-only (analytic)**, in which the published KPSP operating characteristics against the Denver II — sensitivity 60%, specificity 92% (Dhamayanti, 2006) — were applied to the matched sample, because KPSP was not administered in the clinic arm. The KPSP "doubtful" and "possible deviation" categories were counted as screen-positive, following the national referral rule; a sensitivity analysis counted only "possible deviation" as screen-positive.
> The clinical decision threshold was anchored by stating explicitly what a false negative and a false positive each cost in the triage pathway: `[[TO BE COMPUTED: the authors' stated cost ratio — e.g. one missed case is considered equivalent to N unnecessary KPSP administrations]]`, which corresponds to pₜ = `[[TO BE COMPUTED]]`. Net benefit was computed from pooled out-of-fold predictions to avoid optimism, with 2,000 bootstrap resamples for confidence bands, and compared across the model, KPSP-only and the two default strategies at pₜ = `[[TO BE COMPUTED: prespecified analysis thresholds]]`. The same decision curves were produced for the logistic regression and the SES-augmented model, since the latter's accuracy of 0.983 may not translate into net benefit.

### 3.2 (ii) Results prose template

> **Decision-curve analysis.**
> Figure 2 shows net benefit as a function of the threshold probability. The Random Forest exceeded both "screen everyone" and "screen no one" for pₜ between `[[TO BE COMPUTED: lower bound]]` and `[[TO BE COMPUTED: upper bound]]`, with a maximum net benefit of `[[TO BE COMPUTED: max net benefit]]` at pₜ = `[[TO BE COMPUTED]]`. At the clinically anchored threshold of pₜ = `[[TO BE COMPUTED]]`, net benefit was `[[TO BE COMPUTED: RF]]` for the Random Forest, `[[TO BE COMPUTED]]` for logistic regression, `[[TO BE COMPUTED]]` for the decision tree, `[[TO BE COMPUTED]]` for KPSP-only (empirical), `[[TO BE COMPUTED]]` for KPSP-only (analytic), `[[TO BE COMPUTED]]` for screen-everyone and 0 for screen-no-one. The equivalent net-benefit-per-100-children differences were `[[TO BE COMPUTED]]` and `[[TO BE COMPUTED]]` additional cases detected per 100 children screened without an increase in unnecessary KPSP administrations, for Random Forest versus KPSP-only and versus screen-everyone respectively. Beyond pₜ = `[[TO BE COMPUTED: threshold]]` the model's curve crossed below screen-everyone, i.e. above that threshold the model no longer added value over universally screening; the SES-augmented model's curve was `[[TO BE COMPUTED: description of its position relative to the primary model]]`, indicating that the accuracy gain from socioeconomic features `[[TO BE COMPUTED: did/did not]]` translate into net benefit. The analytic KPSP-only strategy (published sensitivity 60%, specificity 92%) produced net benefit of `[[TO BE COMPUTED]]` at the same threshold, `[[TO BE COMPUTED: above/below]]` the model's.

### 3.3 (iii) Figure / table specification

**Figure 2. Decision curves (net benefit) for referral for full KPSP assessment.**
- x-axis: threshold probability pₜ from `[[TO BE COMPUTED: lower]]` to `[[TO BE COMPUTED: upper]]` (labelled also as the implied false-positive:true-positive cost ratio).
- y-axis: net benefit (and, as a secondary y-axis or inset, "standardised net benefit" = net benefit ÷ prevalence, so that 1.0 equals detecting all cases).
- Curves: Random Forest (with bootstrap 95% band), logistic regression, decision tree, KPSP-only (empirical), KPSP-only (analytic), screen-everyone (gently descending line), screen-no-one (horizontal at 0).
- Vertical dashed line(s) at the prespecified clinical threshold(s) `[[TO BE COMPUTED]]`; annotated net-benefit values at those thresholds.
- Caption must state the probability scale used (prior-corrected), the assumed prevalence π₀ `[[TO BE COMPUTED]]`, that predictions are pooled out-of-fold, and that KPSP was not administered in the clinic arm (hence the analytic comparator).

**Table 6 (new). Net benefit at clinically relevant thresholds.**
Columns: Threshold pₜ | Implied cost ratio FP:TP | Random Forest | Logistic regression | Decision tree | KPSP-only (empirical) | KPSP-only (analytic) | Screen everyone | Screen no one | Incremental cases detected per 100 vs screen-everyone | Incremental unnecessary KPSP per 100 vs screen-everyone.

**AUTHOR ACTION:** compute pooled out-of-fold, prior-corrected probabilities and run the Python package `dcurves` (`dcurves.dca(...)` with `outcome`, `modelprob` columns including `all`/`none` and the KPSP-indicator variable as additional strategies, `thresh` grid) — expected output: `fig2_decision_curve.pdf` and `net_benefit_at_thresholds.csv` giving net benefit per strategy per threshold, with the screen-everyone and screen-no-one baselines reproduced as an internal check.

---

## 4. SHAP per-child explanation + worked vignette (Addition 4)

### 4.1 (i) Methods prose to insert

> **Per-child explanations with SHAP.**
> Global permutation importance (50 repeats, F1 scoring) was retained because it is model-agnostic and directly tied to the reported performance metric. To make the model usable at the point of care, per-child explanations were added using SHapley Additive exPlanations (SHAP; Lundberg & Lee, 2017) with the tree-specific `TreeExplainer` (Lundberg et al., 2020). SHAP values were computed for every child in the matched analysis on the prior-corrected probability scale, and aggregated to: (i) a global summary (beeswarm) plot showing, for each feature, the distribution of per-child contributions and the direction of effect; (ii) a mean-absolute-SHAP ranking to be compared against the permutation-importance ranking as a robustness check on the model's interpretation (see §5.3); and (iii) individual waterfall plots. For each child, the explanation is decomposed into the model's expected value (base rate) plus per-feature additive contributions summing to that child's predicted risk.
> For the reported worked example, one child was selected according to a prespecified rule — `[[TO BE COMPUTED: selection rule — e.g. the child with a predicted probability nearest the prespecified referral threshold, so that the explanation illustrates a borderline decision]]` — and the SHAP values were mapped to counseling actions using a prespecified table mapping each modifiable feature to a counselling target, drawn from existing Indonesian primary-care guidance (Kementerian Kesehatan RI, 2016). Explanations of fixed, non-modifiable features (birth weight, child sex, birth order) were shown for transparency but were not mapped to actions, consistent with the study's framing of modifiable caregiving factors as the actionable component.

### 4.2 (ii) Results prose template

> **Per-child explanations.**
> Global mean |SHAP| identified `[[TO BE COMPUTED: top feature]]` as the strongest contributor to individual predictions (mean |SHAP| = `[[TO BE COMPUTED]]` on the probability scale), followed by `[[TO BE COMPUTED: ordered list of next 4 features with mean |SHAP|]]`. The SHAP ranking `[[TO BE COMPUTED: agreed / partially agreed / disagreed]]` with the permutation-importance ranking of screen time (0.223 ± 0.083), child age (0.162 ± 0.076), household smoking (0.112 ± 0.065), birth weight (0.076 ± 0.052), child sex (0.066 ± 0.041) and authoritative parenting (0.043 ± 0.053). The beeswarm plot (Figure 3) showed that higher screen time shifted predictions towards case status by up to `[[TO BE COMPUTED: maximum positive SHAP contribution]]` probability units, whereas higher stimulation score shifted predictions away from case status by up to `[[TO BE COMPUTED: maximum negative SHAP contribution]]`.
> The individual waterfall plots (Figure 4) show that between `[[TO BE COMPUTED: minimum]]` and `[[TO BE COMPUTED: maximum]]` features were needed to account for 80% of the absolute SHAP mass for a given child (median `[[TO BE COMPUTED]]`), i.e. a typical explanation is short enough to be presented to a frontline worker.

### 4.3 Illustrative worked vignette — **ILLUSTRATIVE STRUCTURE ONLY; NOT REAL DATA**

> ⚠️ **The vignette below is a structural template the authors must populate with real model output. Every value is a placeholder. It must not be submitted until every `[[TO BE COMPUTED: …]]` is replaced by an actual number, and the child it describes is real and de-identified. Do not present it as a real case until then.**
>
> **Worked example (illustrative).** A `[[TO BE COMPUTED: age]]`-month-old `[[TO BE COMPUTED: sex]]` child attending the posyandu had the following profile: screen time `[[TO BE COMPUTED: category]]`, household smoking exposure `[[TO BE COMPUTED: category]]`, birth weight `[[TO BE COMPUTED: grams]]`, stimulation score `[[TO BE COMPUTED: score]]`, authoritative parenting score `[[TO BE COMPUTED: score]]`. The model assigned a predicted probability of case status of `[[TO BE COMPUTED: probability]]`, against a base rate of `[[TO BE COMPUTED: expected value]]`; the prediction therefore crossed the referral threshold of `[[TO BE COMPUTED]]` and the child was flagged for full KPSP assessment. The five largest contributions were: screen time `[[TO BE COMPUTED: SHAP value]]` (raising risk), `[[TO BE COMPUTED: feature]]` `[[TO BE COMPUTED: SHAP value]]`, `[[TO BE COMPUTED: feature]]` `[[TO BE COMPUTED: SHAP value]]`, `[[TO BE COMPUTED: feature]]` `[[TO BE COMPUTED: SHAP value]]` (lowering risk), and `[[TO BE COMPUTED: feature]]` `[[TO BE COMPUTED: SHAP value]]`. Of these, `[[TO BE COMPUTED: list of modifiable features among the top contributors]]` are modifiable. The corresponding counselling targets are: `[[TO BE COMPUTED: action 1]]`, `[[TO BE COMPUTED: action 2]]`, `[[TO BE COMPUTED: action 3]]`. The explanation required no information beyond the intake questionnaire, so it is deliverable by the posyandu worker immediately after data entry.
>
> **Template for Figure 4A (waterfall).** x-axis: contribution to predicted probability (negative → positive, left to right), with features ordered by absolute contribution; y-axis: features; base value and final predicted probability annotated at the two ends; colour scale indicating feature value (low → high).

### 4.4 (iii) Figure / table specification

**Figure 3. SHAP summary (beeswarm) plot — global explanation.**
x-axis: SHAP value (change in predicted probability of case status); y-axis: features ordered by mean |SHAP|; each point is one child (n = 51) coloured by that feature's value (low → high, one colour bar shared across features); caption states the probability scale (prior-corrected) and that all 24 candidate risk-factor features are shown or that a top-k subset (`[[TO BE COMPUTED: k]]`) is shown for legibility.

**Figure 4. Individual SHAP explanations.**
- Panel A: waterfall plot for the selected child (see template above).
- Panel B: force/waterfall plot for a second, contrasting child — `[[TO BE COMPUTED: selection rule for the contrast case — e.g. a false negative with the lowest predicted probability]]`.
- Caption must state the selection rule for both children, the base value, the referral threshold, and that the figure is generated from de-identified intake-questionnaire data only.

**Table 7 (new). Feature-level agreement between global explanation methods.**
Columns: Feature | Mean |SHAP| (probability scale, 95% CI over children) | Permutation importance (F1 decrease, mean ± SD, from v1 Table 3) | Rank by SHAP | Rank by permutation importance | Rank shift.

**Table 8 (new). Mapping from modifiable features to counselling targets** (the prespecified action table used for the vignette).
Columns: Feature | Risk direction | Threshold for action | Counselling target | Source of guidance.

**AUTHOR ACTION:** install `shap` and compute `shap.TreeExplainer(model).shap_values(X_holdout)` (or `shap.Explainer(model.predict_proba, ...)` with `shap.Explainer(model)` for the sklearn API) on the prior-corrected probabilities; generate `shap.summary_plot(..., plot_type='beeswarm')` and `shap.plots.waterfall(...)` for the two selected children — expected output: `fig3_shap_beeswarm.pdf`, `fig4_shap_waterfall.pdf`, and a `shap_values_matrix.csv` (children × features) from which the vignette numbers are read.

---

## 5. Confound-robust / channel-invariance analysis (Addition 5)

This is the paper's most distinctive methodological component, because the manuscript already shows that socioeconomic features push accuracy from `[[TO BE COMPUTED: matched/unmatched baseline]]` to 0.983 and F1 to 0.870 on the unmatched sample — evidence that part of the case–control contrast is carried by the recruitment channel. Here that weakness is quantified and bounded rather than reported as a limitation.

### 5.1 5(a) Channel-only baseline

**(i) Methods prose to insert**

> **Channel-only baseline.**
> A deliberate "channel-only" (null) model was fitted using recruitment channel (clinic case arm vs posyandu control arm) as the *only* feature, with the same algorithm family, the same hyperparameters and the same evaluation protocol as the primary model. If questionnaire risk factors carried no information beyond the recruitment channel, the primary model's performance would match this baseline. The **incremental value of the risk factors** is therefore expressed as (a) the difference in cross-validated F1 and recall between the primary model and the channel-only model, and (b) the *partial* performance of the primary model within a single channel (§5.3), which removes channel information entirely. Reported alongside is the channel-only univariate association with case status (`[[TO BE COMPUTED: AUC/or accuracy of channel alone]]`), to make explicit how much of the label is recoverable from channel alone.

**(ii) Results prose template**

> A model using recruitment channel alone achieved accuracy `[[TO BE COMPUTED: channel-only accuracy]]`, F1 `[[TO BE COMPUTED: channel-only F1]]` and recall `[[TO BE COMPUTED: channel-only recall]]` (95% CIs `[[TO BE COMPUTED]]`–`[[TO BE COMPUTED]]`), i.e. channel alone already explains `[[TO BE COMPUTED: proportion of primary-model F1]]` of the primary model's F1. Adding the 24 questionnaire risk factors increased F1 by `[[TO BE COMPUTED: ΔF1]]` (95% CI `[[TO BE COMPUTED]]`–`[[TO BE COMPUTED]]`, paired comparison across the 25 CV folds), so the risk factors carry predictive information beyond channel. Conversely, `[[TO BE COMPUTED: proportion of the primary model's discrimination attributable to channel]]` of the model's apparent performance is channel-attributable, which is the internally-validated magnitude of the confounding flagged by the SES sensitivity analysis (accuracy 0.983, F1 0.870).

**(iii) Table/figure spec**

**Table 9 (new). Channel-only baseline and incremental value of risk factors.**
Columns: Model | Features | Accuracy (95% CI) | Precision | Recall (95% CI) | F1 (95% CI) | Specificity | ΔF1 vs channel-only (95% CI, paired over folds).
Rows: Channel only | All 24 risk factors (primary) | Risk factors + SES (sensitivity) | Risk factors minus channel-correlated features `[[TO BE COMPUTED: list]]`.

**AUTHOR ACTION:** fit the identical `RandomForestClassifier` pipeline (same 500 trees, `max_features='sqrt'`, balanced class weights, same 5×5 repeated stratified CV) with the feature matrix reduced to the channel indicator alone, then compute paired fold-wise F1 differences against the primary model with `scipy.stats.ttest_rel` or a paired bootstrap — expected output: `channel_only_baseline.csv` with per-fold and mean ± SD metrics for both models and the paired ΔF1 confidence interval.

### 5.2 5(b) Matching/balancing on socioeconomic status in addition to age

**(i) Methods prose to insert**

> **Socioeconomic balancing.**
> Because the two recruitment channels differ in socioeconomic composition, and because adding socioeconomic features to the model raised accuracy to 0.983 (F1 0.870), age matching alone does not guarantee comparability of the case and control groups. Two complementary strategies were applied to the primary analysis. First, the 1:2 matching was extended from age alone (caliper ±6 months) to **age plus a socioeconomic anchor** — `[[TO BE COMPUTED: the SES variable chosen as the matching anchor — e.g. maternal education category or household income quintile]]` with caliper `[[TO BE COMPUTED: caliper]]` — yielding `[[TO BE COMPUTED: n cases]]` cases and `[[TO BE COMPUTED: n controls]]`; the primary model was refit on this set under the identical protocol. Second, because matching discards data, an **alternative balancing approach** was applied to the full unmatched sample via propensity-score **weighting** (or matching) on the joint distribution of age and the socioeconomic variables, with balance assessed by standardised mean differences (SMD) before and after balancing, a target of |SMD| < 0.1, and effective sample size reported; the weighted model's performance is reported as a sensitivity analysis. Balance on age and socioeconomic variables is displayed as a Love plot. Neither approach uses the outcome in the balancing step.

**(ii) Results prose template**

> Extending matching from age alone to age plus `[[TO BE COMPUTED: SES anchor]]` retained `[[TO BE COMPUTED: n cases]]` cases and `[[TO BE COMPUTED: n controls]]`. Standardised mean differences before balancing were `[[TO BE COMPUTED: SMD, maternal education]]` for maternal education and `[[TO BE COMPUTED: SMD, income]]` for household income, and after balancing were `[[TO BE COMPUTED]]` and `[[TO BE COMPUTED]]` (all |SMD| `[[TO BE COMPUTED: < / >]]` 0.1; Figure 5). On this set, Random Forest achieved accuracy `[[TO BE COMPUTED]]`, recall `[[TO BE COMPUTED]]` and F1 `[[TO BE COMPUTED]]`, compared with 0.852 ± 0.082, 0.750 ± 0.198 and 0.791 ± 0.141 on the age-only-matched set; the change in F1 was `[[TO BE COMPUTED: ΔF1]]` (95% CI `[[TO BE COMPUTED]]`). Under propensity weighting on the full unmatched sample (effective n = `[[TO BE COMPUTED]]`), F1 was `[[TO BE COMPUTED]]` versus `[[TO BE COMPUTED]]` unmatched and unweighted (`[[TO BE COMPUTED: F1 from v1 unmatched analysis / TO BE COMPUTED]]`). `[[TO BE COMPUTED: interpretation — whether performance is materially reduced once socioeconomic balance is enforced, and by how much]]`.

**(iii) Table/figure spec**

**Table 10 (new). Performance under age-only matching vs age + socioeconomic balancing.**
Columns: Analysis set | Matching/balancing | n cases | n controls | Balance achieved (max |SMD|) | Accuracy (95% CI) | Recall (95% CI) | F1 (95% CI) | ΔF1 vs age-only-matched.
Rows: Age-only matched (primary, from v1) | Age + SES matched | Full unmatched, propensity-weighted | Full unmatched, unadjusted.

**Figure 5. Love plot of covariate balance.**
x-axis: absolute standardised mean difference (0 to `[[TO BE COMPUTED: max]]`); y-axis: age, sex, maternal education, household income, maternal occupation, birth weight, screen time, stimulation score `[[TO BE COMPUTED: confirm covariate list]]`; two shapes (before / after balancing); vertical dashed reference line at |SMD| = 0.1.

**AUTHOR ACTION:** use `sklearn.neighbors.NearestNeighbors` or the `MatchIt`-equivalent Python package `pymatch` for the age+SES caliper matching, and `statsmodels` / `sklearn.linear_model.LogisticRegression` propensities plus `cobalt`-style SMD computation (`tableone` or hand-computed SMD) for the weighted analysis — expected output: `matching_balance.csv` (SMD before/after per covariate), `fig5_love_plot.pdf`, and `performance_ses_balanced.csv` (metrics per analysis set with fold-wise SDs).

### 5.3 5(c) Feature-importance invariance under channel hold-out

**(i) Methods prose to insert**

> **Invariance of feature importance to recruitment channel.**
> If the importance ranking of caregiving factors is an artefact of the clinic-versus-posyandu contrast, it should not survive when channel information cannot be exploited. Three checks were performed: (i) permutation importance was recomputed **separately within the case arm (n = 17) and within the control arm (n = 34)** of the matched set, and the two rankings were compared using Spearman's ρ and the top-3 overlap; (ii) permutation importance was recomputed on a model trained **with channel-correlated features removed** (`[[TO BE COMPUTED: list of features excluded]]`) to test whether the screen-time dominance persists; and (iii) the SHAP-based mean |SHAP| ranking (§4) was compared with the permutation ranking (§4.3, Table 7), since the two methods weight features differently. Agreement is reported with rank-correlation coefficients and their bootstrap confidence intervals; disagreement is reported as disagreement rather than resolved by preference.

**(ii) Results prose template**

> Permutation importance computed within channel gave Spearman's ρ = `[[TO BE COMPUTED: rho, with p or bootstrap CI]]` between the case-arm and control-arm rankings, with `[[TO BE COMPUTED: n shared features in the top 3]]` of the top three features shared. Screen time ranked `[[TO BE COMPUTED: rank in case arm]]` in the case arm and `[[TO BE COMPUTED: rank in control arm]]` in the control arm. After removing channel-correlated features, screen time's permutation importance was `[[TO BE COMPUTED]]` (was 0.223 ± 0.083), and the top-three ordering was `[[TO BE COMPUTED: ordering]]`. Agreement between SHAP and permutation rankings was Spearman's ρ = `[[TO BE COMPUTED]]`. Taken together, the importance ordering was `[[TO BE COMPUTED: stable / partly stable / unstable]]` to the removal of channel information, which `[[TO BE COMPUTED: supports / qualifies]]` the interpretation of screen time and household smoking exposure as genuine modifiable risk signals rather than channel markers.

**(iii) Table/figure spec**

**Table 11 (new). Feature-importance invariance.**
Columns: Feature | Permutation importance, full model (from v1 Table 3) | Permutation importance, within case arm | Permutation importance, within control arm | Permutation importance, channel-correlated features removed | Mean |SHAP| | Rank (full) | Rank (case arm) | Rank (control arm) | Rank (features removed).
Footer row(s): Spearman's ρ between rankings; top-3 overlap; bootstrap CI.

**Figure 6. Importance-ranking stability.** Slope/bump chart with features on the y-axis and four ranking columns on the x-axis (full model | case arm | control arm | channel-correlated features removed), lines connecting each feature across columns, line width/spread indicating rank volatility; annotate Spearman's ρ between adjacent columns.

**AUTHOR ACTION:** call `sklearn.inspection.permutation_importance(model, X_subset, y_subset, scoring='f1', n_repeats=50, random_state=...)` three times (within case arm, within control arm, and on the feature-reduced matrix) and compare rankings with `scipy.stats.spearmanr` — expected output: `importance_invariance.csv` (feature × condition × mean ± SD) and `fig6_rank_stability.pdf`, with the Spearman ρ matrix printed.

---

## 6. Ablation by feature cluster (Addition 6)

### 6.1 (i) Methods prose to insert

> **Feature-cluster ablation.**
> To quantify which cluster of the 24 risk factors drives prediction, four nested feature sets were evaluated under the identical protocol (same algorithm, hyperparameters, 5 × 5 repeated stratified CV, same folds): (1) **perinatal only** (birth weight, low birth weight status, prematurity, birth asphyxia proxy, seizure/hospitalisation history); (2) **perinatal + parenting** (authoritative, authoritarian and permissive subscale scores, primary caregiver); (3) **perinatal + parenting + stimulation** (total stimulation score); and (4) **+ screen time** (the full modifiable-feature set). A fifth set, **demographic only** (child age, sex, birth order, number of under-fives in the household), was included as an age/sex reference because child age is itself the second-ranked predictor in the full model (permutation importance 0.162 ± 0.076). Two reporting conventions are used: cumulative sets (nested, above) to show incremental value, and **leave-one-cluster-out** sets (full model minus each cluster) to show the marginal contribution of each cluster. Given the matched design, the child-age feature is interpreted as a matching covariate as much as a predictor, and this is stated where the demographic-only set is discussed. Because the clusters differ in the number of features, discrimination metrics are reported alongside `[[TO BE COMPUTED: whether any corrected-for-dimensionality metric is used — e.g. no correction, stated explicitly]]`, and the ablation was designed to be interpreted descriptively rather than as a formal test of feature utility.

### 6.2 (ii) Results prose template

> **Ablation.** Table 12 reports performance by feature cluster. The perinatal-only model achieved accuracy `[[TO BE COMPUTED]]`, recall `[[TO BE COMPUTED]]` and F1 `[[TO BE COMPUTED]]`; adding parenting scores changed F1 by `[[TO BE COMPUTED: ΔF1]]` (95% CI `[[TO BE COMPUTED]]`); adding the stimulation score changed F1 by a further `[[TO BE COMPUTED]]`; and adding screen time changed F1 by `[[TO BE COMPUTED]]`, to the full model's 0.791 ± 0.141. The demographic-only reference model achieved F1 `[[TO BE COMPUTED]]`. Leave-one-cluster-out analysis showed that removing `[[TO BE COMPUTED: cluster]]` produced the largest F1 decrease (`[[TO BE COMPUTED]]`, 95% CI `[[TO BE COMPUTED]]`), followed by `[[TO BE COMPUTED: cluster]]` (`[[TO BE COMPUTED]]`). Memory and inference cost per model were `[[TO BE COMPUTED: training time / model size, if reported]]`. The ablation indicates that the modifiable-cluster contribution to discrimination is `[[TO BE COMPUTED: description of whether the modifiable clusters add materially over perinatal-only]]`, which is the empirical basis for the paper's framing of modifiable caregiving factors as the actionable component; where the gain is small, the manuscript states so plainly rather than inferring actionability from importance ranks alone.

### 6.3 (iii) Figure / table spec

**Table 12 (new). Feature-cluster ablation.**
Columns: Feature set | n features | Features included | Accuracy (95% CI) | Precision | Recall (95% CI) | F1 (95% CI) | Specificity | ΔF1 vs previous nested set (95% CI).
Rows: Demographic only | Perinatal only | + Parenting | + Stimulation | + Screen time (full) | Full minus perinatal | Full minus parenting | Full minus stimulation | Full minus screen time.

**Figure 7. Cumulative and leave-one-out ablation.** Two-panel bar chart. Panel A: nested sets on the x-axis (in the fixed order above) with F1 on the y-axis, error bars = SD across the 25 folds, and fold-wise ΔF1 annotated between bars. Panel B: leave-one-cluster-out sets with ΔF1 relative to the full model, zero line marked. Caption states that all sets use identical folds and hyperparameters, and that no correction for differing feature counts was applied.

**AUTHOR ACTION:** run the existing modelling script in a loop over the feature-set definitions (a `dict` of column-name lists) using the same `RepeatedStratifiedKFold(n_splits=5, n_repeats=5)` object and `cross_validate(..., scoring=['accuracy','precision','recall','f1','roc_auc'])` — expected output: `ablation_clusters.csv` (feature set × metric × fold) and `fig7_ablation.pdf`, with paired fold-wise ΔF1 and 95% CIs.

---

## 7. Extra benchmarks: XGBoost/gradient boosting + calibrated logistic regression (Addition 7)

### 7.1 (i) Methods prose to insert

> **Additional benchmarks and the interpretability–performance trade-off.**
> Two additional benchmarks were added. (1) **Gradient boosting (XGBoost; Chen & Guestrin, 2016)** with early stopping on an inner validation split, chosen because boosted trees are the strongest commonly reported tabular baseline and because comparing against them is required to justify retaining a bagged-tree model. (2) **Calibrated logistic regression**, i.e. the existing logistic-regression baseline with probability calibration `[[TO BE COMPUTED: method — e.g. Platt/sigmoid scaling or isotonic regression, fitted within training folds only]]`, included because a simple, fully transparent model with good calibration is a legitimate competitor in a triage setting where decisions are made on probabilities. Models were compared on: discrimination (F1, recall, ROC-AUC and PR-AUC, given the class imbalance), calibration (Brier score, calibration slope and intercept, §2.1), clinical utility (net benefit at the prespecified threshold, §3.1), computational cost (training time, prediction latency) and explanatory transparency (availability of exact per-feature contributions — TreeSHAP for Random Forest and XGBoost, coefficients and odds ratios for logistic regression). The Random Forest is retained **not** because it maximised accuracy by a small margin, but because it offers `[[TO BE COMPUTED: the empirical basis for the claim — e.g. parity in F1/recall with XGBoost within the CI while providing exact tree-based per-child explanations and no additional hyperparameter burden]]`; where XGBoost outperformed Random Forest by a margin that exceeds fold-to-fold variability, the manuscript reports the margin and states the trade-off explicitly rather than asserting Random Forest superiority. With n = 17 matched cases, differences smaller than `[[TO BE COMPUTED: the observed fold-to-fold SD / minimum detectable difference]]` are not interpreted as evidence of superiority.

### 7.2 (ii) Results prose template

> **Benchmarks.** Under identical folds, XGBoost achieved accuracy `[[TO BE COMPUTED]]`, precision `[[TO BE COMPUTED]]`, recall `[[TO BE COMPUTED]]` and F1 `[[TO BE COMPUTED]]` (95% CIs `[[TO BE COMPUTED]]`), and ROC-AUC `[[TO BE COMPUTED]]` / PR-AUC `[[TO BE COMPUTED]]`; the calibrated logistic regression achieved F1 `[[TO BE COMPUTED]]` and Brier score `[[TO BE COMPUTED]]`. Relative to Random Forest (accuracy 0.852 ± 0.082, precision 0.890, recall 0.750, F1 0.791; Brier `[[TO BE COMPUTED]]`), the F1 difference was `[[TO BE COMPUTED: ΔF1 vs XGBoost]]` (95% CI `[[TO BE COMPUTED]]`) and `[[TO BE COMPUTED: ΔF1 vs calibrated LR]]` (95% CI `[[TO BE COMPUTED]]`), i.e. `[[TO BE COMPUTED: within/beyond]]` the fold-to-fold variability. Net benefit at the prespecified threshold was `[[TO BE COMPUTED]]` (Random Forest), `[[TO BE COMPUTED]]` (XGBoost) and `[[TO BE COMPUTED]]` (calibrated logistic regression). Calibration slope was `[[TO BE COMPUTED]]` for XGBoost and `[[TO BE COMPUTED]]` for calibrated logistic regression, against `[[TO BE COMPUTED]]` for Random Forest. Training time per fold was `[[TO BE COMPUTED]]` s (Random Forest), `[[TO BE COMPUTED]]` s (XGBoost) and `[[TO BE COMPUTED]]` s (calibrated logistic regression). All three models support per-child explanations, but `[[TO BE COMPUTED: explanation of whether tree-based SHAP or coefficients were used, and which model is recommended on which basis]]`. The comparison therefore justifies retaining Random Forest on `[[TO BE COMPUTED: the stated basis]]`, and where it does not, the recommendation is `[[TO BE COMPUTED]]`.

### 7.3 (iii) Figure / table spec

**Table 13 (new). Extended model comparison.**
Columns: Model | Hyperparameters tuned (with search space) | Accuracy (95% CI) | Precision | Recall (95% CI) | F1 (95% CI) | ROC-AUC | PR-AUC | Brier score | Calibration slope | Calibration intercept | Net benefit at prespecified pₜ | Training time (s/fold) | Interpretability method.
Rows: Decision tree | Logistic regression | Calibrated logistic regression | Random Forest (primary) | XGBoost/Gradient boosting.

**Figure 8. Interpretability–performance trade-off.** Scatter plot: x-axis = net benefit at the prespecified threshold (or F1); y-axis = a transparency score `[[TO BE COMPUTED: definition of the transparency axis, or replace this figure with a qualitative table row]]`; one point per model; error bars on both axes; points labelled. If no defensible ordinal transparency scale can be defined, replace Figure 8 with a plain ranked table and delete the axis — do not invent a composite score.

**AUTHOR ACTION:** install `xgboost` and run the same `RepeatedStratifiedKFold` scheme with `XGBClassifier(n_estimators=..., eval_metric='logloss', early_stopping_rounds=..., scale_pos_weight=...)` under `RandomizedSearchCV`, plus `CalibratedClassifierCV(LogisticRegression(...), method='sigmoid', cv=<inner>)` fitted inside each training fold; compute Brier/AUC with `sklearn.metrics` — expected output: `benchmark_extended.csv` (model × fold × metric) and `fig8_tradeoff.pdf` (or the Ranked-table alternative), with all hyperparameter grids recorded for the supplementary material.

---

## 8. New references to add (all real; verify bibliographic detail before submission)

- Brier, G. W., "Verification of forecasts expressed in terms of probability," *Monthly Weather Review*, vol. 78, no. 1, pp. 1–3, 1950.
- Chen, T., and Guestrin, C., "XGBoost: A scalable tree boosting system," in *Proc. 22nd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining (KDD)*, 2016, pp. 785–794.
- Collins, G. S., Moons, K. G. M., Dhiman, P., et al., "TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods," *BMJ*, vol. 385, e078378, 2024. doi:10.1136/bmj-2023-078378.
- Lundberg, S. M., and Lee, S.-I., "A unified approach to interpreting model predictions," in *Advances in Neural Information Processing Systems 30 (NIPS 2017)*, 2017.
- Lundberg, S. M., Erion, G., Chen, H., et al., "From local explanations to global understanding with explainable AI for trees," *Nature Machine Intelligence*, vol. 2, pp. 56–67, 2020. `[[VERIFY CITE]]`
- Pozzolo, A. D., Caelen, O., Johnson, R. A., and Bontempi, G., "Calibrating probability with undersampling for unbalanced classification," in *Proc. IEEE Symposium Series on Computational Intelligence (SSCI)*, 2015, pp. 159–166. `[[VERIFY CITE]]`
- Riley, R. D., Ensor, J., Snell, K. I. E., et al., "Calculating the sample size required for developing a clinical prediction model," *BMJ*, vol. 368, m441, 2020. doi:10.1136/bmj.m441.
- Steyerberg, E. W., Vickers, A. J., Cook, N. R., et al., "Assessing the performance of prediction models: a framework for some traditional and novel measures," *Epidemiology*, vol. 21, no. 1, pp. 128–138, 2010. doi:10.1097/EDE.0b013e3181c30fb2.
- Van Calster, B., McLernon, D. J., van Smeden, M., Wynants, L., and Steyerberg, E. W., "Calibration: the Achilles heel of predictive analytics," *BMC Medicine*, vol. 17, art. 230, 2019. doi:10.1186/s12916-019-1466-7.
- van den Goorbergh, R., van Smeden, M., Timmerman, D., and Van Calster, B., "The harm of class imbalance corrections for risk prediction models: illustration and simulation using logistic regression," *Journal of the American Medical Informatics Association*, vol. 29, no. 9, pp. 1525–1534, 2022. doi:10.1093/jamia/cac993. `[[VERIFY CITE]]`
- Vickers, A. J., and Elkin, E. B., "Decision curve analysis: a novel method for evaluating prediction models," *Medical Decision Making*, vol. 26, no. 6, pp. 565–574, 2006. doi:10.1177/0272989X06295361.
- Vickers, A. J., van Calster, B., and Steyerberg, E. W., "Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests," *BMJ*, vol. 352, i6, 2016. `[[VERIFY CITE]]`
- Wolff, R. F., Moons, K. G. M., Riley, R. D., et al., "PROBAST: a tool to assess the risk of bias and applicability of prediction model studies," *Annals of Internal Medicine*, vol. 170, no. 1, pp. 51–58, 2019. doi:10.7326/M18-1376. (Cited only if a PROBAST self-appraisal is added; it is recommended but optional.)

---

## 9. Consolidation notes

- **Table numbering.** v1 uses Tables 1–3. This document proposes Tables 4–13 and Figures 1–8 plus Supplementary Tables S1–S3 and Figure S1. Renumber once the authors decide which items go to supplementary material; a technical journal will likely accept at most 2–3 new main-text figures, so Figures 1 (calibration), 2 (decision curve) and either 3 (SHAP summary) or 6 (importance stability) should be main-text, with the rest supplementary.
- **Consistency risk.** The permutation-importance ranking (screen time 0.223 ± 0.083 as dominant predictor) is asserted with a large SD relative to its mean. Section 5.3 and Figure 6 exist precisely to test whether that ranking is stable; if it is not, the Introduction and Abstract must be revised to soften "dominant predictor" accordingly. Do not pre-commit to the claim.
- **Every `[[TO BE COMPUTED: …]]` in this document is a required input.** Any sentence still containing one must not appear in a submitted manuscript.
