# 04 — Author To-Do: Prioritised Revision Checklist

**Manuscript:** *Random Forest Classification for Early Detection of Developmental Delay Risk in Children Based on Parenting, Stimulation, and Perinatal Risk Factors* (Pertiwi, Aprizal, Setiawan)
**Target venue:** Jurnal ELTIKOM — technical/IT journal; reviewers weight **methodological novelty**.
**Rejection reason to answer:** *"does not sufficiently demonstrate clear novelty or a significant contribution."*
**Companion document:** `02-added-methods-results-draft.md` (contains the insert-ready prose, table/figure specs and executable instructions referenced below).

**How to read this checklist.** Items are split into:

- **Tier 1 — Reframe (text only).** No data analysis required. Can be done immediately.
- **Tier 2 — Re-run analyses on data you already hold.** The 30 cases / 290 controls / 320-questionnaire dataset is sufficient for every item here.
- **Tier 3 — New data collection.** Not achievable in this revision cycle; list them as future work or defer.

Each item states: what to do · tool/library · estimated effort · manuscript section changed · **REQUIRED** or **OPTIONAL**.

**Effort** is expressed in author-working-days (1 `d` ≈ one full working day for one author already familiar with the dataset). Figures assume no learning-curve time beyond a short library lookup.

**Standing rule for every item below:** no number may be reported unless it has been computed from the real dataset. Placeholders in the companion document (`[[TO BE COMPUTED: …]]`) are instructions, not results.

---

## 0. Executive triage

| | Count | What it buys |
|---|---|---|
| **Tier 1 — must do all** | 14 items | Removes the "reframe" half of the rejection and closes most TRIPOD+AI reporting gaps at near-zero cost. |
| **Tier 2 — must do the core 8** | 12 items (8 REQUIRED, 4 OPTIONAL) | This is what actually moves a hard technical reviewer: validity treatment, calibration, decision analysis, ablation, per-child explanation, benchmark rigor. |
| **Tier 3 — declare as future work** | 7 items | Signals a credible research programme; not required for acceptance at ELTIKOM. |
| **Minimum viable revision** | Tier 1 (all) + Tier 2 items marked **REQUIRED-A** | ≈ 12–16 author-days. Below this, the novelty argument remains a reframe only and a hard reviewer may reject again. |

**Dependency chain:** T2.2 (calibration + prior correction) → T2.3 (decision curve) → T2.8 (SHAP on the corrected scale) → T1.5 (abstract numbers). T2.4 (channel baseline) → T2.5 (SES balancing) → T2.6 (invariance). Everything else is independent.

---

## Tier 1 — Reframe (text only; no data needed)

| # | Item | What to do | Tool | Effort | Section changed | Req? |
|---|---|---|---|---|---|---|
| **T1.1** | Retitle | Replace the classifier framing with prediction-model framing. Suggested: *"From Screening to Triage: An Interpretable Random Forest Model of Modifiable Caregiving Risk Factors for Early Detection of Developmental Delay in Indonesia."* Title must name target population and outcome. | — | 0.1 d | Title; running head | **REQUIRED** |
| **T1.2** | Gap table | Replace the one-sentence gap ("literature is concentrated on stunting") with a table: rows = prior Indonesian ML studies [4]–[8] plus international screen-time/parenting prediction work; columns = outcome predicted / inputs used / design / validity treatment / decision analysis. One column per gap you claim to fill must be visibly empty for every prior row. | — | 1.0 d | Introduction §1, new Table 1 | **REQUIRED** |
| **T1.3** | Rewrite "contributions" | Replace the three design-convention statements (no leakage · age matching · permutation importance) with **five falsifiable, gap-mapped claims** (C1 problem, C2 validity, C3 decision, C4 boundary, C5 interpretability), each phrased so that a result could disconfirm it. Hygiene items move to Methods, not the contribution list. | — | 0.5 d | Introduction, final paragraph | **REQUIRED** |
| **T1.4** | Promote the negative result | Move the KPSP secondary analysis (recall of at-risk class 0.15; 267 appropriate / 22 doubtful / 1 possible deviation) from a footnote-like subsection to a **headline contribution and a stated conclusion**. Reframe as: *questionnaire risk factors delimit what they cannot replace in unselected populations.* | — | 0.5 d | Abstract; Introduction; Results III.E; Conclusion | **REQUIRED** |
| **T1.5** | Abstract rewrite | Restructure against the TRIPOD+AI for Abstracts checklist (13 items): state model **development**; target population and setting; data sources; outcome; n and events (30 cases, 290 controls; primary matched 17/34); model type and **output (predicted probability)**; performance with uncertainty; **calibration** and **net benefit**; code availability; external validation as future work. Numbers not yet computed stay as placeholders until Tier 2 lands. | — | 0.5 d | Abstract | **REQUIRED** |
| **T1.6** | TRIPOD+AI adherence statement + completed checklist | Insert a two-sentence adherence statement in Methods, complete the 27-item TRIPOD+AI checklist against the revised manuscript, and add it as Supplementary Table S1. Use the current item-by-item gap assessment in the companion document §1.3 as the starting point. | TRIPOD+AI checklist PDF; spreadsheet | 0.5 d | Methods (new closing subsection); Supplementary S1–S2 | **REQUIRED** |
| **T1.7** | Intended use, users, care pathway | State explicitly: the model is a **diagnostic (cross-sectional) prediction model for clinically ascertained developmental delay**, intended for posyandu workers, used **before** full KPSP administration, to prioritise children for KPSP and referral. Name the decision the model supports. | — | 0.3 d | Introduction §3b; Conclusion | **REQUIRED** |
| **T1.8** | Health inequalities paragraph | Add 2–3 sentences with citations on known developmental-outcome inequalities (socioeconomic gradient, urban–rural access to SDIDTK) in the target population. | Literature; existing refs | 0.3 d | Introduction | **REQUIRED** |
| **T1.9** | Ethics statement | Name the institutional research board / ethics committee and the approval number; restate written informed consent; state de-identification method. | Internal records | 0.1 d | Methods II.A | **REQUIRED** |
| **T1.10** | Funding & conflicts | Add funding source and the funder's role, or state that no specific funding was received; add a conflicts-of-interest statement for all three authors. | — | 0.1 d | End matter | **REQUIRED** |
| **T1.11** | Open science | Add: software versions actually used (scikit-learn `[[TO BE COMPUTED: version]]`, imbalanced-learn `[[TO BE COMPUTED: version]]`, Python `[[TO BE COMPUTED: version]]`, plus `shap`, `xgboost`, `dcurves` versions); data-availability statement with reasons if data cannot be shared; **analytic code DOI or repository link** (e.g. Zenodo/OSF/GitHub). TRIPOD+AI treats these as essential. | Zenodo or OSF for DOI; GitHub | 0.5 d | Methods II.E; end matter | **REQUIRED** |
| **T1.12** | Usability / implementation paragraph | Explain when in the pathway the model is used, what inputs are required, what to do when an input is missing or of poor quality, and the assumed user expertise (posyandu cadre, no statistical training). | — | 0.3 d | Discussion III.F; Conclusion | **REQUIRED** |
| **T1.13** | Honest limitations rewrite | Rewrite Limitations to state: (a) the matched case–control design fixes the case fraction at 33.3%, so predicted probabilities are **not population risks** and require prior correction; (b) channel confounding is now **quantified**, not merely admitted (reference T2.4–T2.6 results); (c) parenting subscale reliabilities α = 0.10 / −0.18 / 0.13 make parenting an exploratory pillar only; (d) the headline predictor rests on a single ordinal screen-time item; (e) 17 matched cases means differences below the fold-to-fold SD are not interpretable. | — | 0.5 d | Discussion III.F | **REQUIRED** |
| **T1.14** | Reviewer-response letter | Draft a point-by-point response mapping the rejection reason ("does not sufficiently demonstrate clear novelty or a significant contribution") to: gap table (T1.2), five falsifiable contributions (T1.3), the validity treatment (T2.4–T2.6), decision analysis (T2.2–T2.3), and the boundary result (T1.4). Internal draft only — do not send until the authors approve. | — | 1.0 d | Separate file | **REQUIRED** |

**Tier 1 total: ≈ 6.2 author-days.**

---

## Tier 2 — Re-run analyses on data you already hold

### Group A — Validity and confounding (highest novelty value)

| # | Item | What to do | Tool / library | Effort | Section changed | Req? |
|---|---|---|---|---|---|---|
| **T2.4** | **Channel-only baseline** | Fit the identical pipeline with recruitment channel as the *only* feature. Report its accuracy/F1/recall and the paired fold-wise ΔF1 versus the full model, with 95% CI. This converts the SES-inflation finding (accuracy 0.983, F1 0.870) from a caveat into a quantified bound on confounding. | `sklearn.ensemble.RandomForestClassifier`; `scipy.stats.ttest_rel`; same `RepeatedStratifiedKFold(n_splits=5, n_repeats=5)` | 0.5 d | Methods II.E (new §); Results (new §); new table | **REQUIRED-A** |
| **T2.5** | **Age + SES matching / balancing** | Extend matching from age alone (caliper ±6 months) to age + a socioeconomic anchor; additionally run propensity weighting on the full unmatched sample. Report standardised mean differences before/after with a Love plot and effective sample size. Report model performance on the balanced set versus 0.852/0.750/0.791. | `sklearn.neighbors.NearestNeighbors` or `pymatch`; propensities via `sklearn.linear_model.LogisticRegression`; SMD via `tableone` | 1.5 d | Methods II.A & II.E; Results; new table + figure | **REQUIRED-A** |
| **T2.6** | **Feature-importance invariance** | Recompute permutation importance (a) within the case arm, (b) within the control arm, (c) with channel-correlated features removed; compare rankings with Spearman's ρ and top-3 overlap. Also compare permutation vs SHAP rankings. **If screen time's dominance (0.223 ± 0.083) does not survive, the Abstract and Introduction must be softened — see T1.13 and T1.5.** | `sklearn.inspection.permutation_importance`; `scipy.stats.spearmanr` | 0.5 d | Results; new table + rank-stability figure | **REQUIRED-A** |

### Group B — Decision quality

| # | Item | What to do | Tool / library | Effort | Section changed | Req? |
|---|---|---|---|---|---|---|
| **T2.2** | **Calibration** | Report Brier score (bootstrap 95% CI), calibration slope and intercept, smoothed calibration curve with decile points and counts, O:E ratio, and low/medium/high risk strata with observed case rates. Apply **prior correction** to a stated target prevalence (see T2.14) because the matched design fixes the case fraction at 1/3. Report both the sampling-frame and prior-corrected scales. Do **not** use Hosmer–Lemeshow. | `sklearn.calibration.calibration_curve`, `sklearn.metrics.brier_score_loss`, `statsmodels.api.Logit` with slope free and with `offset` for slope-fixed | 1.0 d | Methods II.E; Results (new §); new table + figure | **REQUIRED-A** |
| **T2.3** | **Decision-curve / net-benefit analysis** | Compute net benefit over the threshold range on the prior-corrected scale, compared with **screen-everyone**, **screen-no-one**, and **KPSP-only**. KPSP-only has two variants: empirical (community arm only, n = 290; treat doubtful + possible deviation as screen-positive) and analytic (apply published KPSP sensitivity 60% / specificity 92% to the matched sample, because KPSP was not administered in the clinic arm). Report net benefit at the authors' stated cost ratio. | Python package `dcurves` (`dcurves.dca`), or R `dcurves` / `rmda`; bootstrap bands | 0.5 d | Methods II.E; Results (new §); new table + figure | **REQUIRED-A** |
| **T2.7** | **Ablation by feature cluster** | Nested sets: perinatal-only → +parenting → +stimulation → +screen time, plus a demographic-only reference set and leave-one-cluster-out sets. Same folds, same hyperparameters. Report ΔF1 with 95% CIs. This gives the "modifiable factors are actionable" thesis empirical teeth — or refutes it honestly. | Existing modelling script in a loop over column-list `dict`s; `sklearn.model_selection.cross_validate` | 0.5 d | Methods II.E; Results; new table + figure | **REQUIRED-A** |
| **T2.10** | **Confidence intervals on all performance metrics** | The manuscript reports mean ± SD across 25 folds but no CIs. Add bootstrap 95% CIs (2,000 resamples, pooled out-of-fold) for accuracy, precision, recall, F1 for Random Forest, logistic regression and decision tree, including the pooled matrix already published (16 TP / 7 FN / 2 FP / 32 TN; accuracy 0.842, sensitivity 0.696, specificity 0.941). | `sklearn.utils.resample` or `scipy.stats.bootstrap` | 0.5 d | Results III.B; Table 2 (v1) | **REQUIRED-A** |

### Group C — Interpretability and benchmarks

| # | Item | What to do | Tool / library | Effort | Section changed | Req? |
|---|---|---|---|---|---|---|
| **T2.8** | **SHAP global + per-child + worked vignette** | Compute TreeSHAP values on the prior-corrected probability scale for all 51 matched children: beeswarm summary, mean-\|SHAP\| ranking (compare against permutation ranking), and individual waterfall plots for (a) the child nearest the referral threshold and (b) a false negative. Populate the vignette template in the companion document §4.3 with **real, de-identified** values, and add the modifiable-feature → counselling-target mapping table drawn from the SDIDTK/KIA guidance. | `shap` (`shap.TreeExplainer`, `shap.summary_plot`, `shap.plots.waterfall`) | 1.0 d | Methods II.E; Results (new §); figures + tables | **REQUIRED-A** |
| **T2.9** | **XGBoost + calibrated logistic regression** | Add gradient boosting (XGBoost) with early stopping and tuned hyperparameters, and a probability-calibrated logistic regression. Compare on F1/recall, ROC-AUC, PR-AUC, Brier, calibration slope/intercept, net benefit at the prespecified threshold, and training time. Justify retaining Random Forest by the **interpretability–performance trade-off**, and state plainly when a difference is smaller than fold-to-fold variability rather than claiming superiority. | `xgboost.XGBClassifier` + `RandomizedSearchCV`; `sklearn.calibration.CalibratedClassifierCV` fitted inside training folds | 1.5 d | Methods II.E; Results; new table (+ trade-off figure or ranked table) | **REQUIRED** |
| **T2.11** | SET-UP REQUIRED BEFORE ANY PROBABILITY USE | Fix and document the target prevalence π₀ used for prior correction, with its source (national/regional SDIDTK programme figure or a cited community estimate). Every calibration, net-benefit, risk-stratum and SHAP-on-probability result depends on this single stated assumption. If no defensible figure exists, report all probability results on the sampling-frame scale only and say so explicitly — do not invent a prevalence. | Literature / programme data | 0.3 d | Methods II.E | **REQUIRED-A** |

### Group D — Reporting completeness (individual items; cheap)

| # | Item | What to do | Tool | Effort | Section changed | Req? |
|---|---|---|---|---|---|---|
| **T2.12** | Participant characteristics table | Build the missing sample-characteristics table: by arm (clinic cases n = 30, community controls n = 290) and for the matched set (17/34), giving age (community 30.8 ± 16.2 mo; clinic 62.8 ± 26.2 mo), sex (49.3% vs 70.0% boys), birth weight, prematurity, smoking exposure, screen time, stimulation score, parenting subscale scores, and **missingness per variable**. | pandas / `tableone` | 0.5 d | Results III.A; new table | **REQUIRED** |
| **T2.13** | Participant flow diagram | Draw the flow from 302 community responses → 290 controls and 33 clinic responses → 30 eligible → 17 matched (six cases > 65 months unmatched), with exclusion reasons. | Any diagram tool | 0.2 d | Results; new supplementary figure | **REQUIRED** |
| **T2.14** | Predictor dictionary | Full definitions for all 24 features: exact wording, response categories, coding, units, measurement time point, and source (questionnaire item / KIA book). Extends v1 Table 1, which is cluster-level only. | Questionnaire + spreadsheet | 1.0 d | Methods II.C; Supplementary table | **REQUIRED** |
| **T2.15** | Sample-size justification | State honestly how the study size arose and whether it is sufficient. Run the Riley et al. approach for a binary outcome with the event count (primary: 17 events) and candidate parameter count; report EPV and the intended shrinkage. If the calculation shows the dataset is undersized, report that plainly — TRIPOD+AI requires it and reviewers reward candour. | R `pmsampsize`, or the published equations | 0.3 d | Methods II.A | **REQUIRED** |
| **T2.16** | Model specification and reproducibility | Save and publish the fitted model object; publish the full training script; state versions and licences; include a minimal prediction example. TRIPOD+AI item 12g and the Open Science items require this for non-equation models. | `joblib`; repository + Zenodo DOI | 0.5 d | Methods II.E; end matter | **REQUIRED** |
| **T2.17** | Subgroup performance | Report performance by sex and by age band, and by recruitment channel. With 17 matched cases these subgroups will be very small — report them with wide-CI caveats rather than omitting the analysis, and do not over-interpret. | `sklearn.model_selection.cross_validate` with grouped splits | 0.5 d | Results; new table | **OPTIONAL** |
| **T2.18** | Prediction-error review | List the false negatives and false positives from the pooled confusion matrix (16 TP / 7 FN / 2 FP / 32 TN) with their feature profiles and a short clinical commentary; identify whether errors cluster in the older-case or high-age group. | pandas | 0.5 d | Results; new table | **OPTIONAL** |
| **T2.19** | Screen-time dose–response | Because the headline predictor is a single ordinal item, add a dose–response view: case rate (or predicted risk) across screen-time categories, plus a sensitivity run with screen time dichotomised at each cut point. Directly pre-empts the "fragile single item" attack. | pandas; re-run pipeline | 0.5 d | Results; new figure/table | **OPTIONAL** |
| **T2.20** | Parenting subscale diagnostics | Report item-total correlations and α-if-item-deleted for each parenting subscale (current α = 0.10 / −0.18 / 0.13), and state whether the instrument can be repaired on existing data or must be replaced (→ T3.2). | `pingouin` or `statsmodels` reliability functions | 0.3 d | Methods II.B; Supplementary | **OPTIONAL** |
| **T2.21** | PROBAST self-appraisal | Appraise your own study with PROBAST (participants, predictors, outcome, analysis domains) and report the result. Pre-empts a reviewer running PROBAST and finding it unreported. | PROBAST checklist | 0.3 d | Supplementary | **OPTIONAL** |
| **T2.22** | Stability check | Examine model stability (e.g. bootstrap of predictor selection and of individual predictions) rather than only point performance — TRIPOD+AI item 12c expects this. | bootstrap loop | 0.5 d | Methods II.E; Results | **OPTIONAL** |

**Tier 2 totals: ≈ 12.4 author-days for the 11 REQUIRED / REQUIRED-A items; ≈ 15.4 author-days including all OPTIONAL items.**

---

## Tier 3 — New data collection (defer; declare as future work)

| # | Item | What to do | Tool | Effort | Section changed | Req? |
|---|---|---|---|---|---|---|
| **T3.1** | Validated multi-item screen-exposure instrument | Replace the single ordinal screen-time item with a validated caregiver-report measure (total, content type, co-viewing), plus a prospective re-collection on a fresh sample. Removes the single biggest fragility in the headline finding. | Instrument selection + survey platform | 3–6 weeks fieldwork | Methods; future work | **OPTIONAL** |
| **T3.2** | Repair or replace the parenting instrument | Administer a validated PSDQ short form (or a locally validated parenting measure) with adequate item counts per subscale; current α values (0.10 / −0.18 / 0.13) do not support parenting as a headline pillar. | PSDQ short form; reliability analysis | 4–8 weeks | Methods; future work | **OPTIONAL** |
| **T3.3** | Anthropometric z-scores | Compute WHO z-scores and add them as a feature cluster, then re-run the ablation. *Note:* the manuscript states that same-day anthropometry was recorded in the **community arm only**; KPSP and anthropometry were not collected in the clinic arm. A community-arm-only z-score analysis is therefore achievable now as a Tier 2 sub-task (≈0.5 d), but using nutrition as a predictor of case status requires clinic-arm measurement, which is new data. | WHO Anthro / `anthro` R package | 2–4 weeks (0.5 d for the community-arm-only version) | Methods; future work | **OPTIONAL** |
| **T3.4** | External / prospective validation at a second site | The single strongest addition for a prediction-model paper, and the fix TRIPOD+AI expects for a model developed on one dataset. At minimum, pre-register the external validation protocol now so it can be cited in the resubmission limitations. | Multi-site design; TRIPOD+AI validation items | 6–12 months | Discussion; future work | **OPTIONAL (this cycle) / REQUIRED for the full contribution** |
| **T3.5** | Community prevalence estimate | Obtain a defensible estimate of the prevalence of clinically ascertained developmental delay in the target population; this is what makes absolute risk (and therefore deployment thresholds) interpretable. See T2.11. | Population survey or programme data | 3–6 months | Methods; Discussion | **OPTIONAL** |
| **T3.6** | Enlarge and diversify the case sample | 17 matched cases is thin for an ML paper; multi-site case recruitment would stabilise the wide fold-to-fold SDs (largest reported SD 0.257 on decision-tree recall; 0.198 on Random Forest recall) and permit subgroup performance claims. | Multi-site clinic recruitment | 6–12 months | Methods; future work | **OPTIONAL** |
| **T3.7** | Prospective registration / protocol | Register the protocol for the prospective phase (or state that the present study was not registered, which is acceptable retrospectively) to close the TRIPOD+AI open-science gap for the next study. | OSF / ClinicalTrials.gov | 0.2 d | Methods; future work | **OPTIONAL** |

---

## Priority order if time is short

1. **T1.1–T1.4** (retitle, gap table, contributions, promote the negative result) — the pure-reframe core. *1 author-day.*
2. **T2.11** (fix the prevalence assumption) — everything probability-based depends on it. *0.3 d.*
3. **T2.2 + T2.3** (calibration + decision curve) — converts the paper from a classifier into a deployable triage tool. *1.5 d.*
4. **T2.4 + T2.5 + T2.6** (channel baseline, SES balancing, invariance) — the paper's most distinctive methodological component and the direct answer to the rejection. *2.5 d.*
5. **T2.7** (ablation) *0.5 d*; **T2.9** (benchmarks) *1.5 d*; **T2.8** (SHAP + vignette) *1.0 d*.
6. **T1.5, T1.6, T1.13, T1.14** (abstract, TRIPOD+AI checklist, limitations, response letter) — written last, using the real numbers. *2.0 d.*
7. **T2.12–T2.16** (characteristics table, flow diagram, predictor dictionary, sample-size justification, code release) — reporting completeness. *2.5 d.*

**Total for the full Tier 1 + Tier 2-REQUIRED path: ≈ 14–16 author-days.**

---

## What this checklist deliberately does NOT do

- It does **not** promise that any added analysis will improve the results. Several additions (SES balancing, channel-feature removal, calibration) may *reduce* apparent performance. That is the point: a methodologically honest treatment is the novelty, and a reduced but credible number is worth more than an inflated one.
- It does **not** allow any Tier 2 item to be written up before the analysis has actually been run. Every `[[TO BE COMPUTED: …]]` in the companion document is a hard blocker on the sentence containing it.
- It does **not** commit the authors to keeping Random Forest. T2.9 may show XGBoost ahead by more than fold-to-fold variability; if so, the honest response is to switch models and keep the interpretability argument, not to defend the original choice.
