# Response to the Editor — Point-by-Point

**Manuscript:** Random Forest Classification for Early Detection of Developmental Delay Risk in Children Based on Parenting, Stimulation, and Perinatal Risk Factors
**Authors:** Dini Hari Pertiwi, Yarza Aprizal, Eko Setiawan — Institut Teknologi dan Bisnis Palcomtech, South Sumatra, Indonesia
**Journal:** Jurnal ELTIKOM — Jurnal Teknik Elektro, Teknologi Informasi dan Komputer
**Document type:** Response to editor / resubmission cover letter
**Status:** Internal draft — not for external submission until the outstanding analyses marked `[[TO BE COMPUTED: ...]]` have been run on the authors' data.

---

## 1. Editor's comment

> "The manuscript does not sufficiently demonstrate clear novelty or a significant contribution to the existing body of knowledge. We encourage you to refine your study by strengthening its originality, clearly highlighting its contribution."

## 2. Summary of the revision

We accept the assessment. The revised manuscript does not change the study's data or its results; it changes what the study claims to contribute, and it promotes two components that were previously reported as limitations into stated contributions that a reviewer can test.

The previous submission framed the work as the application of Random Forest to a new outcome label. Read against the Indonesian machine learning literature it cites for child health [4]–[8], that framing is accurate but weak: the outcome is new, the method and pipeline (scikit-learn, cross-validation, permutation importance, SMOTE) are not. Its three stated contributions — no label leakage, age-matching, and permutation importance — are correctness requirements rather than original contributions.

The revision therefore does four things:

1. **Repositions the paper as a clinical prediction model study**, reported following TRIPOD+AI conventions [[VERIFY CITE]], with a named clinical decision task (triage: which children are prioritised for full KPSP administration) instead of an algorithm comparison.
2. **Makes the recruitment-channel confound a contribution rather than a caveat.** The previous version disclosed that adding socioeconomic features inflated unmatched-sample performance to accuracy 0.983 and F1 0.870, and then excluded those features. That disclosure is retained and extended into an explicit validity component with a falsification condition.
3. **Adds a decision-analytic layer.** Discrimination alone does not establish that a model should be used. Calibration, decision-curve net benefit, and a cost-sensitive operating threshold are now part of the primary result, positioning the model against "screen everyone" and "KPSP-only" comparators.
4. **Elevates the negative secondary result into a claimed boundary.** The finding that the same questionnaire factors did not reproduce KPSP outcomes in the unselected community sample (recall of the at-risk class 0.15) previously appeared late and read as a weakness; it is now stated as a delimitation of what questionnaire-based risk models can replace.

Nothing in the revision inflates the study. The matched primary sample remains 17 cases and 34 controls; the parenting subscales remain unreliable (α = 0.10, −0.18, 0.13) and parenting is demoted to an exploratory predictor; the boundary result is reported as a boundary, not as a near-miss.

## 3. Point-by-point response

### Comment 1: "The manuscript does not sufficiently demonstrate clear novelty or a significant contribution."

**Response: accepted; the contribution is now stated as five falsifiable, gap-mapped claims in Section 1 (Introduction), and the Introduction gap statement has been converted from a sentence into an explicit table.**

*What changed.* The previous Introduction supported the novelty claim with one sentence — that the Indonesian literature "is heavily concentrated on stunting and nutritional status classification using anthropometric inputs" — and followed it with three design choices (no label leakage, age-matching, permutation importance) labelled as contributions. That is a reason to believe the study is correct; it is not a statement of what the study adds.

The revision replaces this with Table G ("What prior work addressed and what remains unaddressed"), with one row per item in the manuscript's own reference list [1]–[18] plus the methodological literature a prediction-model paper is expected to engage with. Each row names a specific prior work, what it established, and the specific thing it leaves unaddressed. Every row maps to one of the five contributions below, so a reviewer can verify the mapping rather than accept a general assertion of novelty. Rows are included for Walker et al. 2011 [1]; WHO/UNICEF/World Bank 2018 [10]; Kementerian Kesehatan RI 2016 (SDIDTK) [2]; Dhamayanti 2006 (KPSP) [3]; Madigan et al. 2019 [11]; Chonchaiya & Pruksananonda 2008 [12]; Candra et al. 2024 [4]; Dhani et al. 2024 [5]; Syahfitri et al. 2024 [6]; Lestari 2024 [7]; Ratnasari et al. 2024 [8]; Breiman 2001 [9]; Robinson et al. 1995 [13] and Baumrind 1971 [14]; Kariger et al. 2012 [15]; the KIA handbook [16]; WHO 2006 growth standards [17]; Chawla et al. 2002 [18]; plus PROBAST [[VERIFY CITE]], TRIPOD/TRIPOD+AI [[VERIFY CITE]], decision curve analysis (Vickers & Elkin, 2006) [[VERIFY CITE]], and SHAP (Lundberg & Lee, 2017) [[VERIFY CITE]].

The five contributions, each with a falsification condition, are:

| # | Contribution (as now stated in Section 1) | Gap it fills | What would falsify it |
|---|---|---|---|
| C1 | To our knowledge, the first prediction model for early-childhood developmental delay in Indonesia built exclusively from caregiving, perinatal, and demographic risk factors, using no developmental test items, delivered as an interpretable triage aid | Refs [4]–[8]: Indonesian ML confined to stunting/nutrition from anthropometry; [3]: KPSP validated but with no complementary risk model; [2]: manual, capacity-limited national screening | Any published developmental-delay risk model for Indonesian children using exclusively non-test questionnaire factors |
| C2 | Age-matched case–control design plus explicit *quantification and mitigation* of recruitment-channel confounding (channel-only baseline; SES matching/residualisation; channel-invariance check on feature-importance ordering) | No row in Table G quantifies sample-construction confounding; [18] and [4]–[8] address class balance and accuracy, not channel confounding | A channel-only baseline matching the full model's performance, or a SES-matched model collapsing to chance |
| C3 | Calibration, decision-curve net benefit, and a cost-sensitive threshold, specifying the model's place relative to KPSP in a triage pathway | Accuracy-only reporting in [4]–[8]; decision-curve methodology absent from the domain per Table G | Net benefit no greater than screen-all or KPSP-only across the plausible threshold range |
| C4 | Negative secondary result formally delimiting what questionnaire-based risk models cannot replace: the same factors do not reproduce KPSP outcomes in the unselected community (recall of at-risk class 0.15) | Never reported in the cited literature; [11], [12] establish association, not the limits of representation-based prediction | Re-analysis showing the factors do reproduce KPSP outcomes at operationally useful recall |
| C5 | SHAP-based per-child explanations yielding actionable counselling targets for frontline workers, supplementing global permutation importance | [4]–[8] report at most global importances; SHAP not yet applied to per-child counselling targets in this domain | SHAP attributions unstable across folds, or contradicting the permutation-importance ranking |

Two points on the honesty of these claims. C1 is stated as scoped and hedged ("to our knowledge", "in Indonesia", "exclusively caregiving/perinatal factors"), not as an absolute priority claim. C2, C3, and C5 are stated as procedures with stated falsification conditions rather than as established results, because the corresponding analyses are not yet reported in `manuscript-v1.txt`. We do not claim performance advantages; the matched sample of 17 cases does not support them.

**Manuscript sections changed:** Abstract; Section 1 (Introduction) — full rewrite including Table G; Section IV (Conclusion).

### Comment 1 (continued): Changes to the Methods required for the above claims

**Response: the analyses underlying C2, C3, and C5 are specified in Section 2 (II. Research Methods) and will be reported in Section 3 (III. Results and Discussion). Results that are not yet computed are marked in this letter; they will be filled from the authors' existing data before submission.**

*C2 — channel-confounding analysis (Section 2.E).* Three additions. (i) A **channel-only baseline** trained on recruitment-channel indicators alone, reported on the same matched set with the same cross-validation scheme, to establish how much of the Table 2 performance is attributable to channel rather than outcome. (ii) **Socioeconomic matching** in addition to age, or residualisation of the socioeconomic features, with the resulting matched-set size and metrics reported; if SES matching reduces the matched case count below the current 17, that is reported as such rather than worked around. (iii) A **channel-invariance check**: permutation importance recomputed within recruitment channel, and under channel hold-out, and compared with the pooled ranking in Table 3. Required quantities:
`[[TO BE COMPUTED: channel-only baseline performance on the matched set; SES-matched matched-set size and primary metrics; within-channel permutation-importance ranking versus pooled Table 3 ranking]]`

*C3 — calibration and net benefit (Section 2.E, Section 3.B).* Calibration slope and intercept, Brier score, calibration curve, and decision-curve net benefit against two comparators (screen all children; KPSP-only), with the operating threshold selected on a cost-sensitive basis and justified for the screening context where a false negative is more costly than a false positive. Required quantities:
`[[TO BE COMPUTED: calibration slope and intercept, Brier score, calibration curve on the held-out set; net benefit of the model versus screen-all and KPSP-only across a pre-specified threshold range; the selected operating threshold and its sensitivity/specificity trade-off]]`
The existing pooled out-of-fold confusion matrix (16 true positives, 7 false negatives, 2 false positives, 32 true negatives; accuracy 0.842, sensitivity 0.696, specificity 0.941) is retained and will be reported alongside the net-benefit curve. With 17 matched cases these estimates will be imprecise; confidence intervals will be reported and any conclusion drawn from them will be correspondingly limited.

*C5 — per-child explanations (Section 2.E, Section 3.C).* SHAP values for a worked flagged-child vignette, showing which factors drove that child's score and which counselling target follows, together with an agreement check against the permutation-importance ranking. Required quantities:
`[[TO BE COMPUTED: SHAP values for a worked flagged-child vignette; agreement between SHAP rank ordering and permutation-importance rank ordering across folds]]`

*Benchmark addition.* Gradient boosting and calibrated logistic regression will be added to the Table 2 comparison, and the choice of Random Forest will be justified on the interpretability–performance trade-off rather than on the margin of accuracy.

**Manuscript sections changed:** Section 2.E (Model Development and Evaluation) — added procedures; Section 3.B, 3.C (Results and Discussion) — added result targets.

### Comment 1 (continued): Changes to the Results and Discussion

**Response: the two components previously reported as limitations are now reported as results, with their implications stated in place.**

*Recruitment-channel confounding.* Section 3.D previously ended by noting that adding socioeconomic features raised performance to accuracy 0.983 and F1 0.870, and that these features were excluded from the primary model. This disclosure is retained verbatim in substance and extended: the channel-only baseline and SES-matched results are reported next to the primary result, and the discussion states explicitly that part of the case–control contrast tracks recruitment channel. The paragraph no longer reads as a disclaimer for a finding of doubtful validity; it reads as a bounded estimate with a stated falsification condition.

*Negative secondary result.* Section 3.E previously reported recall of the at-risk class of 0.15 for KPSP-outcome prediction in the community sample, together with the finding that only birth weight differed between screening groups (median 2700 g versus 3000 g; p = 0.001) while parenting, stimulation, and screen time did not. This is now presented as a stated boundary of the method class: questionnaire-based risk models do not replace developmental screening in unselected populations, and the model's defensible role is prioritisation for full KPSP administration. The Discussion no longer treats this as an anomaly; it treats it as the paper's most distinctive scientific contribution alongside C2 and C3.

*Parenting.* Consistent with the reported reliabilities (α = 0.10, −0.18, 0.13 for authoritative, authoritarian, and permissive; α = 0.789 for the ten-item stimulation scale), parenting is explicitly demoted from a headline element to an exploratory predictor. The title, abstract, and contributions no longer lean on the parenting pillar. If the reviewers prefer a title that foregrounds the decision framework, "From Screening to Triage: An Interpretable Random Forest Model of Modifiable Caregiving and Perinatal Risk Factors for Early Detection of Developmental Delay in Indonesia" is proposed; a validity-forward alternative is provided in the framing document.

*Limitations.* Section 3.F is retained in full, including the small case sample (30 eligible, 17 age-matched), single-city recruitment, retrospective framing of parenting and stimulation items in the clinic arm, and low parenting-subscale reliability. No limitation has been removed to strengthen the contribution claim.

**Manuscript sections changed:** Section 3.B, 3.C, 3.D, 3.E, 3.F; Abstract.

### Comment 1 (continued): Change to the Conclusion

**Response: Section IV now states what the study contributes rather than restating the metrics.**

The previous conclusion reported the performance figures, named the top predictors, described the secondary analysis as showing that the factors "do not replace formal developmental screening", and listed future work. The revised conclusion states the triage framing, retains all performance figures unchanged, and then states the three qualifications as part of the contribution: that recruitment-channel confounding is quantified and bounded; that calibration and net benefit are required before deployment and are reported as primary results; and that the negative secondary result delimits the method's applicable scope. It also states plainly that the parenting subscales require psychometric repair before parenting can enter any headline claim.

**Manuscript sections changed:** Section IV (Conclusion).

## 4. Change log

| Manuscript section | Change | Addresses |
|---|---|---|
| Title | Two options proposed: triage-framed (lead) and validity-forward | Novelty framing |
| Abstract | Rewritten: triage framing, TRIPOD+AI conventions, channel confounding and calibration stated as contributions, placeholders for not-yet-computed quantities | Novelty / contribution |
| Section 1 Introduction | Rewritten; one-sentence gap statement replaced by Table G with one row per prior work [1]–[18] plus methodological literature; every row mapped to a contribution | "does not sufficiently demonstrate clear novelty" |
| Section 1 Contributions | Five falsifiable, gap-mapped claims (C1–C5), each with a falsification condition; parenting demoted; no performance-based claims | "does not sufficiently demonstrate … a significant contribution" |
| Section 2.E Methods | Channel-only baseline; SES matching/residualisation; channel-invariance check; calibration and decision-curve analysis; cost-sensitive threshold; SHAP per-child explanations; gradient boosting and calibrated logistic regression added to the benchmark | Methodological contribution |
| Section 3.B–3.D Results | Channel-confounding results reported next to the primary result rather than as a caveat; net-benefit and calibration results to be reported | Contribution, internal validity |
| Section 3.E Results | Negative secondary result elevated and framed as the paper's boundary contribution | "clearly highlighting its contribution" |
| Section 3.F Limitations | Retained in full, unsoftened | Integrity |
| Section IV Conclusion | Rewritten to state the contribution and its three qualifications, with all metrics unchanged | Contribution |

## 5. What is not yet in the manuscript

To avoid any appearance of claiming analyses that have not been performed, the following are marked in this revision as pending and will be computed from the authors' existing data before resubmission:

- Channel-only baseline performance on the matched set.
- Socioeconomic-matched matched-set size and primary metrics (and, if the matched case count falls below 17, that outcome).
- Within-channel and channel-hold-out permutation-importance rankings versus the pooled Table 3 ranking.
- Calibration slope, calibration intercept, Brier score, and calibration curve.
- Decision-curve net benefit versus screen-all and KPSP-only at the pre-specified threshold range, and the selected operating threshold.
- SHAP values for the worked flagged-child vignette, and SHAP-versus-permutation rank agreement.
- Gradient boosting and calibrated logistic regression results for the Table 2 comparison.

No result, metric, or p-value in the revised manuscript has been altered from the submitted version, and no figure has been added that is not present in the submitted analysis.

## 6. Bibliographic note

New references introduced in the revision are cited with verified bibliographic details:

- **TRIPOD+AI statement:** G. S. Collins et al., "TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods," *BMJ*, vol. 385, p. e078378, 2024. doi: 10.1136/bmj-2023-078378.
- **PROBAST:** R. F. Wolff et al., "PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies," *Annals of Internal Medicine*, vol. 170, no. 1, pp. 51–58, 2019. doi: 10.7326/M18-1376.
- **Decision curve analysis:** A. J. Vickers and E. B. Elkin, "Decision Curve Analysis: A Novel Method for Evaluating Prediction Models," *Medical Decision Making*, vol. 26, no. 6, pp. 565–574, 2006. doi: 10.1177/0272989X06295361.
- **SHAP:** S. M. Lundberg and S.-I. Lee, "A Unified Approach to Interpreting Model Predictions," in *Advances in Neural Information Processing Systems (NeurIPS 2017)*, vol. 30, pp. 4765–4774, 2017.
- **Calibration:** B. Van Calster, D. J. McLernon, M. van Smeden, L. Wynants, and E. W. Steyerberg, "Calibration: the Achilles heel of predictive analytics," *BMC Medicine*, vol. 17, no. 1, p. 230, 2019. doi: 10.1186/s12916-019-1466-7.

No DOI, volume, issue, or page number has been invented anywhere in this revision. All existing references [1]–[18] are reused as they stand in the submitted manuscript.

---

*Draft prepared for internal review. Not for external submission until the outstanding analyses in Section 5 have been computed and the `[[VERIFY CITE]]` entries resolved.*
