Response to Editor and Reviewers: Resubmission Re-framing Letter
Date: October 16, 2026
To: Editor-in-Chief and Editorial Board, Jurnal ELTIKOM: Jurnal Teknik Elektro, Teknologi Informasi dan Komputer
From: Dini Hari Pertiwi, Yarza Aprizal, Eko Setiawan
Institution: Institut Teknologi dan Bisnis Palcomtech, Palembang, Indonesia
Manuscript Title (Revised): Recruitment-Channel Confounding Outweighs Risk Factors in Machine-Learning Case–Control Models of Developmental Delay: A Validity Study
(Alternative Title: When Case–Control Machine Learning Learns the Recruitment Channel: A Validity Study of Developmental-Delay Risk Prediction in Indonesia)
Our Original Submission Title: Random Forest Classification for Early Detection of Developmental Delay Risk in Early Childhood
Dear Editor-in-Chief and Editorial Board of Jurnal ELTIKOM,
We wish to express our sincere gratitude to the Editorial Board and the anonymous reviewers for evaluating our previous submission. The initial decision to reject our manuscript was grounded in the assessment that the work lacked sufficient novelty and presented an incremental application of standard classification algorithms without a decisive technical or methodological contribution.
We took this evaluation seriously. Rather than performing minor cosmetic revisions or attempting to tune hyperparameters on the same data, our research team undertook an exhaustive, independent code audit and algorithmic reproduction of our entire data pipeline (clean.py → pipeline.py → matched_final.py).
This audit led to a profound scientific finding that prompted a complete, principled pivot: we have reframed the manuscript from an unverified "screening/triage tool" into a rigorous methodological validity and reproducibility study.
Below, we detail why this pivot fundamentally transforms the paper, how it directly resolves the novelty concern, and the specific empirical discoveries now documented in the revised manuscript.
1. Why the Pivot Resolves the Novelty and Contribution Deficiency
In the regional biomedical informatics literature in Indonesia, hundreds of articles report machine learning classifiers applied to small observational clinical datasets, almost universally claiming high accuracy (80%–95%) and advocating for automated screening tools. However, such studies routinely suffer from three pervasive flaws: 1. They overlook severe recruitment-channel confounding inherent in two-stream recruitment (e.g., hospital/clinic cases versus community/posyandu controls); 2. They do not release or audit code, leaving subtle preprocessing and cohort selection errors undetected; and 3. They fail to conduct formal calibration adjustments or decision curve analyses against existing routine healthcare standards.
Our revised manuscript offers a rare, bit-for-bit reproducible validity study that directly addresses these systemic vulnerabilities. Rather than claiming our model is ready for clinical deployment, we demonstrate empirically—using our own audited data and code—that two-stream case–control models predominantly learn recruitment-channel disparities rather than clinical pathology. This genre of critical validity study is virtually non-existent in the Indonesian health informatics literature and provides substantial educational and methodological value to the readers of Jurnal ELTIKOM.
2. Summary of Key Audit Discoveries and New Empirical Evidence
The revised manuscript presents five substantive contributions that were entirely absent from the initial submission:
A. Exact Bit-for-Bit Reproduction and Identification of Bug #2 (Cohort Leakage)
We successfully reproduced the classification performance reported in Table 2 of our original manuscript to three decimal places (Run A: Random Forest accuracy 0.852 ± 0.082, precision 0.890, recall 0.750, F1 0.791; out-of-fold matrix: TP 16, FN 7, FP 2, TN 32). This reproduction confirmed that Table 2 was computed from an uncorrected 57-row dataset containing 23 cases and 34 controls.
Our code audit revealed that in matched_final.py (lines 34–48), 6 older clinical cases (aged 66–72 months) that could not be matched to community controls (due to an age cutoff of 59 months in Posyandu controls) were correctly logged as unmatched, but were accidentally retained during dataframe concatenation (cases.assign(case=1)).
When the pipeline is corrected to the true 1:2 age-matched cohort (Run B, N = 51; 17 cases and 34 controls), model performance declines across all metrics: - Random Forest F1 drops from 0.791 to 0.695 ± 0.173. - Recall drops from 0.750 to 0.647. - Random Forest no longer dominates across all metrics: Logistic Regression achieves a higher sensitivity/recall (0.667 vs. 0.647).
B. Discovery and Rectification of Bug #1 (Lexicographical Column Sorting)
In our original manuscript, we reported that parental style subscales were psychometrically unreliable (Cronbach's α = +0.100 for authoritative, −0.178 for authoritarian, and +0.132 for permissive) and demoted parenting style to an exploratory variable.
Our audit uncovered that in clean.py (line 51), the helper function executed sorted(cols), which sorted survey question labels alphabetically based on raw question string text rather than survey index order. This scrambled the 9 questionnaire items across distinct constructs.
When the correct survey grouping is restored, Cronbach's alpha values for authoritative and permissive styles immediately recover to acceptable reliability (+0.567 and +0.591, respectively; authoritarian remains low at +0.160). We show how a single line of sorting code can inadvertently invalidate a legitimate clinical domain.
C. Empirical Proof of Recruitment-Channel Confounding
To test whether our model learned true developmental delay risk or institutional recruitment stream, we constructed a negative control channel-only model using strictly four parental socio-economic status (SES) variables: maternal education, paternal education, family income, and maternal employment status.
The results provide definitive evidence of channel confounding: - 4-variable SES Baseline F1: 0.876 ± 0.126 - 23-feature Clinical Risk Model F1: 0.695 ± 0.173 - Paired difference: ΔF1 = −0.181, 95% CI: [−0.500, 0.176], p = 0.00014
The four socio-economic status features significantly outperformed the full 23-feature clinical risk-factor model. This proves that the predictive signal in two-stream recruitment models is dominated by socio-economic differences between clinic families and community families, not neurodevelopmental pathology.
D. Calibration, Benchmark Models, and Population Re-anchoring
We benchmarked the Random Forest model against XGBoost and Calibrated Logistic Regression. Random Forest manifested severe probability miscalibration (calibration slope 2.656, Brier score 0.144), whereas XGBoost was substantially better calibrated (slope 0.801, Brier 0.136). We adjusted the model for community prevalence by re-anchoring from the artificial sampling prevalence (π_s = 0.3333) to the empirical Posyandu at-risk prevalence (π₀ = 0.0793), demonstrating monotonic risk stratification across low (0.059), medium (0.176), and high (0.765) risk strata.
E. Decision Curve Analysis (DCA) and Clinical Utility Limits
Rather than claiming that our model is universally effective, our decision curve analysis demonstrates clear operational boundaries: - At low decision thresholds (0.05–0.15), the risk model offers positive net benefit (0.311 to 0.225) compared to treat-all and treat-none strategies. - However, at decision thresholds $p_t \ge 0.20$, the model is strictly inferior to standard analytic developmental screening (KPSP, net benefit 0.187 vs. model 0.132 at $p_t = 0.20$).
3. Overview of Resubmission Package
In accordance with our new framing, the complete manuscript has been overhauled:
- 01-framing.md: New title options, concise abstract (~230 words), completely rewritten Introduction featuring an explicit gap table contrasting regional literature, original submission, and current validity study, structured around four falsifiable claims.
- 02-methods-results.md: Comprehensive reporting following TRIPOD+AI and PROBAST standards, detailing data audits, exact code audit procedures, full reproduction tables (Table 1 through Table 6), and specifications for Decision Curve (Figure 1) and SHAP feature attribution (Figure 2).
- 03-discussion.md: In-depth discussion of the anatomy of both programming bugs, the systemic problem of recruitment channel confounding in LMICs, critical lessons for Indonesian medical informatics, and candid limitations.
- 05-author-todo.md: An operational checklist in Indonesian outlining our commitments regarding population prevalence justification, raw data cleaning, code repository publication, and ethical governance.
We believe this transparent, rigorous, and self-correcting validity study provides an exemplary contribution to Jurnal ELTIKOM, offering valuable cautionary lessons for biomedical data scientists across Indonesia. We look forward to the editorial decision.
Sincerely,
Dini Hari Pertiwi, Yarza Aprizal, Eko Setiawan
Institut Teknologi dan Bisnis Palcomtech, Palembang, Indonesia