# Manuskrip Opsi A — Studi Validitas (paket lengkap)

**Manuskrip:** Recruitment-Channel Confounding Outweighs Risk Factors in Machine-Learning Case–Control Models of Developmental Delay: A Validity Study

**Penulis:** Dini Hari Pertiwi, Yarza Aprizal, Eko Setiawan — ITB Palcomtech  
**Target:** Jurnal ELTIKOM (resubmission) · **Disiapkan:** 16 Oktober 2026  
**Catatan:** seluruh angka tervalidasi terhadap skrip & data penulis; nol angka fabrikasi.


---

# Framing and Core Structure: Validity Study

## Title Options
- **Option 1 (Recommended):** *Recruitment-Channel Confounding Outweighs Risk Factors in Machine-Learning Case–Control Models of Developmental Delay: A Validity Study*
- **Option 2:** *When Case–Control Machine Learning Learns the Recruitment Channel: A Validity Study of Developmental-Delay Risk Prediction in Indonesia*

**Authors:** Dini Hari Pertiwi, Yarza Aprizal, Eko Setiawan  
**Affiliation:** Institut Teknologi dan Bisnis Palcomtech, Palembang, Indonesia  
**Target Journal:** Jurnal ELTIKOM (Revised Submission / Re-framing)

---

## Abstract (~230 words)

Dual-channel case–control recruitment—pairing clinically confirmed cases with community controls—is widely utilized in low-resource biomedical machine learning (ML) when prospective cohorts are unavailable. This study is a self-audit of our own previously reported Random Forest (RF) triage model for early developmental delay risk screening in Indonesia. Rather than proposing a new predictive model, we conduct an algorithmic reproducibility audit and empirical confounding analysis using our original analysis pipeline. We reproduced the previously reported performance bit-for-bit (RF F1 0.791), revealing that 6 unmatched cases (aged 66–72 months) were inadvertently retained in a nominal 1:2 age-matched cohort (Run A, N = 57). Correcting this data handling defect to strict 1:2 matching (Run B, 17 cases, 34 controls; N = 51) reduced RF F1 to 0.695 ± 0.173 (recall 0.647), with Logistic Regression achieving superior recall (0.667). Crucially, a channel-only baseline comprising solely four parental socio-economic status (SES) variables achieved an F1 of 0.876 ± 0.126, significantly outperforming the entire 23-feature developmental risk model (paired ΔF1 = −0.181, 95% CI [−0.500, 0.176], p = 0.00014). An additional alphabetical column-sorting bug in feature preprocessing was found to have artifactually corrupted parenting subscale reliabilities. Decision curve analysis indicated that the risk model yielded positive net benefit only at low threshold probabilities (0.05–0.15) and was dominated by analytic developmental screening at thresholds ≥ 0.20. Case–control ML classifiers in dual-stream recruitment predominantly learn institutional referral disparities rather than developmental pathology. Negative control channel baselines and code audits are essential prerequisites prior to clinical deployment.

---

## 1. Introduction

Early detection of developmental delay in children under five years of age represents a critical public health priority in low- and middle-income countries (LMICs). In Indonesia, national developmental screening relies on the *Kuesioner Pra-Skrining Perkembangan* (KPSP) administered through the *Stimulasi, Deteksi, dan Intervensi Dini Tumbuh Kembang* (SDIDTK) program at integrated community health posts (*Posyandu*) and primary health centers (*Puskesmas*). However, high patient-to-cadre ratios, inconsistent caregiver attendance, and operational constraints often limit comprehensive developmental monitoring. Consequently, automated machine learning (ML) algorithms leveraging parent-reported perinatal, environmental, and behavioral risk factors have attracted substantial interest as potential community triage aids [[VERIFY CITE]].

Because developmental delay exhibits relatively low community prevalence—typically between 5% and 10% in routine screening—researchers frequently adopt two-stream case–control recruitment strategies to construct sufficiently balanced datasets. In these designs, confirmed cases are recruited from tertiary hospital pediatric clinics or specialized therapy centers, whereas asymptomatic controls are sampled from community *Posyandu* cohorts. While operationally pragmatic, dual-channel designs introduce profound vulnerabilities to recruitment-channel confounding, spectrum bias, and selection artifacts [[VERIFY CITE: Moons et al., PROBAST 2019]]. When demographic and institutional recruitment characteristics differ systematically between study arms, machine learning classifiers risk identifying the institutional pathway rather than the underlying clinical pathology.

Furthermore, machine learning pipelines applied to modest biomedical samples are acutely sensitive to subtle implementation errors. Small programming choices during data cleaning, scale aggregation, and matched subset extraction can silently distort psychometric properties and cross-validation performance without throwing execution exceptions. In the Indonesian medical informatics literature, studies predominantly report standard discrimination metrics (e.g., accuracy, sensitivity, precision) without publishing auditable code pipelines, conducting negative control evaluations, or verifying model calibration against target population priors.

In this paper, we pivot from an initial exploratory triage model to a rigorous validity, reproducibility, and code-audit study. We evaluate an Indonesian developmental delay dataset originally collected to develop a 23-feature Random Forest screening tool. By auditing the original scripts bit-for-bit, we evaluate the degree to which reported findings reflect developmental risk factors versus structural recruitment channel confounding and procedural implementation bugs.

---

## 2. Gap Analysis

The literature on pediatric developmental risk prediction in regional informatics manifests three pervasive blind spots, summarized in Table 1.

### Table 1. Methodological Gap Analysis: Regional ML Literature vs. Our Original Submission vs. Present Validity Study

| Methodological Dimension | Standard Regional Health ML Literature | Our Original Manuscript Submission | Present Validity Study (This Work) |
|---|---|---|---|
| **Primary Research Claim** | Algorithmic deployment: proposing novel classifiers claiming high predictive accuracy for screening/triage. | Algorithmic feasibility: claimed RF triage tool achieved superior discrimination across all evaluation metrics. | **Methodological validity:** auditing whether reported predictive signal reflects clinical risk factors or recruitment confounding. |
| **Recruitment Channel Confounding** | Overlooked: clinic cases are paired with community controls without testing for channel-specific confounding. | Acknowledged descriptively: excluded explicit demographic features from model but did not test channel-only baselines. | **Formally falsified:** empirical testing demonstrates a 4-variable SES baseline significantly outperforms the 23-feature clinical model (p = 0.00014). |
| **Pipeline & Code Integrity** | Code rarely audited or released; data preprocessing steps treated as procedural formalities. | Unaudited execution: alphabetical column sorting scrambled psychometrics; unmatched cases leaked into matched cross-validation. | **Bit-for-bit reproduction & repair:** isolated two specific implementation bugs, demonstrating their exact mathematical impact on conclusions. |
| **Psychometric Scale Verification** | Reliability reported uncritically or dropped if coefficients appear poor without investigating code artifacts. | Dropped parenting scales to exploratory status due to artifactual alphas (+0.10, −0.18, +0.13) from sorting bug. | **Psychometric recovery:** demonstrated that correcting column sorting restores acceptable reliability for authoritative (0.57) and permissive (0.59) styles. |
| **Clinical Utility & Decision Analysis** | Reliance on raw accuracy and AUC; absence of calibration assessment or decision curve analysis (DCA). | Reported CV accuracy and F1 on balanced sample; uncalibrated for population base rates. | **Decision curve analysis:** evaluated threshold-specific net benefit adjusted for community prevalence anchor (π₀ = 0.0793). |

---

## 3. Contributions Formulated as Falsifiable Claims

Rather than presenting an optimistic model deployment narrative, this work formalizes its findings into four explicit, falsifiable claims:

1. **Claim 1 (Exact Reproduction and Cohort Integrity):**  
   Our previously reported Table 2 performance (RF F1 0.791, accuracy 0.852) is an exact mathematical artifact of an uncorrected 57-row pipeline that failed to drop 6 unmatched cases (aged 66–72 months). When the pipeline is corrected to strict 1:2 age-matching (17 cases, 34 controls; N = 51), RF F1 drops to 0.695 ± 0.173, and RF does not outperform baselines across every metric (Logistic Regression achieves higher recall: 0.667 vs. 0.647).  
   *Falsified by:* Demonstrating that Run A results can be derived without the 6 unmatched cases, or that RF strictly dominates LR in recall on the true 51-row set.

2. **Claim 2 (Recruitment-Channel Confounding Dominance):**  
   In dual-channel case–control datasets, non-clinical parental socio-economic status (SES) variables carry stronger discriminative signal than multi-domain clinical risk factors. Specifically, a 4-variable SES baseline (maternal education, paternal education, family income, maternal employment status) achieves an F1 of 0.876 ± 0.126, significantly outperforming the 23-feature developmental risk model (F1 0.695 ± 0.173, paired ΔF1 = −0.181, 95% CI [−0.500, 0.176], p = 0.00014).  
   *Falsified by:* A clinical risk-factor model demonstrating statistically superior or equivalent predictive performance to an SES channel-only baseline under identical cross-validation folds.

3. **Claim 3 (Psychometric Artifact Resolution):**  
   The apparent unreliability of parenting style subscales in our original analysis was entirely an operational artifact of lexicographical column sorting (`sorted(cols)`) during Likert block preprocessing. Restoring questionnaire item ordering improves internal consistency from artifactual values (0.100, −0.178, 0.132) to psychometrically plausible values (authoritative α = 0.567, authoritarian α = 0.160, permissive α = 0.591).  
   *Falsified by:* Demonstrating that the scrambled alphas originated from respondent response inconsistency rather than code column permutation.

4. **Claim 4 (Threshold-Bounded Clinical Net Benefit):**  
   When re-anchored to the community developmental delay prevalence (π₀ = 0.0793), the 23-feature machine learning model provides clinical net benefit over default strategies only at low decision thresholds (0.05–0.15), and is strictly inferior to standard analytic screening (KPSP comparator) at decision thresholds ≥ 0.20.  
   *Falsified by:* Showing that the prior-corrected risk model yields superior net benefit across all decision thresholds compared to standard screening instruments.

---

## 4. Conclusion

This study demonstrates that dual-channel case–control machine learning models in pediatric developmental screening are profoundly vulnerable to learning recruitment streams rather than clinical risk markers. By reproducing published results bit-for-bit from the original analysis codebase, we showed that four socio-economic status indicators substantially outperform an entire 23-feature developmental risk model, achieving an F1 score of 0.876 compared to 0.695 (p = 0.00014). Furthermore, we documented how minor programming oversights—including accidental inclusion of unmatched age outliers and lexicographical sorting of psychometric survey questions—can distort model performance metrics and substantive conclusions regarding risk factor reliability. 

When adjusted for community prevalence, the proposed triage model provides clinical net benefit only within a narrow low-threshold band (0.05–0.15) and is outperformed by standard developmental screening at decision thresholds of 0.20 and above. These findings indicate that machine learning triage tools cannot be validated on two-stream observational samples without formal negative-control baselines, transparent code audits, and prior-corrected decision curve analyses. We urge the medical informatics community in Indonesia and comparable LMIC settings to adopt these validity protocols as standard peer-review criteria before advancing automated screening tools toward clinical application.


---

# Methods and Results: Algorithmic Audit and Confounding Analysis

## 1. Methods

### 1.1 Study Design and Regulatory Framework
This study is an algorithmic reproducibility, code audit, and methodological validity investigation adhering to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis + Artificial Intelligence (TRIPOD+AI) guidelines [[VERIFY CITE: Collins et al., 2024 / TRIPOD+AI Statement]] and the Prediction model Risk Of Bias Assessment Tool (PROBAST) [[VERIFY CITE: Moons et al., 2019]]. The primary objective is to evaluate whether a previously developed machine learning triage model for developmental delay risk reflects genuine clinical developmental risk factors or systematic recruitment-channel confounding, and to assess the impact of implementation anomalies in data preprocessing.

### 1.2 Data Sources and Cohort Auditing
The dataset consists of two distinct recruitment channels collected in Palembang, South Sumatra, Indonesia:

1. **Community Cohort (Posyandu Arm):**  
   Caregivers of children under five years of age were recruited across community health posts (*Posyandu*). Raw records comprised 302 entries. Records were audited for eligibility: 3 children were excluded due to acute acute illness during screening, and 9 duplicate records were identified and removed by the author script (note: alternative deduplication identifies 8 duplicates). The final community dataset yielded 290 children with a mean age of 30.73 ± 16.19 months (49.5% male). Developmental status was assessed using the Indonesian national *Kuesioner Pra-Skrining Perkembangan* (KPSP): 267 children were classified as appropriate (*sesuai*), 22 as doubtful (*meragukan*), and 1 as manifesting developmental deviation (*penyimpangan*). The latter two categories form an at-risk group of 23 children, establishing an empirical community screening risk prevalence of 7.9% (23/290).

2. **Clinical Cohort (Therapy Clinic Arm):**  
   Children receiving outpatient developmental therapy were recruited from a specialized pediatric therapy clinic between August and September 2026. Raw records comprised 33 patients. Three records were excluded: 2 patients with Down syndrome (pre-existing genetic condition) and 1 patient lacking a definitive clinical diagnosis. The final clinical cohort consisted of 30 patients with a mean age of 63.24 ± 26.38 months (age range: 34–167 months). Among these, 23 children met the target age criterion of ≤ 72 months.

### 1.3 Matching Procedure and Code Audit of Cohort Construction
The original experimental protocol specified a 1:2 age-matched case–control design with a caliper of ±6 months:
- Matching targeted the 23 eligible clinic cases (≤ 72 months) against the 290 community controls.
- Matching was executed sequentially via `matched_final.py` (lines 34–48). For each clinic case, the available control pool within ±6 months was sampled (up to 2 controls).
- Due to the age distribution truncation of the community cohort (maximum control age 59 months), 6 clinic cases (aged 66, 72, 71, 71, 66, and 69 months; mean case age 53.1 months, mean control age 46.4 months) could not be matched with any control.
- **Code Audit (Bug #2 - Retention of Unmatched Cases):** Although the script correctly tracked these 6 unmatched cases in a diagnostic list (`unmatched_case_ages`), the final dataset concatenation step invoked the unpruned case dataframe:
  ```python
  matched = pd.concat([matched_controls.assign(case=0),
                       cases.assign(case=1)])
  ```
  Consequently, all 23 cases were concatenated with the 34 matched controls, producing a 57-row dataset rather than the nominal 51-row matched cohort (17 cases + 34 controls). In this study, we designate the 57-row dataset as **Run A (Author Construction)** and the strictly matched 51-row dataset (excluding the 6 unmatched cases) as **Run B (Corrected Matched Cohort)**.

### 1.4 Psychometric Instrument Preprocessing Audit
Caregivers completed a 9-item Likert questionnaire measuring three parenting typologies (authoritative, authoritarian, and permissive; 3 items each) and a stimulation activity scale (*stim_score*).
- **Code Audit (Bug #1 - Lexicographical Item Permutation):** In `clean.py` (line 51), the aggregation block processed questionnaire columns as follows:
  ```python
  def likert_block(df, prefix, mapping, n):
      cols = [c for c in df.columns if c.startswith(prefix)]
      out = pd.DataFrame({f'{i+1:02d}': df[c].map(mapping)
                          for i, c in enumerate(sorted(cols))})
  ```
  Because `sorted(cols)` sorted column names alphabetically based on raw question string text rather than survey index order, items measuring distinct psychological dimensions were scrambled. Group 01–03, 04–06, and 07–09 each collapsed into a heterogeneous blend of authoritative, authoritarian, and permissive questions.
- We audited the psychometric impact by computing Cronbach's alpha (α) under both the corrupted alphabetical ordering and the true questionnaire mapping. The stimulation scale was unaffected by this bug.

### 1.5 Machine Learning Pipeline and Confounding Evaluation
All classification pipelines were evaluated using 5-fold cross-validation repeated 5 times (5×5 CV, matching our original pipeline):
- **Feature Sets:** Our original risk model comprised 23 features: 21 numerical features (age, birth order, under-5 siblings, maternal age category, antenatal care visits, birth weight, low birth weight indicator, prematurity, cried at birth, exclusive breastfeeding, household smoking exposure, daily screen time, 3 parenting subscales, stimulation score, 2 maternal pregnancy illness indicators, and 3 child medical history indicators) and 2 categorical features (child sex and primary caregiver) encoded via one-hot transformation.
- **Channel Confounding Baseline (SES Baseline):** To evaluate whether models learned institutional referral stream rather than developmental delay pathology, we constructed a negative control channel-only model using strictly four parental socio-economic status (SES) variables: maternal education (`edu_mom`), paternal education (`edu_dad`), household income (`income`), and maternal employment status (`mom_work`). We also evaluated a combined feature set (23 risk features + 4 SES variables).
- **Classifiers:** Logistic Regression (LR), Decision Tree (DT), and Random Forest (RF) using our original hyperparameters, supplemented by gradient boosting (XGBoost) and Platt-calibrated Logistic Regression.
- **Statistical Testing:** Differences in F1 scores across CV repetitions were evaluated using two-tailed paired t-tests.

### 1.6 Calibration, Risk Stratification, and Decision Curve Analysis
- **Sampling Prevalence Adjustment:** In the 1:2 matched cohort, the sampling prevalence is π_s = 0.3333 (17/51). Because case–control sampling distorts predicted probabilities, uncalibrated ML outputs cannot be interpreted as absolute patient risk.
- **Population Re-anchoring:** We anchored baseline population prevalence at π₀ = 0.0793, corresponding to the empirical rate of at-risk children identified in the community *Posyandu* arm (23/290). Predicted log-odds were recalibrated using Bayes' rule prior correction:
  $$\text{logit}(p_{\text{adj}}) = \text{logit}(\hat{p}) - \ln\left(\frac{\pi_s}{1 - \pi_s}\right) + \ln\left(\frac{\pi_0}{1 - \pi_0}\right)$$
- **Decision Curve Analysis (DCA):** Standardized net benefit was calculated across decision thresholds ($p_t$) from 0.05 to 0.40 [[VERIFY CITE: Vickers & Elkin, 2006]]:
  $$\text{Net Benefit} = \frac{\text{TP}}{N} - \frac{\text{FP}}{N} \left(\frac{p_t}{1 - p_t}\right)$$
  Net benefit was benchmarked against treat-all, treat-none, and an analytic comparator representing routine developmental screening (KPSP) modeled at sensitivity 0.60 and specificity 0.92 [[VERIFY CITE]].
- **Feature Attribution:** TreeSHAP [[VERIFY CITE: Lundberg & Lee, 2017]] was computed on Run B models to evaluate global mean absolute SHAP attributions and local case vignettes.

---

## 2. Results

### 2.1 Cohort Flow and Data Audit
The audit of participant flows across both recruitment arms is presented in Table 2.

### Table 2 (Table 1 Specification). Data Audit and Participant Flow Across Recruitment Channels

| Cohort Metric | Community Arm (*Posyandu*) | Clinical Arm (Pediatric Therapy Clinic) |
|---|---|---|
| **Raw Records Captured** | 302 | 33 |
| **Exclusions** | 12 total: 3 acute illness, 9 duplicate entries* | 3 total: 2 Down syndrome, 1 no clinical diagnosis |
| **Final Audited Cohort** | **290** | **30** |
| **Target Age Eligible (≤ 72 months)** | 290 (all children ≤ 59 months) | 23 (7 patients aged > 72 months excluded) |
| **Mean Age (Months) ± SD** | 30.73 ± 16.19 | 63.24 ± 26.38 |
| **Age Range (Months)** | [[TO BE COMPUTED: Min–Max community age]] | 34–167 |
| **Sex Distribution (% Male)** | 49.5% | [[TO BE COMPUTED: Clinic sex distribution]] |
| **Developmental Screening Status** | 267 Appropriate, 22 Doubtful, 1 Deviation (23 at-risk; 7.9%) | 30 Confirmed clinical developmental delay |
| **Available for 1:2 Age Matching** | 290 | 23 |
| **Successfully Matched (±6 mo caliper)** | 34 | 17 (6 unmatched: ages 66, 69, 71, 71, 66, 72) |

*\*Note: An alternative deduplication rule yields 8 duplicates (291 final community rows).*

---

### 2.2 Code Audit: Resolution of Psychometric Scale Corruption
Evaluating the impact of `clean.py` line 51 confirmed that alphabetical column sorting severely corrupted survey psychometrics (Table 3). Under the original script, internal consistency appeared near zero or negative, leading the original investigators to erroneously dismiss parenting style as psychometrically invalid. Restoring the correct column grouping demonstrates that authoritative and permissive parenting subscales maintain acceptable reliability.

### Table 3. Psychometric Scale Reliability: Corrupted Script vs. Restored Grouping

| Instrument Subscale | Cronbach's α (`clean.py` line 51, `sorted(cols)`) | Cronbach's α (Restored Survey Order) | Psychometric Status |
|---|---|---|---|
| **Authoritative Parenting** | **+0.100** | **+0.567** | Restored to acceptable exploratory threshold |
| **Authoritarian Parenting** | **−0.178** | **+0.160** | Low reliability |
| **Permissive Parenting** | **+0.132** | **+0.591** | Restored to acceptable exploratory threshold |
| **Stimulation Scale (`stim_score`)** | **+0.789** | **+0.789** | Unaffected by bug (consistent) |

---

### 2.3 Model Reproduction and Effect of Unmatched Case Retention
Re-executing our original modeling pipeline on the unpruned 57-row dataset (Run A) reproduced published Table 2 exactly across all classifiers to three decimal places (Table 4). 

When the pipeline was executed on the true 1:2 matched cohort (Run B, N = 51), predictive performance dropped noticeably across all models. Random Forest mean F1 dropped from 0.791 to 0.695, and sensitivity (recall) dropped from 0.750 to 0.647. Furthermore, on the true matched cohort, Random Forest did not strictly dominate all baselines: Logistic Regression achieved a higher mean recall (0.667 vs. 0.647).

### Table 4 (Table 2 Specification). Classification Performance: Reproduction of Our Published Table 2 (Run A) vs. Corrected Matched Set (Run B)

| Model | Evaluation Run | Rows (Cases / Controls) | Accuracy (Mean ± SD) | Precision | Recall (Sensitivity) | F1-Score | Pooled Out-of-Fold Matrix (TP / FN / FP / TN) |
|---|---|---|---|---|---|---|---|
| **Logistic Regression** | **Run A (Our Original Code)** | 57 (23 / 34) | 0.839 ± 0.079 | 0.859 | 0.740 | 0.777 | [[TO BE COMPUTED: LR Run A OOF]] |
| | **Run B (Corrected)** | 51 (17 / 34) | 0.815 ± 0.120 | 0.733 | 0.667 | 0.676 | [[TO BE COMPUTED: LR Run B OOF]] |
| **Decision Tree** | **Run A (Our Original Code)** | 57 (23 / 34) | 0.731 ± 0.112 | 0.669 | 0.628 | 0.629 | [[TO BE COMPUTED: DT Run A OOF]] |
| | **Run B (Corrected)** | 51 (17 / 34) | 0.722 ± 0.130 | 0.563 | 0.607 | 0.573 | [[TO BE COMPUTED: DT Run B OOF]] |
| **Random Forest** | **Run A (Our Published Table 2)** | **57 (23 / 34)** | **0.852 ± 0.082** | **0.890** | **0.750** | **0.791** | **TP 16 / FN 7 / FP 2 / TN 32** (Acc 0.842, Prec 0.889, Rec 0.696, F1 0.780) |
| | **Run B (Corrected)** | **51 (17 / 34)** | **0.827 ± 0.092** | **0.834** | **0.647** | **0.695** | **TP 11 / FN 6 / FP 4 / TN 30** (Acc 0.804, Prec 0.733, Rec 0.647, F1 0.688) |

---

### 2.4 Empirical Proof of Recruitment-Channel Confounding
Evaluating the channel-only baseline against the developmental risk-factor model on the corrected 51-row cohort provided decisive evidence of recruitment confounding (Table 5).

### Table 5. Channel Confounding Experiment: Clinical Risk Models vs. Socio-Economic Baseline (Run B, N = 51)

| Model Specification | Feature Composition | Mean F1-Score ± SD | Paired Comparison vs. Channel Baseline |
|---|---|---|---|
| **A. Clinical Risk Model** | 21 numerical + 2 categorical risk factors (our original design) | **0.695 ± 0.173** | **Paired ΔF1 (A − B) = −0.181** |
| **B. Channel-Only Baseline** | **4 SES variables only** (`edu_mom`, `edu_dad`, `income`, `mom_work`) | **0.876 ± 0.126** | **95% CI: [−0.500, 0.176], p = 0.00014** |
| **C. Combined Model** | 23 clinical risk factors + 4 SES variables | **0.902 ± 0.140** | — |

The 4-variable SES baseline significantly outperformed the entire 23-feature clinical risk-factor model (p = 0.00014). This demonstrates that the predictive accuracy reported in two-stream case–control triage models is largely driven by institutional referral disparities (SES differences between clinic and community populations) rather than specific developmental delay etiology.

---

### 2.5 Model Calibration and Benchmark Models
Model calibration metrics on the matched scale (5×5 CV, Run B) are summarized in Table 6.

### Table 6 (Table 3 Specification). Calibration and Benchmark Performance (Run B, Matched Scale)

| Classifier | Calibration Slope | Calibration Intercept | Brier Score | AUROC | Mean Predicted Probability | Observed Case Rate |
|---|---|---|---|---|---|---|
| **Random Forest** | 2.656 | 0.274 | 0.144 | 0.896 | 0.381 | 0.3333 |
| **XGBoost** | 0.801 | 0.060 | 0.136 | 0.848 | 0.310 | 0.3333 |
| **Calibrated Logistic Regression** | 3.269 | 1.423 | 0.152 | 0.903 | 0.336 | 0.3333 |

XGBoost achieved superior calibration relative to Random Forest (slope 0.801 vs. 2.656; Brier score 0.136 vs. 0.144), whereas Random Forest showed substantial overconfidence.

#### Population Risk Stratification (Anchor π₀ = 0.0793)
Grouping patients into predicted risk tertiles under the prior-corrected anchor (π₀ = 0.0793) confirmed monotonic risk separation across strata:
- **Low Risk:** n = 17, observed cases = 1 (rate 0.059, mean predicted probability 0.0398)
- **Medium Risk:** n = 17, observed cases = 3 (rate 0.176, mean predicted probability 0.0869)
- **High Risk:** n = 17, observed cases = 13 (rate 0.765, mean predicted probability 0.2187)

---

### 2.6 Decision Curve Analysis and Clinical Utility
Standardized net benefit across decision threshold probabilities ($p_t$) is detailed in Table 7 (specification for Figure 1).

### Table 7. Decision Curve Analysis: Prior-Corrected Net Benefit Across Decision Thresholds (π₀ = 0.0793)

| Decision Threshold ($p_t$) | Risk Model Net Benefit | Screen-All Strategy Net Benefit | Analytic KPSP Comparator (Se 0.60, Sp 0.92) | Clinical Interpretation |
|---|---|---|---|---|
| **0.05** | **0.311** | 0.298 | 0.197 | Model superior |
| **0.10** | **0.259** | 0.259 | 0.194 | Model superior / equal to screen-all |
| **0.15** | **0.225** | 0.216 | 0.191 | Model superior |
| **0.20** | **0.132** | 0.167 | **0.187** | **KPSP comparator superior; screen-all superior** |
| **0.25** | **0.059** | 0.111 | **0.182** | **KPSP comparator superior; screen-all superior** |
| **0.30** | **0.059** | 0.048 | **0.177** | **KPSP comparator superior** |
| **0.40** | **0.000** | −0.111 | **0.164** | **KPSP comparator superior; model zero net benefit** |

**Figure 1 Specification (Decision Curve):**  
Plotting standardized net benefit (Y-axis: −0.2 to 0.4) against threshold probability $p_t$ (X-axis: 0.05 to 0.40). The curve depicts:
1. *Risk Model (Solid Blue):* Peaks at low thresholds but drops steeply at $p_t \ge 0.20$.
2. *Analytic KPSP (Dashed Green):* Maintains stable positive net benefit (0.197 to 0.164) across all thresholds.
3. *Screen-All (Dotted Red):* Crosses below zero at $p_t \approx 0.35$.
4. *Screen-None (Solid Gray):* Horizontal line at zero.  
*Key Takeaway:* The ML model provides clinical value only in low-threshold risk-tolerant scenarios ($p_t \le 0.15$). At clinical thresholds of $p_t \ge 0.20$, standard KPSP screening provides superior net benefit. Claims that the model outperforms standard triage across all decision thresholds are empirically disproven.

---

### 2.7 Feature Attribution and Clinical Vignette
Global TreeSHAP attribution on the corrected Run B Random Forest model is summarized in Table 8 (specification for Figure 2). Daily screen time (`screening_time`) was the primary discriminative feature, followed by authoritative parenting and maternal pregnancy history.

### Table 8 (Figure 2 Specification). Global Top Feature Importance (TreeSHAP on Run B)

| Rank | Feature Identifier | Description | Mean Absolute SHAP Value |
|---|---|---|---|
| 1 | `num__screen_time` | Daily screen duration | **0.130** (0.1301) |
| 2 | `num__par_authoritative` | Authoritative parenting score | **0.032** (0.0321) |
| 3 | `num__preg_any` | Any maternal pregnancy complication | **0.028** (0.0281) |
| 4 | `num__age_months` | Child age in months | 0.0260 |
| 5 | `num__ebf` | Exclusive breastfeeding duration | 0.0255 |
| 6 | `cat__caregiver_Ibu` | Primary caregiver: Mother | 0.0254 |
| 7 | `num__anc` | Antenatal care visit count | 0.0228 |
| 8 | `num__birth_weight` | Birth weight (kg) | 0.0226 |

#### Local Patient Attribution Vignette
For an illustrative clinical case correctly identified by the model (predicted probability $\hat{p} = 0.793$), the local feature contributions were:
- `screen_time`: **+0.166** (strongly increases predicted risk)
- `ebf`: **+0.060**
- `par_authoritative`: **+0.050**
- `ill_recurrent`: **+0.042**
- `stim_score`: **+0.027**
- `birth_weight`: **+0.019**

This case vignette confirms that while individual behavioral features align with clinician intuition, the model's overall predictive utility cannot be divorced from institutional channel bias.


---

# Discussion and Methodological Lessons

## 1. Principal Findings

This validity study provides an empirical demonstration of how dual-channel case–control recruitment and minor code-level implementation defects undermine the integrity of biomedical machine learning (ML) models. By executing an auditable, bit-for-bit reproduction of our own previously published Random Forest (RF) triage model for pediatric developmental delay in Indonesia, we uncovered two critical insights.

First, the apparent predictive performance of the 23-feature clinical risk model is largely attributable to institutional recruitment-channel confounding rather than developmental pathology. A negative-control baseline consisting solely of four parental socio-economic status (SES) variables achieved an F1-score of 0.876 ± 0.126, significantly outperforming the complete 23-feature clinical risk model (F1 = 0.695 ± 0.173, p = 0.00014). In dual-channel designs where cases are recruited from specialized therapy clinics and controls from community health posts (*Posyandu*), the classifier predominantly learns structural socio-economic divergence between patient referral pathways.

Second, the study demonstrates the disproportionate impact of minor programming oversights in small-sample biomedical ML. Two subtle errors—lexicographical column sorting in survey preprocessing and inadvertent retention of unmatched cases in cohort construction—substantively distorted both the psychometric conclusions and cross-validation benchmarks of the original study. When corrected, the primary classification metrics declined (RF F1 decreased from 0.791 to 0.695; recall decreased from 0.750 to 0.647), and the claim that Random Forest strictly dominated all baseline algorithms was invalidated, with Logistic Regression achieving superior sensitivity (0.667 vs. 0.647). Furthermore, decision curve analysis revealed that the clinical utility of the risk model is confined strictly to low decision thresholds (0.05–0.15) and is dominated by standard routine screening (KPSP) at decision thresholds ≥ 0.20.

---

## 2. Anatomy of Two Implementation Bugs

### 2.1 Bug #1: Lexicographical Item Permutation in Psychometrics
In modern health informatics, survey questionnaires measuring behavioral constructs are frequently ingested into automated dataframes. In our original preprocessing script (`clean.py`, line 51), we used `sorted(cols)` to organize questionnaire items. While syntactically benign, Python’s string sort reordered question labels alphabetically by question prompt text rather than preserving survey item order. 

As a consequence, the 9 Likert items—originally designed to capture three distinct parenting typologies (authoritative, authoritarian, and permissive; 3 items each)—were scrambled. Each 3-item block collapsed into a composite of divergent behavioral constructs. This generated severe psychometric artifacts, producing near-zero or negative Cronbach’s alpha coefficients (authoritative α = +0.100, authoritarian α = −0.178, permissive α = +0.132). These distorted metrics misled us into concluding that parental behavior was psychometrically unmeasurable in this cohort, demoting parenting style to an exploratory variable.

Restoring the true survey item sequence immediately recovered acceptable internal consistency for authoritative (α = 0.567) and permissive (α = 0.591) styles. This demonstrates how a single automated sorting operation can silently erase a legitimate clinical risk domain from subsequent scientific analysis.

### 2.2 Bug #2: Data Leakage and Retention of Unmatched Cases
In 1:2 age-matched case–control studies, strict matching criteria are essential to prevent age-related confounding. In `matched_final.py` (lines 34–48), the matching loop correctly flagged 6 clinical cases (aged 66–72 months) that lacked community controls within the ±6-month caliper (because community controls were truncated at 59 months). However, during final cohort concatenation, the script concatenated the entire unpruned case dataframe (`cases.assign(case=1)`) with the 34 matched controls.

This unintended leak inflated the dataset from 51 to 57 rows, appending 6 unmatched, older clinical cases without corresponding community controls. Because older clinical cases presented with distinct clinical and exposure profiles, their presence artificially boosted cross-validation discrimination metrics (Run A RF F1 = 0.791). Re-running the pipeline on the strictly matched 51-row dataset (Run B) revealed a marked drop in performance: RF F1 fell by nearly 0.10 to 0.695, and recall dropped from 0.750 to 0.647. Furthermore, on the true matched cohort, Random Forest lost its across-the-board superiority: Logistic Regression attained a higher recall (0.667 vs. 0.647).

---

## 3. The Mechanism of Recruitment-Channel Confounding

The most consequential finding of this audit is that multi-feature developmental risk models can be severely confounded by the institutional architecture of participant recruitment. In low- and middle-income countries, families who actively seek specialized pediatric outpatient therapy clinics typically possess higher formal education, higher household income, and greater health literacy compared to the broader population attending routine community *Posyandu* centers.

When machine learning algorithms are trained to differentiate clinic cases from community controls, they exploit any covariate distribution shift between the two recruitment streams. When evaluated in isolation, 4 parental SES variables (`edu_mom`, `edu_dad`, `income`, `mom_work`) yielded an F1 score of 0.876 ± 0.126. In contrast, the 23 clinical, perinatal, and environmental risk factors carefully selected by domain experts yielded an F1 score of only 0.695 ± 0.173. The paired difference (ΔF1 = −0.181, p = 0.00014) confirms that the model's discriminative power is dominated by institutional selection artifacts rather than clinical developmental pathology.

This finding exposes a fundamental vulnerability in case–control machine learning: without negative control baselines (e.g., channel-only models), researchers risk mistaking institutional selection mechanisms for true clinical predictive signal.

---

## 4. Methodological Implications for Indonesian Health Informatics

Our findings carry direct implications for the growing literature on applied healthcare machine learning in Indonesia and regional biomedical informatics:

1. **Mandatory Channel-Only Negative Controls:**  
   Studies employing multi-stream recruitment (e.g., hospital cases vs. community controls) must routinely include negative control models trained strictly on recruitment artifacts (demographics, administrative markers, interview timing). If non-clinical administrative or SES features outperform clinical features, the model cannot be deployed as a clinical triage tool [[VERIFY CITE: PROBAST / Moons et al., 2019]].

2. **Prevalence Distortion and Prior Correction:**  
   Artificially matched 1:2 case–control cohorts enforce an artificial sampling prevalence ($\pi_s = 0.3333$). Deploying raw model probabilities in primary care environments where community prevalence is roughly 7.9% ($\pi_0 = 0.0793$) causes severe probability miscalibration. Calibration curves and prior-odds corrections must be standard reporting requirements.

3. **Clinical Utility Beyond Discrimination (Decision Curve Analysis):**  
   Reporting accuracy or AUC alone is insufficient for clinical triage tools. As demonstrated in our decision curve analysis, the risk model yielded positive net benefit only at low threshold probabilities ($p_t = 0.05–0.15$). At decision thresholds $p_t \ge 0.20$, the model was strictly inferior to simple analytic KPSP screening. Machine learning tools should only be recommended when they demonstrably exceed routine, low-cost screening tools across relevant decision thresholds [[VERIFY CITE: Vickers & Elkin, 2006]].

4. **Open Code and Pipeline Auditing:**  
   Subtle bugs like `sorted(cols)` and unpruned dataframe concatenation do not trigger runtime errors and remain invisible during standard peer review unless executable scripts and data pipelines are fully audited. Journal editors and reviewers must mandate public code repositories for published predictive models.

---

## 5. Limitations

Several limitations of the present validity investigation should be emphasized:

1. **Sample Size Constraints:**  
   The audited matched cohort is modest (N = 51; 17 clinical cases and 34 community controls). While this sample size accurately reflects the empirical dataset used in the original investigation, confidence intervals around cross-validation metrics remain wide.

2. **Analytic Nature of the KPSP Comparator:**  
   In the clinical arm, developmental delay was confirmed through clinical diagnosis and therapy admission; the formal KPSP screening instrument was not administered to clinic patients. Consequently, the KPSP comparator in our decision curve analysis was modeled analytically using published diagnostic sensitivity (0.60) and specificity (0.92) rather than head-to-head empirical testing.

3. **Assumption of Population Prevalence Anchor ($\pi_0$):**  
   The baseline prevalence anchor ($\pi_0 = 0.0793$) was derived from the empirical rate of at-risk children in the Posyandu community arm (23/290). This empirical anchor represents an operational assumption that requires formal verification through regional epidemiological surveys in South Sumatra.

4. **Observational and Cross-Sectional Nature:**  
   Although TreeSHAP identified daily screen time as the predominant behavioral contributor (mean absolute SHAP = 0.130), this association reflects observational correlation and recruitment-stream differences; it cannot be interpreted as a direct causal etiology.


---

# Response to Editor and Reviewers: Resubmission Re-framing Letter

**Date:** October 16, 2026  
**To:** Editor-in-Chief and Editorial Board, *Jurnal ELTIKOM: Jurnal Teknik Elektro, Teknologi Informasi dan Komputer*  
**From:** Dini Hari Pertiwi, Yarza Aprizal, Eko Setiawan  
**Institution:** Institut Teknologi dan Bisnis Palcomtech, Palembang, Indonesia  
**Manuscript Title (Revised):** *Recruitment-Channel Confounding Outweighs Risk Factors in Machine-Learning Case–Control Models of Developmental Delay: A Validity Study*  
*(Alternative Title: When Case–Control Machine Learning Learns the Recruitment Channel: A Validity Study of Developmental-Delay Risk Prediction in Indonesia)*  
**Our Original Submission Title:** *Random Forest Classification for Early Detection of Developmental Delay Risk in Early Childhood*

---

Dear Editor-in-Chief and Editorial Board of Jurnal ELTIKOM,

We wish to express our sincere gratitude to the Editorial Board and the anonymous reviewers for evaluating our previous submission. The initial decision to reject our manuscript was grounded in the assessment that the work lacked sufficient novelty and presented an incremental application of standard classification algorithms without a decisive technical or methodological contribution.

We took this evaluation seriously. Rather than performing minor cosmetic revisions or attempting to tune hyperparameters on the same data, our research team undertook an exhaustive, independent code audit and algorithmic reproduction of our entire data pipeline (`clean.py` → `pipeline.py` → `matched_final.py`). 

This audit led to a profound scientific finding that prompted a complete, principled pivot: **we have reframed the manuscript from an unverified "screening/triage tool" into a rigorous methodological validity and reproducibility study.**

Below, we detail why this pivot fundamentally transforms the paper, how it directly resolves the novelty concern, and the specific empirical discoveries now documented in the revised manuscript.

---

### 1. Why the Pivot Resolves the Novelty and Contribution Deficiency

In the regional biomedical informatics literature in Indonesia, hundreds of articles report machine learning classifiers applied to small observational clinical datasets, almost universally claiming high accuracy (80%–95%) and advocating for automated screening tools. However, such studies routinely suffer from three pervasive flaws:
1. They overlook severe recruitment-channel confounding inherent in two-stream recruitment (e.g., hospital/clinic cases versus community/posyandu controls);
2. They do not release or audit code, leaving subtle preprocessing and cohort selection errors undetected; and
3. They fail to conduct formal calibration adjustments or decision curve analyses against existing routine healthcare standards.

Our revised manuscript offers **a rare, bit-for-bit reproducible validity study** that directly addresses these systemic vulnerabilities. Rather than claiming our model is ready for clinical deployment, we demonstrate empirically—using our own audited data and code—that two-stream case–control models predominantly learn recruitment-channel disparities rather than clinical pathology. This genre of critical validity study is virtually non-existent in the Indonesian health informatics literature and provides substantial educational and methodological value to the readers of *Jurnal ELTIKOM*.

---

### 2. Summary of Key Audit Discoveries and New Empirical Evidence

The revised manuscript presents five substantive contributions that were entirely absent from the initial submission:

#### A. Exact Bit-for-Bit Reproduction and Identification of Bug #2 (Cohort Leakage)
We successfully reproduced the classification performance reported in Table 2 of our original manuscript to three decimal places (Run A: Random Forest accuracy 0.852 ± 0.082, precision 0.890, recall 0.750, F1 0.791; out-of-fold matrix: TP 16, FN 7, FP 2, TN 32). This reproduction confirmed that Table 2 was computed from an uncorrected 57-row dataset containing 23 cases and 34 controls. 

Our code audit revealed that in `matched_final.py` (lines 34–48), 6 older clinical cases (aged 66–72 months) that could not be matched to community controls (due to an age cutoff of 59 months in Posyandu controls) were correctly logged as unmatched, but were accidentally retained during dataframe concatenation (`cases.assign(case=1)`). 

When the pipeline is corrected to the true 1:2 age-matched cohort (Run B, N = 51; 17 cases and 34 controls), model performance declines across all metrics:
- Random Forest F1 drops from 0.791 to 0.695 ± 0.173.
- Recall drops from 0.750 to 0.647.
- Random Forest no longer dominates across all metrics: Logistic Regression achieves a higher sensitivity/recall (0.667 vs. 0.647).

#### B. Discovery and Rectification of Bug #1 (Lexicographical Column Sorting)
In our original manuscript, we reported that parental style subscales were psychometrically unreliable (Cronbach's α = +0.100 for authoritative, −0.178 for authoritarian, and +0.132 for permissive) and demoted parenting style to an exploratory variable. 

Our audit uncovered that in `clean.py` (line 51), the helper function executed `sorted(cols)`, which sorted survey question labels alphabetically based on raw question string text rather than survey index order. This scrambled the 9 questionnaire items across distinct constructs. 

When the correct survey grouping is restored, Cronbach's alpha values for authoritative and permissive styles immediately recover to acceptable reliability (+0.567 and +0.591, respectively; authoritarian remains low at +0.160). We show how a single line of sorting code can inadvertently invalidate a legitimate clinical domain.

#### C. Empirical Proof of Recruitment-Channel Confounding
To test whether our model learned true developmental delay risk or institutional recruitment stream, we constructed a negative control channel-only model using strictly four parental socio-economic status (SES) variables: maternal education, paternal education, family income, and maternal employment status.

The results provide definitive evidence of channel confounding:
- **4-variable SES Baseline F1:** **0.876 ± 0.126**
- **23-feature Clinical Risk Model F1:** **0.695 ± 0.173**
- **Paired difference:** **ΔF1 = −0.181, 95% CI: [−0.500, 0.176], p = 0.00014**

The four socio-economic status features significantly outperformed the full 23-feature clinical risk-factor model. This proves that the predictive signal in two-stream recruitment models is dominated by socio-economic differences between clinic families and community families, not neurodevelopmental pathology.

#### D. Calibration, Benchmark Models, and Population Re-anchoring
We benchmarked the Random Forest model against XGBoost and Calibrated Logistic Regression. Random Forest manifested severe probability miscalibration (calibration slope 2.656, Brier score 0.144), whereas XGBoost was substantially better calibrated (slope 0.801, Brier 0.136). We adjusted the model for community prevalence by re-anchoring from the artificial sampling prevalence (π_s = 0.3333) to the empirical Posyandu at-risk prevalence (π₀ = 0.0793), demonstrating monotonic risk stratification across low (0.059), medium (0.176), and high (0.765) risk strata.

#### E. Decision Curve Analysis (DCA) and Clinical Utility Limits
Rather than claiming that our model is universally effective, our decision curve analysis demonstrates clear operational boundaries:
- At low decision thresholds (0.05–0.15), the risk model offers positive net benefit (0.311 to 0.225) compared to treat-all and treat-none strategies.
- However, at decision thresholds $p_t \ge 0.20$, the model is strictly inferior to standard analytic developmental screening (*KPSP*, net benefit 0.187 vs. model 0.132 at $p_t = 0.20$).

---

### 3. Overview of Resubmission Package

In accordance with our new framing, the complete manuscript has been overhauled:
- **`01-framing.md`:** New title options, concise abstract (~230 words), completely rewritten Introduction featuring an explicit gap table contrasting regional literature, original submission, and current validity study, structured around four falsifiable claims.
- **`02-methods-results.md`:** Comprehensive reporting following TRIPOD+AI and PROBAST standards, detailing data audits, exact code audit procedures, full reproduction tables (Table 1 through Table 6), and specifications for Decision Curve (Figure 1) and SHAP feature attribution (Figure 2).
- **`03-discussion.md`:** In-depth discussion of the anatomy of both programming bugs, the systemic problem of recruitment channel confounding in LMICs, critical lessons for Indonesian medical informatics, and candid limitations.
- **`05-author-todo.md`:** An operational checklist in Indonesian outlining our commitments regarding population prevalence justification, raw data cleaning, code repository publication, and ethical governance.

We believe this transparent, rigorous, and self-correcting validity study provides an exemplary contribution to *Jurnal ELTIKOM*, offering valuable cautionary lessons for biomedical data scientists across Indonesia. We look forward to the editorial decision.

Sincerely,

**Dini Hari Pertiwi, Yarza Aprizal, Eko Setiawan**  
Institut Teknologi dan Bisnis Palcomtech, Palembang, Indonesia


---

# Daftar Tindak Lanjut & Tugas Penulis (Author Action Items)

Dokumen ini memuat daftar tugas dan pemeriksaan penting yang harus diselesaikan oleh tim penulis (Dini Hari Pertiwi, Yarza Aprizal, Eko Setiawan) sebelum naskah versi studi validitas di-submit ulang ke Jurnal ELTIKOM.

---

## 1. Justifikasi Prevalensi Populasi ($\pi_0$) untuk Kalibrasi Model

Dalam analisis kalibrasi dan *Decision Curve Analysis* (DCA) naskah revisi, probabilitas prediksi model telah dire-kalibrasi dari prevalensi sampling *matched* ($\pi_s = 0.3333$ atau rasio 1:2) ke nilai anchor prevalensi populasi komunitas:
$$\pi_0 = 0.0793 \quad (7.9\%)$$
Angka ini diturunkan secara empiris dari proporsi anak berisiko di lengan Posyandu (23 anak dengan KPSP *meragukan* atau *penyimpangan* dari total 290 anak).

**Tugas Penulis:**
1. Konfirmasi apakah prevalensi 7.9% ini representatif untuk populasi anak balita di Kota Palembang / Sumatera Selatan.
2. Cari rujukan literatur pendukung (misalnya data Riskesdas / SKI Kemenkes RI, Profil Kesehatan Dinas Kesehatan Kota Palembang, atau publikasi epidemiologi tumbuh kembang anak di Indonesia) yang melaporkan angka prevalensi keterlambatan perkembangan balita berkisar antara 5%–10%.
3. Cantumkan rujukan tersebut pada subbab Methods (bagian *Prior Correction*) untuk memperkuat legitimasi anchor $\pi_0$.

---

## 2. Perbaikan dan Pembersihan Anomali Data Mentah

Audit data independen menemukan beberapa ketidakkonsistenan pada berkas data mentah yang perlu diklarifikasi dan diperbaiki oleh tim penulis:

### A. Outlier Usia pada Data Klinik (111 dan 167 Bulan)
- **Temuan:** Pada lengan klinik, ditemukan pasien dengan usia tercatat **111 bulan** (~9.2 tahun) dan **167 bulan** (~13.9 tahun). Rentang usia balita/anak prasekolah maksimal adalah 72 bulan (6 tahun).
- **Aksi Penulis:**
  - Periksa kembali catatan rekam medis asli: apakah ini salah ketik (misal: 11 bulan tertulis 111 bulan, atau input dalam hitungan hari/minggu)?
  - Jika pasien tersebut memang berusia sekolah/remaja yang menjalani terapi wicara/okupasi, pastikan kriteria eksklusi (usia > 72 bulan) diterapkan secara konsisten sebelum analisis, dan jelaskan alasan eksklusi tersebut di naskah.

### B. Baris KPSP dengan Jumlah Jawaban "Ya" = 1 di Data Posyandu
- **Temuan:** Ditemukan satu baris data anak di Posyandu dengan total jawaban "Ya" hanya **1** dari 10 pertanyaan KPSP. Baris ini secara otomatis masuk sebagai kategori "penyimpangan" dan menjadi bagian dari 23 kasus *at-risk*.
- **Aksi Penulis:** Verifikasi lembar kuesioner fisik anak tersebut. Apakah benar anak tersebut hanya mampu melakukan 1 dari 10 instrumen perkembangan (kondisi deviasi sangat berat), atau apakah kader salah mengisi/memasukkan data?

### C. Rekonsiliasi Aturan Deduplikasi (290 vs. 291 Baris)
- **Temuan:** Skrip asli penulis menghasilkan **290 baris** komunitas (menghapus 9 baris duplikat), sedangkan verifikasi independen mendeteksi **8 duplikat** (menghasilkan 291 baris).
- **Aksi Penulis:**
  - Tetapkan kunci unik (*composite key*) deduplikasi yang pasti (misal: `Nama Anak` + `Tanggal Lahir` + `Nama Ibu`).
  - Dokumentasikan kriteria deduplikasi secara eksplisit di skrip `clean.py` dan pada teks naskah.

### D. Tanggal Pemeriksaan di Luar Periode Kampanye
- **Temuan:** Mayoritas data Posyandu dikumpulkan pada periode kampanye pertengahan 2026, tetapi terdapat catatan tanggal pemeriksaan tertanggal **2022-11-28** dan **2023-11-26**.
- **Aksi Penulis:** Klarifikasi apakah baris-baris ini merupakan data uji coba (*pilot testing*) kuesioner tahun sebelumnya yang tidak sengaja terbawa ke master sheet. Jika ya, baris tersebut sebaiknya dieksklusi secara rapi.

### E. Standarisasi Nama Enumerator (11 Varian Ejaan)
- **Temuan:** Terdapat hingga 11 variasi penulisan nama untuk enumerator/kader yang sama (akibat perbedaan spasi, gelar, atau singkatan).
- **Aksi Penulis:** Lakukan standarisasi kolom enumerator di master data agar tidak menimbulkan masalah bila kelak dilakukan analisis efek klaster (*inter-rater reliability*).

### F. Dokumentasi Data Berat Lahir Buku KIA yang Kosong (18.5%)
- **Temuan:** Kolom berat lahir berdasarkan buku KIA kosong pada **56 dari 302 responden (18.5%)**. Skrip analisis saat ini mengatasinya dengan menggunakan nilai ingatan ibu (*maternal recall*).
- **Aksi Penulis:** Tuliskan prosedur *fallback* penanganan data hilang ini secara transparan pada bagian instrumen pengumpulan data di naskah.

---

## 3. Rilis Repositori Kode & Data Terbuka (Open Science)

Salah satu keunggulan utama studi validitas ini adalah klaim replikabilitas *bit-for-bit*. Reviewer dan editor ELTIKOM akan sangat menghargai ketersediaan kode:
1. **Bersihkan dan Anonimkan Data:**
   - Hapus seluruh informasi identitas pribadi (PII) sebelum publikasi: nama lengkap anak, nama orang tua, alamat jalan/RT/RW, dan nomor telepon.
   - Gunakan ID anonim (misal: `POS-001`, `KLN-001`).
2. **Kompilasi Repositori:**
   - Siapkan repositori publik di GitHub atau Zenodo (untuk mendapatkan DOI permanen).
   - Masukkan skrip pipeline yang sudah diaudit: `clean.py` (yang sudah diperbaiki bug `sorted()`-nya), `pipeline.py`, `matched_final.py` (yang sudah diperbaiki penanganan 6 kasus tak-termatch), serta skrip evaluasi kalibrasi & DCA.
   - Sertakan file `README.md` dan `requirements.txt` yang memungkinkan pembaca menjalankan ulang pipeline dalam sekali perintah.

---

## 4. Persetujuan Etik Penelitian dan Informed Consent

Sebagai studi kesehatan anak yang melibatkan data pasien klinik dan balita Posyandu:
1. **Nomor Keterangan Lolos Kaji Etik (*Ethical Clearance*):**
   - Pastikan nomor surat izin/lolos kaji etik dari Komisi Etik Penelitian Kesehatan (KEPK) terkait dicantumkan di naskah pada bagian *Declarations / Ethical Approval*.
2. **Informed Consent:**
   - Nyatakan secara tertulis di naskah bahwa seluruh orang tua/wali peserta telah menandatangani formulir persetujuan setelah penjelasan (*informed consent*) sebelum data kuesioner dan rekam medis diambil.

