Reliability and validity of early adolescent mental health scale: an analysis of cross-sectional and longitudinal data of the global early adolescent study

Abstract

Objective: Assessment of mental health problems among young adolescents remains challenging because of a scarcity of suitable measurement tools. A 6-item questionnaire, the Early Adolescent Mental Health Scale (EAMHS), was designed to measure the mental health of young adolescents in urban, low-resource settings and was used in the Global Early Adolescent Study (GEAS). This study aims to assess the reliability and validity of the EAMHS. Method: Through a secondary data analysis of cross-sectional and longitudinal datasets from the GEAS, we first refined the underlying constructs of the EAMHS using exploratory factor analysis and assessed its internal reliability. We then took a set of measurement invariance tests to verify the cross-group, cross-cultural, and cross–time-point validity of the refined scale. Subsequently, we conducted a hypothesis test of construct validity and responsiveness using linear regression, incorporating adjustments for the complex sampling design. We carried out calibration between the EAMHS and the Patient Health Questionnaire—4 (PHQ-4) through a single-group design kernel equipercentile equating method, and obtained suggested cut-off scores at each site based on the first follow-up (T2) data. The discriminative ability of the scale with suggested cut-offs was assessed against the adolescents’ PHQ-4 scores of 6 or higher at the second follow-up (T3) using receiver operating characteristic (ROC) curve analyses. Results: The initial 6-item EAMHS was reduced to a 4-item version of a unidimensional structure. Scalar invariance was achieved across genders, age groups, and cultures in longitudinal datasets of 5 study sites. The 4-item EAMHS demonstrated acceptable internal reliability and structural validity, as well as excellent construct validity relative to that of the PHQ-4. Suggested EAMHS cut-off scores showed fair to good discriminative ability in the subsequent follow-up round, with area under the curve (AUC) values ranging from 0.71 (95% CI = 0.64-0.74) in Kinshasa to 0.82 (95% CI = 0.77-0.87) in Shanghai. Conclusion: The findings verified the EAMHS as a consistent and accurate measure, supporting its use for monitoring and evaluating general emotional disturbance within the study context. Diversity & Inclusion Statement: We worked to ensure sex and gender balance in the recruitment of human participants. We worked to ensure that the study questionnaires were prepared in an inclusive way.