Understanding leisure-related program effects by using process data in the HealthWise South Africa Project
Loading...
Date
2008
Journal Title
Journal ISSN
Volume Title
Publisher
American Academy for Park and Recreation Administration
Abstract
As the push for evidence-based programming gathers
momentum, many human services programs and interventions are under
increased scrutiny to justify their effectiveness across different conditions
and populations. Government agencies and the public want to be assured
that their resources are being put to good use on programs that are effective
and efficient. Thus, programs are increasingly based on theory and evaluated
through randomized control trials using longitudinal data. Despite this progress,
hypothesized outcomes are often not detected and/or their effect sizes
are small. Moreover, findings may go against intuition or “gut feelings” on
the part of project staff. Given the need to understand how program implementation
issues relate to outcomes, this study focuses on whether process
measures that focus on program implementation and fidelity can shed light
on associated outcomes. In particular, we linked the process evaluation of the
HealthWise motivation lesson with outcomes across four waves of data collection.
We hypothesized that HealthWise would increase learners’ intrinsic
and identified forms of motivation, and decrease amotivation and extrinsic
motivation. We did not hypothesize a direction of effects on introjected motivation
due to its conceptual ambiguity. Data came from youth in four intervention
schools (n = 902, 41.1%) and five control schools (n = 1291, 58.9%)
who were participating in a multi-cohort, longitudinal study. The schools
were in a township near Cape Town, South Africa. For each cohort, baseline
data are collected on learners as they begin grade 8. We currently have four
waves of data collected on the first cohort, which is the focus of this paper.
The mean age of the sample at wave 3 was 15.0 years (SD = .86) and 51%
of students were female. Results suggested that there was evidence of an
overall program effect of the curriculum on amotivation regardless of fidelity
of implementation. Compared to the control schools, all treatment school
learners reported lower levels of amotivation in wave 4 compared to wave 3,
as hypothesized. Using process evaluation data to monitor implementation fi147
delity, however, we also conclude that the school with better trained teachers
who also reported higher levels of program fidelity had better outcomes than
the other schools. We discuss the implications of linking process data with outcome
data and the associated methodological challenges in linking these data.
Description
Keywords
Adolescents, Implementation fidelity, Leisure motivation, Prevention, Process data
Citation
Caldwell, L.L., et al. (2008). Understanding leisure-related program effects by using process data in the HealthWise South Africa Project. Journal of Park and Recreation Administration, 26(2):146-162