Development and reliability of an audit tool to assess the school physical activity environment across 12 countries

Abstract

OBJECTIVES: Schools are an important setting to enable and promote physical activity. Researchers have created a variety of tools to perform objective environmental assessments (or ‘audits’) of other settings, such as neighborhoods and parks; yet, methods to assess the school physical activity environment are less common. The purpose of this study is to describe the approach used to objectively measure the school physical activity environment across 12 countries representing all inhabited continents, and to report on the reliability and feasibility of this methodology across these diverse settings. METHODS: The International Study of Childhood Obesity, Lifestyle and the Environment (ISCOLE) school audit tool (ISAT) data collection required an in-depth training (including field practice and certification) and was facilitated by various supporting materials. Certified data collectors used the ISAT to assess the environment of all schools enrolled in ISCOLE. Sites completed a reliability audit (simultaneous audits by two independent, certified data collectors) for a minimum of two schools or at least 5% of their school sample. Item-level agreement between data collectors was assessed with both the kappa statistic and percent agreement. Inter-rater reliability of school summary scores was measured using the intraclass correlation coefficient. RESULTS: Across the 12 sites, 256 schools participated in ISCOLE. Reliability audits were conducted at 53 schools (20.7% of the sample). For the assessed environmental features, inter-rater reliability (kappa) ranged from 0.37 to 0.96; 18 items (42%) were assessed with almost perfect reliability (Κ = 0.80–0.96), and a further 24 items (56%) were assessed with substantial reliability (Κ = 0.61–0.79). Likewise, scores that summarized a school’s support for physical activity were highly reliable, with the exception of scores assessing aesthetics and perceived suitability of the school grounds for sport, informal games and general play. CONCLUSIONS: This study suggests that the ISAT can be used to conduct reliable objective audits of the school physical activity environment across diverse, international school settings.