Psychometric Properties of a Scale Focusing on Perceived Attributes of a Health Promoting School Approach

ABSTRACT

Objectives: The purpose of this paper was to investigate the psychometric properties of a newly-designed scale intended to measure the perceived attributes of a Health Promoting School initiative and its context in terms of factor structure, reliability and predictive validity. The scale was developed to explore possible predictors of the adoption of the Healthy School approach (HS) in Qu�bec.

Methods: Data were gathered from a 2007 cross-sectional study of 107 schools and 141 participants (school principals and school health promotion delegates). The scale was based on 7 attributes borrowed from the theories on diffusion of innovation. The factor structure of the scale was tested by exploratory and confirmatory factor analyses.

Results: The final scale included 14 items capturing 4 factors: school contextual barriers, collective efficacy, anticipated benefits and relative advantages. Reliability, in terms of internal consistency of the factors, ranged from a high of 0.85 to a low of 0.60. Three of the 4 factors significantly predicted HS adoption.

Conclusion: Overall, the scale showed good psychometric properties and may be useful to assess the attributes that could influence the adoption of this type of initiative.

Key words: Health promoting school; innovation; scale development; psychometrics

La traduction du r�sum� se trouve � la fin de l'article. Can J Public Health 2009;100(4):389-92.

In the last two decades, the "Health Promoting School" and the "Coordinated School Health Program", respectively from the World Health Organization and the US Centers for Disease Control and Prevention, have been advocated as effective approaches to promote health-related behaviours and well-being among school communities.1,2 These innovative approaches require moving from practices that rely mainly on classroom-based health education models to a more comprehensive, integrated model that includes both educational and environmental initiatives.3-8 In the province of Qu�bec (Canada), such a comprehensive approach has been offered since 2004 to all schools on a voluntary basis through the Healthy School approach (HS). As of March 2007, 17% of Qu�bec schools had adopted it.9

Despite the recognized potential of these approaches, few studies have evaluated the schools' capacity to implement them.10,11 Research on diffusion of such global approaches is therefore warranted to better assess their feasibility and efficiency in different contexts.12-14 Diffusion studies have consistently reported that key attributes of innovations, as perceived by prospective adopters, are a consistent factor explaining much of the variance in innovations' adoption rates.15-17 However, a review on diffusion of innovations also suggested that these attributes are not sure determinants of the adoption of complex innovations in organizations.16 The purpose of this paper was to assess the psychometric properties of a scale we developed to measure the HS attributes, as perceived by key school players; such an instrument is not yet available, though it could be useful to investigate their influence on HS adoption. Our results may prove valuable to guide strategic changes aimed at dissemination of the approach on a broader scale.

Studies on perceived attributes of innovation15,16 have been widely influenced by Rogers' Diffusion of innovation model17 which suggested 5 "standard" attributes, 4 of which seemed more relevant with regard to HS: 1) Relative advantages, the degree to which an innovation is perceived as an improvement compared to the idea or program it supersedes; 2) Compatibility, the degree to which an innovation is perceived as consistent with existing values, experiences and needs of potential adopters; 3) Complexity, the degree to which an innovation is perceived as complex to understand and use; and 4) Observability of benefits, the degree to which the benefits of an innovation are visible to intended adopters. Recent studies have also highlighted the potential contribution of the adaptability of the innovation, which refers to the potential of an innovation to be adapted or refined by the adopters to suit their needs.16,18 Perceived feasibility to implement an innovation in "real-world" settings may also strongly influence adoption of that innovation.18,19 Consequently, we retained 2 more attributes for scale construction: Beliefs in Collective efficacy to implement the innovation, and perceived Barriers in the school context.16,18

METHODS

Design and participants

Data reported in this article are part of a larger investigation that examined factors helping or hampering dissemination and adoption of HS in Qu�bec and were gathered from a 2007 crosssectional postal survey. Participants in this study were key players of schools, i.e., school principals and school health promotion delegates, belonging to school boards that had given them some information regarding HS. The questionnaire was pre-tested in a sample of 8 participants representing the 2 key players. Adjustments were made after the pre-test. Monetary incentives, prenotification, reminder letters, second questionnaire copy sent at follow-up, and assurance of confidentiality were included to increase response rates.20 Approval from the Ethics Committee of the Universit� du Qu�bec en Outaouais was obtained before conducting the study.

Among the 250 schools selected, 190 (76%) returned at least one questionnaire. A participant response rate of 58% was obtained (i.e., 291 out of 500). The assessment of HS attributes required basic knowledge of the innovation by participants. Furthermore, only participants who answered a minimum of 25 items on the initial 30-item scale were kept. Adoption status of HS had to be based on formal decisions within school to be considered in the study. The final sample included 107 schools and 141 school participants (86 principals and 55 health promotion delegates). Average participant age was 46 � 8.7 years, and the majority (62%) were women. Over half (55%) had a degree in education with 19 � 10.1 years of experience in a school setting.

Measures

The initial scale included 30 items related to the 7 previouslydefined attributes, some adapted from scales employed in similar contexts,21-23 others developed from constructs drawn from the literature on innovation diffusion. Participants had to indicate their level of agreement on a 5-point Likert scale ranging from "strongly agree" to "strongly disagree".

Adoption of HS was the dependent variable and referred to the formal decision of a school to adopt the approach or not. School principals were considered as privileged sources of information for this variable.

Statistical analysis

Prior to any analyses, items with extreme values, presenting skewness coefficients >1 and kurtosis >1, having more than 5 missing values, or not being correlated with at least 1 of the other items (r<0.30), were withdrawn. For exploratory and confirmatory analyses, the sample was divided into two groups, i.e., Group A and Group B, which correspond respectively to school principals and school health promotion delegates.

Exploratory Factor Analysis

Principal component analysis (PCA) was performed on group A (N=86) to investigate the factor structure of the scale. The initial solution was subjected to Varimax rotation with Kaiser's normalization (KMO) and based on known criteria.24

Cronbach?fs alpha was used to establish internal consistency of the subscales with a criterion of alpha acceptability of .0.60.25 Analyses were performed with SPSS, version 15.

Confirmatory Factor Analyses (CFA)

LISREL 8.8 was conducted in group B (N=55) and the overall group (N=141) to validate the model determined by exploratory analyses and to assess model fit, with the following indices: the �2 test (nonsignificant �2 is desirable); the ratio �2 / degree of freedom (df)<2; the comparative fit index (CFI); the Tucker-Lewis index (TLI); the goodness-of-fit index (GFI), where values above 0.90 or 0.95 indicate acceptable and excellent fit, respectively; the root mean square error of approximation (RMSEA); and the root mean square residual (RMR), where values below .08 or .05 indicate acceptable and excellent fit, respectively. 26

Predictive Validity

Logistic regression analysis was undertaken to examine which perceived factors contributed to adoption vs. non-adoption of HS. As adoption is a school-based variable, scores from the 2 key players were averaged for each subscale prior to analysis.

RESULTS

Exploratory factor analysis

Results from the KMO and Bartlett's sphericity analysis showed that the data were suitable for factor analysis. One item with more than 5% of missing values was withdrawn from the 30 items prior to analysis. Iterative factorial analyses were performed to eliminate items strongly saturating on many factors, having low loading or not displaying a sufficient saturation threshold.27 The final analysis included 14 of the 29 items and resulted in a 4-factor solution accounting for 65% of the total variance in scores (Table 1). Factor loadings of the 14 items were all above 0.57.

The first factor of the model was School contextual barriers reflecting constraints related to HS implementation context. This factor explained the greatest percentage of variance (see Table 1). For 2 of the 4 items (#1, #4), the PCA results coincided with the classification operated a priori, while for the 2 others (#2, #3), they were initially classified under Complexity. The second factor was labelled Anticipated benefits, with 4 of its 5 items (#5, #7, #8, #9) congruent with the a priori classification intended to measure the construct Observability of benefits, and 1 item linked to Compatibility (#6). The third factor was related to one component of the HS context. The label Collective efficacy was retained for this factor. The a priori classification differed only for a single item (#11) that was classified as a Complexity attribute. The fourth component Relative advantages corresponded to the a priori classification.

Internal consistency

Reliability was good (α = 0.85) for the factor School contextual barriers and adequate for Anticipated benefits (α = 0.77) as well as for Collective efficacy (α = 0.73) (Table 1). The internal consistency of the fourth factor Relative advantages was lower (α = 0.60).

Confirmatory factorial analyses

The 14-item scale distributed on 4 factors was submitted to CFA on group B and the entire sample to verify the model established by exploratory factor analysis. According to maximum likelihood estimation, the results revealed adequate data fit (Table 2). Indeed, the great majority of adjustment indices respected the required threshold for significance.28 Indices representation quality also showed good adjustment to the empirical data. Correlation coefficients between these items were all higher than 0.30, and residual values were not above 2.58.29

Predictive validity of HS-perceived attributes

Perceived attributes of HS that reliably predicted adoption were School contextual barriers, Anticipated benefits and Collective efficacy (Table 3).

DISCUSSION

The final version of the scale included 14 items capturing 4 attributes of the HS approach. Each subscale was found to have good reliability, though Relative advantages, with only 2 items, had lower internal consistency. The 4-factor model was congruent with some of the constructs of innovation diffusion, but without reproducing exactly the attribute categories found in Rogers' model.16 Hence, Compatibility was not an attribute that stood out. Only 1 of the 3 items retained to measure this factor was kept: "HS is compatible with the wish of Qu�bec schools to establish a partnership between school, family and community", which was linked to the factor Anticipated benefits. Although its content refers to HS compatibility with certain values of the present school reform, it can also be perceived by school participants as an anticipated benefit. Concerning the attribute Observability, items were adapted to reflect that benefits of HS were more anticipated than observable. Items were worded accordingly and most of them were associated with the factor labelled Anticipated benefits. The third attribute of Rogers' model is Complexity. Three of 4 items retained to measure this attribute were found under the School contextual barriers factor in our model. Rogers' Complexity actually refers to 2 aspects of complexity: "difficult to understand" and "difficult to use". The 3 items associated with School contextual barriers concern difficulty in using HS, which is strongly connected to implementation context. Logically, the 2 attributes are therefore related. The item referring to "theoretical" complexity was excluded from the model because of low loading. Finally, all Adaptation items were withdrawn during factor analysis. This attribute might be more relevant during the implementation phase of HS since the actual challenge of transposing the innovation into practice occurs during that phase, when adaptations may be required.

Validity findings showed that Relative advantages was the only factor not significantly associated with adoption although previous studies had consistently identified it as one with great predictive value for adoption. We found that the greatest contribution in predicting HS adoption was made by the factor School contextual barriers. It could be argued, as mentioned by Greenhalgh et al.,16(p. 590) that "interaction between the innovation and its potential context is generally a more valid and useful construct than innovation attributes".

Limitations

Given that our goal was to develop an instrument for school respondents aware of the HS innovation, we constructed our sample accordingly. Because respondents were more likely to be "early adopters", 16 results should be interpreted within this context. Different factors may be more important at different stages of the diffusion process, and the extent to which the scale would be appropriate for these populations should be evaluated in future work.

On the whole, the scale appears to possess good psychometric properties and a conceptually coherent factor structure, which suggests that it has the potential to serve as a research tool to further explore dimensions that could have an impact on adoption of the HS approach. It also provides an indication of the successful early stage of diffusion for this type of innovation which can be tailorbased on the study findings reported here.

[Sidebar]

R�SUM�

Objectifs : L'objectif de cet article est de pr�senter les qualit�s psychom�triques d'un nouvel indice mesurant les attributs per�us d'une approche �cole en sant� (A�S), en termes de structure factorielle, de fid�lit� et de validit� pr�dictive.

M�thodes : Les donn�es proviennent d'une enqu�te postale men�e � l'automne 2007 aupr�s de 107 �coles du Qu�bec et de 141 participants scolaires (directeurs et responsables de la promotion de la sant�). La th�orie de la diffusion de Rogers ainsi que d'autres �crits sur la diffusion des innovations ont servi � la construction de l'indice. Deux analyses, exploratoire et confirmatoire, ont permis de tester la structure factorielle de l'indice.

R�sultats : L'indice obtenu inclut 14 items r�partis en 4 facteurs : contraintes dans l'environnement scolaire, efficacit� collective, b�n�fices anticip�s et avantages relatifs. La consistance interne de ces facteurs varie entre ,85 et ,60. Les trois premiers facteurs ont permis de pr�dire l'adoption de l'A�S.

Conclusion : Globalement, l'indice poss�de de bonnes qualit�s psychom�triques et peut s'av�rer utile pour �valuer l'influence des attributs de ce type d'approche sur son adoption par les �coles.

Mots cl�s : �cole en sant�; innovation; indice; qualit�s psychom�triques

[Reference]

REFERENCES

1. World Health Organization Regional Office for the Western Pacific. Health- Promoting Series 5: Regional Guidelines. Development of Health-Promoting Schools: A Framework for Action. Manila, Philippines: World Health Organization, 1996.

2. Marx E, Wooley SF, Northrop D (Eds.). Health is Academic. A Guide to Coordinated School Health Programs. New York, NY: Teachers College Press, 1998.

3. St Leger L, Nutbeam D. A model for mapping linkages between health and education agencies to improve school health. J Sch Health 2000;70(2):45-50.

4. Parsons C, Stears D, Thomas C. The health promoting school in Europe: Conceptualising and evaluating the change. Health Educ J 1996;55(3):311-21.

5. Hoelscher DM, Evans A, Parcel GS, Kelder SH. Designing effective nutrition interventions for adolescents. J Am Diet Assoc 2002;102(3):S52-S63.

6. James WPT, Rigby NJ, Leach RJ, Kumanyika S, Lobstein T, Swinburn B. Global Strategies to Prevent Childhood Obesity: Forging a Societal Plan That Works. London, UK: International Association for the Study of Obesity/International Obesity Task Force, 2006.

7. Stewart-Brown S. What Is the Evidence on School Health Promotion in Improving Health or Preventing Disease and, Specifically, What Is the Effectiveness of the Health Promoting Schools Approach? Copenhagen, Denmark: World Health Organization, 2006.

8. Warren JM, Henry CJK, Lightowler HJ, Bradshaw SM, Perwaiz S. Evaluation of a pilot school programme aimed at the prevention of obesity in children. Health Promot Int 2003;18(4):287-96.

9. Minist�re de la Sant� et des Services sociaux. Rapport annuel de gestion 2006- 2007 du minist�re de la Sant� et des Services sociaux. Qu�bec, QC: Gouvernement du Qu�bec, 2008.

10. Deschesnes M, Martin C, Jomphe Hill A. Comprehensive approaches to school health promotion: How to achieve broader implementation? Health Promot Int 2003;18(4):387-96.

11. Micucci S. Environmental interventions to improve nutrition and increase physical activity in children and youth. In: Thomas H, Ciliska D, Micucci S, Wilson-Abra J, Dobbins M (Eds.), Effectiveness of Physical Activity Enhancement and Obesity Prevention Programs in Children and Youth. Hamilton, ON: Effective Public Health Practice Project, 2004;223-64.

12. Inchley J, Muldoon J, Currie C. Becoming a health promoting school: Evaluating the process of effective implementation in Scotland. Health Promot Int 2006;22(1):65-71.

13. Barry MM, Domitrovich C, Lara MA. The implementation of mental health promotion programmes. Promot Educ 2005;12(Suppl 2):30-36.

14. Kiefer L, Frank J, Di Ruggiero E, Dobbins M, Manuel D, Gully PR, Mowat D. Fostering evidence-based decision-making in Canada: Examining the need for a Canadian population and public health evidence centre and research network. Can J Public Health 2005;96(3):I1-I19.

15. Greenberg MR. The diffusion of public health innovations. Am J Public Health 2006;96(2):209-10.

16. Greenhalgh T, Robert G, MacFarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Q 2004;82(4):581-629.

17. Rogers EM. Diffusion of Innovations. New York: Free Press, 2003.

18. Oldenburg B, Parcel GS. Diffusion of innovations. In: Glanz K, Rimer BK, Lewis FM (Eds.), Health Behavior and Health Education: Theory, Research, and Practice. San Francisco, CA: Jossey-Bass, 2002;312-34.

19. Sussman S, Valente TW, Rohrbach LA, Skara S, Pentz M-A. Translation in the health professions: Converting science into action. Eval Health Prof 2006;29(1):7-32.

20. Edwards P, Roberts IG, Clarke MJ, Di Guiseppi C, Wentz R, Kwan I, et al. Methods to increase response rates to postal questionnaires. Cochrane Database Syst Rev 2003;2:1-18.

21. Goldman D. Perceptions of innovations as predictors of implementation levels: The diffusion of a nation-wide health education campaign. Health Educ Q 1992;21(4):429-45.

22. Lafferty CK. Diffusion of an asset building innovation in three Portage County school districts: A model of individual change [dissertation]. Ann Arbor, MI: Kent State University, 2001.

23. Parcel GS, O'Hara-Tompkins NM, Harrist RB, Basen-Engquist KM, McCormick LK, Gottlieb NH, Eriksen MP. Diffusion of an effective tobacco prevention program. Part II: Evaluation of the adoption phase. Health Educ Res 1995;10(3):297-307.

24. Altman DG. Practical Statistics for Medical Research. Boca Raton, FL: CRC Press, 1991.

25. Nunnally JC, Bernstein IH. Psychometric Theory, 3rd ed. New York: McGraw- Hill, 1994.

26. Loehlin JC. Latent Variable Models: An Introduction to Factor, Path, and Structural Equation Analysis. Mahwah, NJ: Lawrence Erlbaum Associates, 1998.

27. Stevens J. Applied Multivariate Statistics for the Social Sciences. Mahwah, NJ: Lawrence Erlbaum Associates, 2002.

28. Hu LT, Bentler PM. Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychol Methods 1998;3:424-53.

29. J�reskog KG, S�rbom D. Lisrel 7: A Guide to the Program and Applications. Chicago, IL: SPSS Inc., 1988.

Received: November 27, 2008

Accepted: April 15, 2009

[Author Affiliation]

Marthe Deschesnes, PhD,1 Fran�ois Trudeau, PhD,2 Mababou K�b�, MA3

[Author Affiliation]

Author Affiliations

1. Institut national de sant� publique du Qu�bec and Universit� du Qu�bec en Outaouais, Qu�bec, QC

2. Universit� du Qu�bec � Trois-Rivi�res, Trois-Rivi�res, QC

3. Institut national de sant� publique du Qu�bec, Qu�bec, QC

Correspondence and reprint requests: Dr. Marthe Deschesnes, Institut national de sant� publique du Qu�bec, 945, Wolfe Avenue, Qu�bec (QC) G1V 5B3, Tel: 418- 650-5115, ext. 5540, Fax: 418-650-1064, E-mail: Marthe.Deschesnes@inspq.qc.ca

Acknowledgements: This study was supported by a grant from the Fonds qu�b�cois de la recherche sur la soci�t� et la culture (FQRSC) in collaboration with the Fonds de la recherche en sant� du Qu�bec (FRSQ), the Ministry of Health and Social Services and the Centre de recherche en pr�vention de l'ob�sit�.