D4 Published development or research report or study
Kiusaamisen vastaisten menetelmien arviointi : seitsemän arviointiin valitun menetelmän käytettävyys, juurrutettavuus ja tuloksellisuus (2023)
Evaluation of anti-bullying methods : usability, sustainability and outcomes of seven methods selected for the evaluation
Rumpu, N., Markkanen, E.-L., Hyvärinen, N., Anttila, N., Danschu, P., Kuvaja, M., Romantschuk, M., & Sainio, M. (2023). Kiusaamisen vastaisten menetelmien arviointi : seitsemän arviointiin valitun menetelmän käytettävyys, juurrutettavuus ja tuloksellisuus. Kansallinen koulutuksen arviointikeskus. Julkaisut / Kansallinen koulutuksen arviointikeskus, 11:2023. https://www.karvi.fi/sites/default/files/sites/default/files/documents/KARVI_1123.pdf
JYU authors or editors
Publication details
All authors or editors: Rumpu, Niina; Markkanen, Eeva-Liisa; Hyvärinen, Nelli; Anttila, Niina; Danschu, Pekka; Kuvaja, Marko; Romantschuk, Micaela; Sainio, Miia
ISBN: 978-952-206-761-6
eISBN: 978-952-206-762-3
Journal or series: Julkaisut / Kansallinen koulutuksen arviointikeskus
ISSN: 2342-4176
eISSN: 2342-4184
Publication year: 2023
Publication date: 06/04/2023
Number in series: 11:2023
Number of pages in the book: 260
Publisher: Kansallinen koulutuksen arviointikeskus
Place of Publication: Helsinki
Publication country: Finland
Publication language: Finnish
Persistent website address: https://www.karvi.fi/sites/default/files/sites/default/files/documents/KARVI_1123.pdf
Publication open access: Openly available
Publication channel open access: Open Access channel
Publication is parallel published (JYX): https://jyx.jyu.fi/handle/123456789/92915
Additional information: Sammanfattning: Utvärdering av antimobbningsmetoder : utvärdering av sju utvalda metoder i fråga om användbarhet, förankring och resultat
Abstract
In 2021–2022, the Finnish Education Evaluation Centre (FINEEC) carried out an evaluation of anti-bullying methods. The evaluation is part of the Ministry of Education and Culture’s operational programme for the prevention of bullying, violence and harassment.
In this evaluation, anti-bullying methods mean operating models or programmes that are aimed at preventing bullying, intervening in bullying, or both and that are organised, developed and monitored by external actors instead of the schools themselves. In this evaluation, anti-bullying measures mean bullying prevention, intervention and aftercare. The purpose of aftercare is to ensure appropriate support after an instance of bullying. In addition to anti-bullying measures, methods can also include broader measures aimed at strengthening wellbeing and peaceful working conditions at school.
The evaluation provides information about the good practices and operating methods of the evaluated anti-bullying methods and highlights areas that both the schools using the methods and the actors coordinating them should pay attention to in order to ensure that the methods work as intended. The purpose of the evaluation is not to set the different anti-bullying methods against each other or compare their superiority, but to disseminate information on the effective practices of different methods and highlight the needs to develop them.
In autumn 2021, FINEEC organised an open call for all actors coordinating anti-bullying methods used in basic education in Finland. The total number of applications received was 27. Based on pre-determined evaluation criteria, a total of seven methods were selected for the evaluation. Because the aim of the evaluation was to produce information on different methods and their suitability to different situations, methods focusing on different anti-bullying measures and different needs were selected for the evaluation. The choice of methods was also influenced by their use in Finland: only methods that are currently used in Finnish schools were deemed eligible for the evaluation. The selected methods were Gutsy Go, B-Stop, KiVa Koulu, the Safety Skills Education model of Oulu, the Peer Support programme, mediation at school (Verso and Resto) and peer support activities.
The anti-bullying methods selected for the evaluation are different in terms of their objectives, influencing mechanisms and operating methods. The starting points and framework of each method have been taken into account in the evaluation. The methods were evaluated through three cross-cutting terms that were formulated to implement the evaluation: the usability of the method, sustainability and the outcome. These themes were defined utilising recent research data, according to which effective anti-bullying methods have certain factors in common.
The evaluation data consisted of self-evaluations carried out by the actors coordinating the methods included in the evaluation, instructions and materials related to the use of the methods and school surveys in which school staff, pupils and guardians evaluated both their school’s anti-bullying measures in general and one of the methods included in the evaluation in particular. Responses to the survey were received from 245 members of school staff, 12,081 pupils and 2,111 guardians. In addition to this, supplementary data was collected through group interviews with school staff and pupils. This evaluation report presents results regarding the usability, sustainability and outcomes of the methods included in the evaluation.
Anti-bullying measures are significantly influenced by the operating culture of the school, which is not always consistent with the anti-bullying method used by the school. According to the evaluation results, schools apply anti-bullying methods and link them to each other in various ways. Almost one third of schools reported using more than one method as their primary anti-bullying method. This makes it challenging to evaluate the outcomes of individual methods and the anti-bullying measures of schools solely on the basis of the used method. Although school staff, pupils and guardians are primarily satisfied with the anti-bullying work of schools, continuous development of operating methods is needed. Particular attention should be paid to ensuring that subtle, indirect bullying is also intervened in consistently.
According to the evaluation, the usability of the methods was contributed to by clear instructions and easy-to-use materials suitable for the school. The usability of methods was deemed stronger in schools where the role of the evaluated method was deemed significant in terms of bullying prevention and intervention.
The sustainability of methods in schools was facilitated by staff that was committed to the use of the method, support from management and the integration of the method into other school operations and efforts to improve wellbeing. Unfortunately, in many schools responsibility for using anti-bullying methods rested with only a few individuals, making the integration of the method into the everyday operations of the school more challenging. This also puts the continued use of the method at risk due to possible staff changes. The sustainability of anti-bullying methods was stronger in schools where the role of the method in bullying prevention work was clear.
Anti-bullying work requires schools to constantly monitor situations and evaluate and develop their operations. However, according to the evaluation results, the evaluation of the use of anti-bullying methods and the development of anti-bullying measures were limited in many schools. Even in schools where a particular anti-bullying method has been used for a long time and its measures have been integrated into everyday operations, it is important to keep evaluating measures systematically and continue developing them.
Based on the evaluation results, the following recommendations were issued:
1.
Schools should clarify the role of anti-bullying methods in bullying prevention work.
To make the bullying prevention efforts of schools more structured, they should be made a part of schools’ annual plans. Different functions and events aimed at increasing community cohesion and participation should be included in schools’ annual plans to make them a natural part of the everyday operations of schools and prevention work. Sufficient amount of lessons or hourly resources should be reserved for the use of anti-bullying methods and going over matters related to them. However, it is important to keep in mind that, in addition to individual lessons, bullying prevention should also be integrated into the everyday operation and teaching of schools.
According to the evaluation data, pupils and guardians are not sufficiently aware of how bullying is prevented in schools. From the perspective of preventive work, it is important to define what is meant by the preventive work of the school, what it encompasses and what is the role of the used anti-bullying method in this work. As regards bullying prevention work, it is essential for pupils and guardians to be aware of the measures aimed at preventing bullying at school. It is also important for schools to provide pupils and guardians with information about these measures so as to increase their understanding of the complexity of the bullying phenomenon.
2.
Bullying intervention methods should be made visible.
According to the evaluation data, many pupils and guardians were not aware of how schools intervene in bullying. This can contribute to pupils being too afraid to report bullying that they experience or witness because they are unaware of how instances of bullying are handled by the school. The fact is that school staff do not notice all instances of bullying, and according to the evaluation data, pupils find it difficult to report online bullying, in particular.
Schools should inform both guardians and pupils of how reported bullying is intervened in and of the methods used to investigate situations. Schools should also ensure that the entire school community operates consistently and uses mutually agreed-upon methods to handle instances of bullying.
3.
Schools should reserve sufficient resources for anti-bullying work.
It is important to ensure that schools reserve a sufficient amount of time for carrying out anti-bullying work. It is also important for schools to consider the time frame of their bullying prevention efforts. Sufficient time must also be reserved for investigating bullying situations, especially in cases of prolonged bullying. It is also important to strengthen the knowledge related to this matter in schools.
Education providers should allocate sufficient personnel resources for schools so that their bullying prevention efforts are effective. Even if responsibility for the use of an anti-bullying method has been assigned to specific people, it is important for schools to ensure that the entire school community takes responsibility for anti-bullying work. School management also plays an important role in ensuring that the staff is committed to shared operating methods and in directing anti-bullying work. According to the evaluation results, in many schools the responsibility for the use of an anti-bullying method rested with only a few designated persons, and in many schools it was felt that it is difficult to recruit persons who could be assigned responsibility for a method. One of the factors contributing to this could be the amount of work involved, which is not always subject to sufficient separate compensation. With this in mind, education providers should guarantee sufficient financial resources for schools that use anti-bullying methods to ensure that the assignment of responsibility for methods is carried out equally and fairly.
4.
The concurrent use of different anti-bullying methods should be more effectively considered in their development.
According to the evaluation data, schools are satisfied with the anti-bullying methods that they use, and many schools use several anti-bullying methods concurrently. One factor contributing to challenges related to the adoption of anti-bullying methods at schools is their laboriousness, which may lead to schools adapting methods to their own needs. As such, it is important for the actors coordinating the methods to provide schools with support for their adoption, in addition to which schools should also consider which factors support their adoption of anti-bullying methods.
To facilitate the commitment of schools to using anti-bullying methods, the actors coordinating the methods should consider whether it would be possible to define minimum objectives for the use of the methods to facilitate their adoption at schools. Since schools use several methods concurrently, it is also worth considering how this concurrent use could be facilitated and how schools could apply the methods without compromising the desired outcomes.
5.
The continuous evaluation of the methods should be developed.
According to the evaluation results, schools make rather little use of the evaluation material that the methods provide for evaluating measures. Especially in the case of methods that have been used for a long time, it is important to also consider how schools could develop measures. Many schools only collect oral feedback about measures. However, according to the evaluation results, more systematic data collection provided schools with good opportunities to also develop their own operations.
If the anti-bullying work of schools is to be based on evidence-based methods, the methods need to be developed through continuous research. This can only be done if sufficient resources and funding is reserved for this purpose. Many of the methods included in the evaluation were primarily focused on strengthening community cohesion for the purpose of preventing bullying. From a research perspective, it is also worth considering what kind of approaches are suitable for evaluating different methods. In particular, more information is needed about how methods supporting participation and community cohesion affect the community as a whole.
6.
The role of pupils and guardians in the anti-bullying work of schools should be strengthened.
One of the factors that affect bullying are the norms of the peer group, which are not always visible to adults. According to the evaluation results, pupils would like to be more involved in the anti-bullying work of their own schools. The involvement of pupils is relevant from the point of view of the outcomes of anti-bullying work, as the atmosphere at school and the prevailing social norms among pupils are important factors affecting bullying, information on which can be obtained primarily from the pupils themselves. Bullying is a complex group phenomenon, with the members of the group playing a crucial role in whether it continues. As such, the school community should provide sufficient opportunities and space for the pupils’ own activities that strengthen anti-bullying norms among pupils and support everyone being able to feel like an important member of the school community.
According to the evaluation results, nearly two out of five guardians would have liked to be more involved in the anti-bullying work of schools. Instead of carrying out one-way communications, schools should consider different ways of involving guardians in their measures. This involvement could consists of e.g. in-depth discussion about the bullying phenomenon, with the aim of building understanding between guardians and schools on bullying, its prevention and what should be done if bullying occurs. Especially in terms of preventing online bullying, it is important for pupils and guardians to also participate in the anti-bullying work of schools.
In this evaluation, anti-bullying methods mean operating models or programmes that are aimed at preventing bullying, intervening in bullying, or both and that are organised, developed and monitored by external actors instead of the schools themselves. In this evaluation, anti-bullying measures mean bullying prevention, intervention and aftercare. The purpose of aftercare is to ensure appropriate support after an instance of bullying. In addition to anti-bullying measures, methods can also include broader measures aimed at strengthening wellbeing and peaceful working conditions at school.
The evaluation provides information about the good practices and operating methods of the evaluated anti-bullying methods and highlights areas that both the schools using the methods and the actors coordinating them should pay attention to in order to ensure that the methods work as intended. The purpose of the evaluation is not to set the different anti-bullying methods against each other or compare their superiority, but to disseminate information on the effective practices of different methods and highlight the needs to develop them.
In autumn 2021, FINEEC organised an open call for all actors coordinating anti-bullying methods used in basic education in Finland. The total number of applications received was 27. Based on pre-determined evaluation criteria, a total of seven methods were selected for the evaluation. Because the aim of the evaluation was to produce information on different methods and their suitability to different situations, methods focusing on different anti-bullying measures and different needs were selected for the evaluation. The choice of methods was also influenced by their use in Finland: only methods that are currently used in Finnish schools were deemed eligible for the evaluation. The selected methods were Gutsy Go, B-Stop, KiVa Koulu, the Safety Skills Education model of Oulu, the Peer Support programme, mediation at school (Verso and Resto) and peer support activities.
The anti-bullying methods selected for the evaluation are different in terms of their objectives, influencing mechanisms and operating methods. The starting points and framework of each method have been taken into account in the evaluation. The methods were evaluated through three cross-cutting terms that were formulated to implement the evaluation: the usability of the method, sustainability and the outcome. These themes were defined utilising recent research data, according to which effective anti-bullying methods have certain factors in common.
The evaluation data consisted of self-evaluations carried out by the actors coordinating the methods included in the evaluation, instructions and materials related to the use of the methods and school surveys in which school staff, pupils and guardians evaluated both their school’s anti-bullying measures in general and one of the methods included in the evaluation in particular. Responses to the survey were received from 245 members of school staff, 12,081 pupils and 2,111 guardians. In addition to this, supplementary data was collected through group interviews with school staff and pupils. This evaluation report presents results regarding the usability, sustainability and outcomes of the methods included in the evaluation.
Anti-bullying measures are significantly influenced by the operating culture of the school, which is not always consistent with the anti-bullying method used by the school. According to the evaluation results, schools apply anti-bullying methods and link them to each other in various ways. Almost one third of schools reported using more than one method as their primary anti-bullying method. This makes it challenging to evaluate the outcomes of individual methods and the anti-bullying measures of schools solely on the basis of the used method. Although school staff, pupils and guardians are primarily satisfied with the anti-bullying work of schools, continuous development of operating methods is needed. Particular attention should be paid to ensuring that subtle, indirect bullying is also intervened in consistently.
According to the evaluation, the usability of the methods was contributed to by clear instructions and easy-to-use materials suitable for the school. The usability of methods was deemed stronger in schools where the role of the evaluated method was deemed significant in terms of bullying prevention and intervention.
The sustainability of methods in schools was facilitated by staff that was committed to the use of the method, support from management and the integration of the method into other school operations and efforts to improve wellbeing. Unfortunately, in many schools responsibility for using anti-bullying methods rested with only a few individuals, making the integration of the method into the everyday operations of the school more challenging. This also puts the continued use of the method at risk due to possible staff changes. The sustainability of anti-bullying methods was stronger in schools where the role of the method in bullying prevention work was clear.
Anti-bullying work requires schools to constantly monitor situations and evaluate and develop their operations. However, according to the evaluation results, the evaluation of the use of anti-bullying methods and the development of anti-bullying measures were limited in many schools. Even in schools where a particular anti-bullying method has been used for a long time and its measures have been integrated into everyday operations, it is important to keep evaluating measures systematically and continue developing them.
Based on the evaluation results, the following recommendations were issued:
1.
Schools should clarify the role of anti-bullying methods in bullying prevention work.
To make the bullying prevention efforts of schools more structured, they should be made a part of schools’ annual plans. Different functions and events aimed at increasing community cohesion and participation should be included in schools’ annual plans to make them a natural part of the everyday operations of schools and prevention work. Sufficient amount of lessons or hourly resources should be reserved for the use of anti-bullying methods and going over matters related to them. However, it is important to keep in mind that, in addition to individual lessons, bullying prevention should also be integrated into the everyday operation and teaching of schools.
According to the evaluation data, pupils and guardians are not sufficiently aware of how bullying is prevented in schools. From the perspective of preventive work, it is important to define what is meant by the preventive work of the school, what it encompasses and what is the role of the used anti-bullying method in this work. As regards bullying prevention work, it is essential for pupils and guardians to be aware of the measures aimed at preventing bullying at school. It is also important for schools to provide pupils and guardians with information about these measures so as to increase their understanding of the complexity of the bullying phenomenon.
2.
Bullying intervention methods should be made visible.
According to the evaluation data, many pupils and guardians were not aware of how schools intervene in bullying. This can contribute to pupils being too afraid to report bullying that they experience or witness because they are unaware of how instances of bullying are handled by the school. The fact is that school staff do not notice all instances of bullying, and according to the evaluation data, pupils find it difficult to report online bullying, in particular.
Schools should inform both guardians and pupils of how reported bullying is intervened in and of the methods used to investigate situations. Schools should also ensure that the entire school community operates consistently and uses mutually agreed-upon methods to handle instances of bullying.
3.
Schools should reserve sufficient resources for anti-bullying work.
It is important to ensure that schools reserve a sufficient amount of time for carrying out anti-bullying work. It is also important for schools to consider the time frame of their bullying prevention efforts. Sufficient time must also be reserved for investigating bullying situations, especially in cases of prolonged bullying. It is also important to strengthen the knowledge related to this matter in schools.
Education providers should allocate sufficient personnel resources for schools so that their bullying prevention efforts are effective. Even if responsibility for the use of an anti-bullying method has been assigned to specific people, it is important for schools to ensure that the entire school community takes responsibility for anti-bullying work. School management also plays an important role in ensuring that the staff is committed to shared operating methods and in directing anti-bullying work. According to the evaluation results, in many schools the responsibility for the use of an anti-bullying method rested with only a few designated persons, and in many schools it was felt that it is difficult to recruit persons who could be assigned responsibility for a method. One of the factors contributing to this could be the amount of work involved, which is not always subject to sufficient separate compensation. With this in mind, education providers should guarantee sufficient financial resources for schools that use anti-bullying methods to ensure that the assignment of responsibility for methods is carried out equally and fairly.
4.
The concurrent use of different anti-bullying methods should be more effectively considered in their development.
According to the evaluation data, schools are satisfied with the anti-bullying methods that they use, and many schools use several anti-bullying methods concurrently. One factor contributing to challenges related to the adoption of anti-bullying methods at schools is their laboriousness, which may lead to schools adapting methods to their own needs. As such, it is important for the actors coordinating the methods to provide schools with support for their adoption, in addition to which schools should also consider which factors support their adoption of anti-bullying methods.
To facilitate the commitment of schools to using anti-bullying methods, the actors coordinating the methods should consider whether it would be possible to define minimum objectives for the use of the methods to facilitate their adoption at schools. Since schools use several methods concurrently, it is also worth considering how this concurrent use could be facilitated and how schools could apply the methods without compromising the desired outcomes.
5.
The continuous evaluation of the methods should be developed.
According to the evaluation results, schools make rather little use of the evaluation material that the methods provide for evaluating measures. Especially in the case of methods that have been used for a long time, it is important to also consider how schools could develop measures. Many schools only collect oral feedback about measures. However, according to the evaluation results, more systematic data collection provided schools with good opportunities to also develop their own operations.
If the anti-bullying work of schools is to be based on evidence-based methods, the methods need to be developed through continuous research. This can only be done if sufficient resources and funding is reserved for this purpose. Many of the methods included in the evaluation were primarily focused on strengthening community cohesion for the purpose of preventing bullying. From a research perspective, it is also worth considering what kind of approaches are suitable for evaluating different methods. In particular, more information is needed about how methods supporting participation and community cohesion affect the community as a whole.
6.
The role of pupils and guardians in the anti-bullying work of schools should be strengthened.
One of the factors that affect bullying are the norms of the peer group, which are not always visible to adults. According to the evaluation results, pupils would like to be more involved in the anti-bullying work of their own schools. The involvement of pupils is relevant from the point of view of the outcomes of anti-bullying work, as the atmosphere at school and the prevailing social norms among pupils are important factors affecting bullying, information on which can be obtained primarily from the pupils themselves. Bullying is a complex group phenomenon, with the members of the group playing a crucial role in whether it continues. As such, the school community should provide sufficient opportunities and space for the pupils’ own activities that strengthen anti-bullying norms among pupils and support everyone being able to feel like an important member of the school community.
According to the evaluation results, nearly two out of five guardians would have liked to be more involved in the anti-bullying work of schools. Instead of carrying out one-way communications, schools should consider different ways of involving guardians in their measures. This involvement could consists of e.g. in-depth discussion about the bullying phenomenon, with the aim of building understanding between guardians and schools on bullying, its prevention and what should be done if bullying occurs. Especially in terms of preventing online bullying, it is important for pupils and guardians to also participate in the anti-bullying work of schools.
Keywords: primary and lower secondary education; bullying; school bullying; pre-emption; early intervention; well-being
Contributing organizations
Ministry reporting: Yes
VIRTA submission year: 2023
