news, education,
Not much evidence exists about whether programs designed to boost participation of women and girls in science, technology, engineering and maths really work, but that’s set to change. Evaluation of STEM gender equity will become a requirement for certain government grants in 2021 after a study by an Australian National University researcher found only seven out of a sample of 337 programs had publicly available evaluation data. The Department of Industry, Science, Energy and Resources will ask all recipients of the Women in STEM and Entrepreneurship grants to follow an new evaluation guide. Research associate for the Office of the Women in STEM ambassador and author of the evaluation guide, Isabelle Kingsley, said there was little evidence to show the hundreds of programs to tackle STEM equity issues were effective. “It is a real shame and a big problem that needs to be addressed,” Ms Kingsley said. “Some of the words that people used for evaluation, they said it makes them feel trepidation, that they feel daunted, under qualified, that it’s too time consuming, too expensive, and all those things. They just kind of hear evaluation want to run away.” The lack of accessible data on what worked and what doesn’t work in these initiatives was highlighted in a recently published paper by Dr Merryn McKinnon of the ANU’s Centre for the Public Awareness of Science. The mapping exercise as part of the Australian Academy of Science’s Decadal Plan for Women in STEM found out of 337 programs and initiatives offered nationally over the five years to 2018, only seven had publicly facing evaluation data and only one went beyond self-reports of satisfaction and enjoyment. The most common initiatives were scholarships for tertiary studies followed by student camps, workshops, conferences, networks and competitions. Ms Kingsley said often evaluation was not made public to assist other projects and was not critical enough to be useful. “The problem with those programs that are evaluated a lot of the time they’re seen as like a pat on the back, good news reporting rather than actively trying to evaluate what worked and what didn’t and how to improve it and actually really digging deep into into all the aspects of the program, even looking at unintended consequences.” READ MORE: Some projects have already started to incorporate evaluation into the design of the program, such as the STEM Women database developed by the Australian Academy of Science. Australian Academy of Science spokesperson Madison Hartill-Law said a first-year impact report had been published online. “Apart from evaluating to understand whether it’s having an impact and making a difference around gender equity in Australia, it’s also really important to check in with those women and make sure we’re being valuable to them,” she said. Ms Kingsley said the most effective programs aimed to address underlying systemic and cultural factors that hold women back in scientific careers. “If we’re providing scholarships to get women or girls into STEM then we also need to create a system and a culture that will keep them in there.” She said the next step will be to create an online hub for organisations to publish their evaluations in a central location.
/images/transform/v1/crop/frm/33pRA5ArzT57tWtt8VHHenS/06dfab0f-0bd1-47cf-a796-d1d96be7e082.jpg/r0_299_5616_3472_w1200_h678_fmax.jpg
Not much evidence exists about whether programs designed to boost participation of women and girls in science, technology, engineering and maths really work, but that’s set to change.
Evaluation of STEM gender equity will become a requirement for certain government grants in 2021 after a study by an Australian National University researcher found only seven out of a sample of 337 programs had publicly available evaluation data.
The Department of Industry, Science, Energy and Resources will ask all recipients of the Women in STEM and Entrepreneurship grants to follow an new evaluation guide.
Research associate for the Office of the Women in STEM ambassador and author of the evaluation guide, Isabelle Kingsley, said there was little evidence to show the hundreds of programs to tackle STEM equity issues were effective.
“It is a real shame and a big problem that needs to be addressed,” Ms Kingsley said.
“Some of the words that people used for evaluation, they said it makes them feel trepidation, that they feel daunted, under qualified, that it’s too time consuming, too expensive, and all those things. They just kind of hear evaluation want to run away.”
The lack of accessible data on what worked and what doesn’t work in these initiatives was highlighted in a recently published paper by Dr Merryn McKinnon of the ANU’s Centre for the Public Awareness of Science.
The mapping exercise as part of the Australian Academy of Science’s Decadal Plan for Women in STEM found out of 337 programs and initiatives offered nationally over the five years to 2018, only seven had publicly facing evaluation data and only one went beyond self-reports of satisfaction and enjoyment.
The most common initiatives were scholarships for tertiary studies followed by student camps, workshops, conferences, networks and competitions.
Ms Kingsley said often evaluation was not made public to assist other projects and was not critical enough to be useful.
“The problem with those programs that are evaluated a lot of the time they’re seen as like a pat on the back, good news reporting rather than actively trying to evaluate what worked and what didn’t and how to improve it and actually really digging deep into into all the aspects of the program, even looking at unintended consequences.”
Some projects have already started to incorporate evaluation into the design of the program, such as the STEM Women database developed by the Australian Academy of Science.
Australian Academy of Science spokesperson Madison Hartill-Law said a first-year impact report had been published online.
“Apart from evaluating to understand whether it’s having an impact and making a difference around gender equity in Australia, it’s also really important to check in with those women and make sure we’re being valuable to them,” she said.
Ms Kingsley said the most effective programs aimed to address underlying systemic and cultural factors that hold women back in scientific careers.
“If we’re providing scholarships to get women or girls into STEM then we also need to create a system and a culture that will keep them in there.”
She said the next step will be to create an online hub for organisations to publish their evaluations in a central location.
There is scant publicly available evidence on whether programs which encourage women and girls into careers in STEM are effective. Picture: Shutterstock
