How can I tell if my outreach to women is effective?

How do you know that volunteer work you’re doing to encourage women to enter technology is effective? I get asked fairly often to volunteer from programs such as technology summer camps, mentoring, promotional websites and science fairs, and I’ve often wondered whether they are worthwhile.

While I like to think that every little bit helps, the fact is that programs targeted towards youth often have unintended consequences. Take the example of “girls in technology summer camp”. Maybe girls that attend technology summer camp would rather spend their vacation doing other things, and walk away annoyed and less inclined towards careers in technology than they were before. Maybe the girls that choose to attend are all girls that were planning to enter [science, technology, engineering and mathematics (STEM)] careers in the first place, and it has no effect. Even worse, maybe summer camps cause girls to make poorly-informed decisions to go into technical careers. Maybe they present an incomplete view which causes attendees to pursue STEM degrees, only to drop out when they realize what the degree is really about. Maybe they don’t realistically present the aptitudes required, and encourage attendees who are unlikely to succeed to pursue certain careers.

While the outcomes above might not be likely, they are possible, and as I do more of this type of work, I’d like to better understand its impact. As a minimum, I’d like to make sure that my work isn’t actually discouraging girls from entering the field, and at best I’d like to figure out what type of activity, per hour spent, has the most impact.

Does anyone know of any research on this subject, or have any thoughts in general?

Firstly, kudos for getting involved in outreach work – it’s great that you’re so committed to helping encourage the next generation of women into exciting and rewarding STEM careers!

Secondly, this is an interesting question. One of my favourite things about science in general and scientists in particular is the desire for evidence to support their stances. It’s perfectly reasonable to inquire about effective outreach and indeed, any outreach programme worth its salt should be asking the questions ‘is this effective?’ and ‘how do we make it more so?’. Unfortunately, those questions then lead to the meta-questions of ‘how do we define effectiveness?’ and ‘how do we measure it?’, which are far more woolly and hard to pin down.

From your question it seems like you have several different criteria for what could make a programme effective, and none of these are invalid. It could be that it provides an accurate, well-rounded view of what a particular degree is like in order to appeal to the ‘right kind’ of person, or it encourages everybody regardless of pre-existing interest to consider STEM subjects. These of course are two different aims, and it’s important to consider which we want our outreach to achieve when we’re planning and executing it. Is one necessarily better than the other?

Add to the mix the fact that there are so many types of outreach you could be doing and that they all link to different intended outcomes and it becomes very hard to measure them all to one standard. An outreach programme for a particular engineering school within a university may consider ‘success’ to be an increase in women applying for their school, whereas a wider-reaching, more generalised outreach programme may measure success in the number of woman who tick ‘I am more interested in science than I was before’ after attending an activity. The outcomes are also rather self-selecting; a summer camp may look more successful at recruiting young people into STEM subjects, but usually because the people who choose to attend them are already interested in those subjects to begin with. So unfortunately the question about which activity has the most impact per hour spent is nearly impossible to answer, at least when considering the wide range of possible outreach activities and the huge range of audiences and goals.

It is possible to reflect a little on why women choose to study STEM subjects and reverse-engineer the sort of outreach which would provide those influences. In this article a variety of factors are highlighted that show things like spending time outdoors and doing math problems and logic games were more positively influential for girls than boys. These are things that could possibly be worked into an outreach programme to make STEM subjects even more enticing to women. An even bigger influence seems to be classes and teachers – something I will touch upon a little later.

The good news is that even without rigorous data it’s possible to make sure your work is not in vain; instead of worrying about trying to find the most effective outreach activity, I’d recommend instead maximising effectiveness with the outreach you’re already doing by following these steps:

Remember what drew you to STEM in the first place and share your passion and enthusiasm.

The best spokespeople for STEM subjects will always be the people who are passionate about what they do and are able to share that with the people around them. Make sure you communicate that passion well, also — work on your presentation and communication skills so you can get your message across clearly and enjoyably for your audience.

Work with a programme whose methods and aims you agree with/enjoy.

There are lots of different outreach approaches out there run by a variety of groups and people*. Find the one you enjoy most; as above, you’re going to be most effective when you’re having fun. It’s better to have a positive impact on a smaller group (for example) than to force yourself to work with large crowds and to be miserable the whole time.

Evaluate your programme and look for ways to improve it.

Good outreach programmes should always have a clearly-defined evaluation programme to monitor effectiveness and give means for improvement. If the programme you’re working with doesn’t have this, work with them to create one!

Talk to the people you’re working with and ask their opinions.

The best way of figuring out how effective you’re being is to ask the people you’re communicating with, right then and there. Talk to them about what they find interesting about science (or maths, or engineering, or technology) and how you can help them explore that. Ask them what they feel the barriers are, what sort of support would help them, and what sort of things they’d like to see or do in the future. Feedback like that can be immediate and very helpful, especially when paired with effective formal evaluation.

Outreach can be effective on an individual level; aim for that as much as a general effectiveness.

While there’s a lot to be said about raising the awareness of the importance of STEM with the general public, when it comes down to it you’re really trying to affect individuals. Try and connect with the people you’re working with, find out what makes them tick and share your experiences. If someone is swithering or unsure about their options all it might take is one good personal experience to encourage them to pursue technology over another option.

Consider other options you might not have thought of.

We all know that working with young people is an effective way of encouraging more people into STEM, but think about other ways you might be able to positively influence their experiences. It’s been shown that good classroom experiences positively affect the uptake of STEM subjects later in life, so providing science and technology teachers with training which gives them more confidence in the classroom might actually positively affect a whole class. Providing equipment for clubs or access to journals for students can also contribute to a better experience in schools and an increased likelihood of STEM uptake.

Lastly, all the outreach in the world is only worth so much if the environment that recruits are entering is unwelcoming or toxic to them.

Therefore it’s also incredibly important to improve the STEM environment for women in order to retain them after they’ve entered. Work with your company or university to support women entering the area and make sure your voice is heard on important matters of monitoring and equality!

2 thoughts on “How can I tell if my outreach to women is effective?”

Thanks so much for putting this together. The advice for how to pick an outreach activity where each individual can be maximally effective is really useful, and I love the list of resources and references.