Costs and benefits of reaching parents for survey participation through school communication systems: testing the potential of a straightforward convenience sample

Vercruyssen A. (2014). Costs and benefits of reaching parents for survey participation through school communication systems: testing the potential of a straightforward convenience sample, Survey Methods: Insights from the Field. Retrieved from http://surveyinsights.org/?p=4292.

Abstract

Drawing a random sample for surveys from official registers is an expensive and time-consuming procedure in Belgium as in other countries. In order to obtain a sample for a study on work-family life balance amongst working mothers in Flanders (the Dutch-speaking part of Belgium), I opted for a promising, economical procedure by first selecting children in schools and day-care facilities. Given the compulsory nature of schooling in Belgium and the widespread use of day-care facilities, this offers a straightforward ‘convenience’ sampling for reaching working parents, and mothers in particular. In this paper, I describe the details and advantages of this cost-effective sampling strategy and the experiences with this test case. I also discuss the possible methodological downsides and how to avoid them in future research.

Copyright

Drawing random samples for surveys from official registers can be an expensive, time-consuming endeavour. The costs and time required to complete the procedure can demand significant resources – resources that are often not available to researchers. Therefore, I experimented a different sampling procedure for a study on work-family balance among working women: sampling mothers via their children’s schools and day-care facilities. Based on the simple logic that where there are children, there are usually mothers, I selected an area in Flanders (the Dutch-speaking part of Belgium) in which I used schools and day-care facilities as the sample frame.

Gaining the co-operation of schools and day-care organizations and being allowed to use their official communication systems for written contact turned out to be a relatively cost-effective strategy for reaching a broad sample of mothers. However, privacy concerns of the school boards led to some unexpected methodological challenges for this convenience sample. I present the lessons learned from using the official communication systems of schools to reach parents, or in this case mothers, and discuss how this type of sampling can be optimized.

1. A straightforward frame for reaching parents

I selected parents with children in day care, pre-school, primary school or secondary school in the selected suburban region. In Flanders, this is a cost-effective way to select parents for various reasons: the common practice of taking babies and toddlers to day care, the very widespread attendance of toddlers in pre-school, and the mandatory nature of primary and secondary school education in the whole of Belgium.

Although school is only compulsory between the ages of 6 and 18 in Belgium, almost all Flemish children from the age of three attend pre-school (97.8% in 2007, Flemish Ministry of Education and Training, 2008; 2011). Pre-schools, primary and secondary schools in Belgium are free from entrance fees. However, day-care organizations do cost money, which combined with the rising number of working mothers relying on such child-care services (Vanpée et al., 2003) could explain why it is nowadays more common for children in Flanders to start free pre-school at two and a half years of age rather than staying another six months in costly day care facilities. Moreover, during the last decade, the Ministry of Education and Training (2007) has also focused more on the age of two and a half rather than three as the starting point for pre-school participation. This means that almost all Flemish children from the age of two and a half or three onwards are ‘reachable’ through their schools.

With regard to children up to three years of age, the use of day-care organizations from the age of three months is already relatively common practice in Belgium (Vanpée et al., 2003). This starting age is consistent with the 15 weeks of maternity leave for Belgian (working) mothers. In 2004, approximately 56% of all children up to the age of three in Flanders attended day care on a regular basis, by 2009 this was 63.2% and the number continues to rise every year (Hedebouw & Peetermans, 2009). This means that about two thirds of Flemish children between three months and three years are reachable through day-care facilities.

With the widespread use of day-care organizations for babies and toddlers, the common practice of toddlers attending pre-school in Flanders from the age of two and a half (or three) onwards, and compulsory education from 6 to 18 years of age, sampling day-care facilities, pre-schools, primary and secondary schools is a very straightforward way to sample parents in Flanders. In comparison with going through a long, costly procedure to obtain a random sample from the population register, sampling parents through these institutions is incredibly fast. Lists of schools are freely available on the Internet, for example the website of the Flemish Ministry of Education and Training. Day-care organizations are also easy to find online nowadays. Most towns and cities in Flanders now also provide lists online and even have websites completely devoted to helping parents find a suitable day-care facility in their district.

1.2. A suburban test case focussing on mothers

For a study on how working mothers combine work and family life, I chose a suburban area in the province of East Flanders to test this sampling strategy. The majority of the inhabitants of Flanders lives in suburban regions, such as the one chosen (Kesteloot, 2003). The selected area consists of three adjacent villages, surrounded by two medium-sized cities and two medium-sized municipalities. This was chosen for the test case, because the combined size of the villages was large enough to have a sizeable number of schools and day-care facilities within one suburban area. The selected sampling area includes seven pre-school and primary schools, one large secondary school, four formally registered day-care facilities (accommodating a minimum of nine children) and three privately registered day-care businesses (one-woman facilities accommodating up to eight children).

This test case employs two-stage cluster sampling. I first selected a geographic area in which all the schools and day care organizations were located. Within these schools and day-care facilities, all mothers were selected for the sample. The limitation of the geographical area was necessary for the test case, in order to keep the sample size manageable in view of available resources. The limitation of only including mothers was due to the nature of the research project, focussing on how work life and family life are balanced by mothers rather than by parents in general. Both limitations were specific to this test case. However, the strategy of sampling parents through schools and day-care organizations can obviously be used on a nationwide scale and for fathers as well as mothers. Accordingly, this is a promising, cost-effective contact procedure for reaching mothers (or parents in general) with some possible methodological challenges.

2. A promising, economical contact procedure…

To contact all the mothers, I obtained permission to use the official communication systems of the selected schools and day-care organizations. The communication systems of the larger day-care facilities tend to work by using a small ‘post box’ for each set of parents or a ‘diary’ booklet in which the day of each child is briefly described. It is a common practice to put letters in the diary or post box and parents know that important written communications come through these channels. In the private day-care facilities, it was more usual to make an agreement with the organizer to give the introductory letter directly to the parents if a post box or diary system was not available.

The communication systems of the schools all work with a diary/agenda for each child. In pre-school, this often resembles the system used by day-care organizations. Pre-school teachers usually expect parents to sign the page of the previous week’s activities during the weekend, to acknowledge that they have read all important messages noted in or provided with the diary. For children in primary and secondary school, there is also a requirement for agendas to be signed weekly by parents to acknowledge they have received (and read) all the listed communications. Teachers check on a weekly basis whether parents have signed the communication sheet and reprimand children if a signature is missing.

This means the communication systems of the schools are similar to recorded post, as a signature is required to confirm receipt of letters. For the day-care institutions, using their communication system is more similar to using regular post. The major advantage of both communication systems is that there are no postage costs, which saves significant financial resources. A downside is that there is the possibility of some letters being lost by the children. The risk of losing letters of course also exists when using regular postal services. Specific measures might also be necessary in the case of divorced parents. When targeting both parents, using separate envelopes addressed to the mother and father for all children is the simplest starting point. The co-operation of the schools is also essential if both divorced parents need to be targeted, especially if there are specific custody arrangements.

3. …with some possible methodological challenges

Although this sampling strategy has many benefits compared with obtaining a random sample from the official population register, and can save significant financial resources (and time) compared with a regular (registered) postal survey, I did come across some unexpected methodological challenges. I list these challenges and also make suggestions about how to avoid them in future research. Overall, I wish to show how this type of convenience sampling of parents can be an economical and straightforward way to obtain a viable sample for research.

3.1. Two-stage cluster sampling and first stage non-response

As we are dealing with two-stage cluster sampling, refusal can occur at the cluster level as well as the individual level. The schools or day-care organizations act as gatekeepers in terms of accessing potential respondents, which is the first hurdle to deal with in the sampling strategy. However, in this test case almost all the schools and day-care facilities co-operated and had a positive attitude towards the survey. Only one school refused – the smallest primary school (< 40 pupils) in the area. This refusal was due to privacy concerns of the parent-teacher association (PTA), although some kind of power struggle between the PTA and the principal (who was in favour of participation) seems to have contributed to the refusal.

With regard to the day-care organisations, only one private facility (with up to eight children) refused. The day-care mother was “not interested” and did not want us to contact parents through her. The second contact ended with an even clearer refusal. The other two private day-care businesses and four day-care organisations – two ‘mini crèches’ (up to 23 children) and two ‘day-care centres’ (a minimum of 24 children) –happily participated.

To convince these gatekeepers in the first stage, making use of an authority or survey sponsor effect (e.g. Dillman, 2000; Groves et al., 1992; Stoop, 2012) seemed to help.The authority or sponsor effect should also be used on the managers of day-care organizations. As many one-woman, private day-care facilities are run by mothers themselves, convincing them by focussing on the social responsibility to help out and the potential scarcity of surveys (e.g. Groves et al., 1992) for parents when selecting people through their children might also be helpful.

3.2. Determining the sample size among anonymous sample units

A second downside of two-stage cluster sampling is that the sample size is not known in advance (see e.g. Salehi & Seber, 1997). Moreover, two-stage cluster sampling requires a larger sampling size than random sampling to come to the same effective sample size, which may increase the sampling error. However, given that all mothers in all schools and day-care facilities in the area were selected for this study, the effective sample size was only affected by refusal at the school or day-care level. Determining the actual sample size of unique mothers or parents while having to deal with the privacy regulations of schools and organisations can be challenging. Identifyingunique mothers in an anonymous sample and preventing (anonymous) mothers from receiving multiple survey requests through different children required some effort from the schools as well as from us.

As the method implies working with information concerning minor children, guarantees regarding privacy and anonymity are essential for schools and day-care facilities. This required an identification system for a sample in which the identity of children or parents remains private combined with a need to have unique codes that could be reliably used over repeated communications. The challenge of developing such an identification system was relatively easily overcome through an agreement to use the schools’ standard pupil identification number in combination with a logically chosen, two-letter code referring to each school. A similar system was used for the youngest pre-school children and most of the day-care facilities, but with a combination of class or group numbers followed by a reference to the personal, unique icon (e.g. ‘teddy bear’) they use instead of numbers for the children. This code was also preceded by a two-letter designation referring to the name of the organization. Accordingly, only anonymous lists of identification numbers and numbers combined with references to icons were provided.

The benefit of using existing identification lists was that these were readily available, although the full lists with names remained in the custody of the schools. Therefore, the schools involved knew who the envelopes needed to be delivered to, but the survey organizers remained unaware of the identity of the children and their mothers. This allowed guaranteed anonymity, but maintained the ability to follow up specific sample units, as the schools (and day-care organizations) kept the full list and only need to match coded envelopes with the correct child.

To prevent mothers from receiving multiple copies, the help of the schools and day-care facilities was also needed. Most schools had information on sibling constellations readily available. This means they could provide pupil lists with only the identification number of the oldest child of a unique mother, preventing mothers from receiving more than one survey request. For two schools, this meant going into each classroom (with permission of the school) to identify whether there were older siblings in the school (or other participating schools). If so, the identification number of the younger child was deleted from the list by the teacher. In smaller suburban areas, such as the one where this survey was conducted, teachers tended to be well informed about the sibling constellations of their pupils. Of course, the children themselves were also helpful in signalling the teacher if they had older siblings in one of the relevant schools. In the day-care facilities, the personnel was instructed to inform each mother – as the study only focussed on women – that she only needed to participate through the letter sent via the oldest child if she had other children in the participating schools.

On top of this, it was stressed in every introductory letter – both for schools and day-care organizations – that only one survey needed to be completed per mother, preferably using the code for the oldest child, if multiple copies were received. On the included response card (also see section 3.3.), a comment box was offered. Twelve mothers informed us through the response cards that they had received multiple copies and would participate using the code received through an older child.

Based on the final lists with pupil identification numbers and the estimated total number of unique mothers using the day-care organizations, 1756 introductory letters were distributed in the schools and 123 in the day-care facilities. Subtracting the twelve mothers who informed us they had received multiple copies, this produces a sample size of a maximum of 1867 unique mothers – the sample size used to calculate response rates. This probably leads to a slightly overestimated sample size and a slightly underestimated response rate of 29.9% for the original survey (558 of 1867 mothers). Nevertheless, a more conservative estimate of the response rate is better than an overestimated one. Combined with a basic question survey among the non-respondents, there is data for 974 out of the 1867 mothers or 52.2% of the sample.

The Dillman total design survey method is an advisable strategy to use (see Dillman, 1978; 2000). This method is renowned for its success rates, due to a well-defined contact procedure starting with a properly-designed introductory letter, followed by reminders sent at regular time intervals and concluding with a thank-you card for the respondents or a registered postal mailing of a second set of the complete survey package for people who have not participated at this stage.

What is methodologically sound, however, does not always align with what is legally and financially possible. As there are general privacy restrictions regarding the use of information on minor children, and given additional specific school policies, some schools would only allow us to use their communication system a limited number of times. For the data collection itself, some schools opposed the total design survey method due to fear of parental complaints about receiving repeated communications from an outside party. The day-care institutions did not mention any issues with the original design, but the issues raised by the schools required that the overall survey design needed to be altered for the study.

More specifically, the data collection period needed to be shortened to four weeks, implying cutting out one reminder and refraining from sending a complete second survey package to non-respondents. Moreover, some schools wanted to work with response cards attached to the introductory letter, on which mothers could state whether they wanted to participate or not – i.e. a permission card. On this response card, a box for comments was also included, such as comments about receiving multiple letters or giving reasons for non-participation. Having a response card phase probably created an extra burden and threshold for participation. However, to offset this possible barrier to response, it was allowed to send a short basic question survey to the non-respondents, which enabled the investigation of the possibility of non-response bias. Analyses revealed that the non-respondents had significantly more time pressure and issues in combining work and family life (Vercruyssen, Roose, & Van de Putte, 2011). Indications of non-response bias due to time pressure and issues in combining work and family life were later also found in other surveys on work and family life (Vercruyssen, Van de Putte, & Stoop, 2011; Vercruyssen et al., 2013), showing that such a bias is not an artefact specific to this survey in the selected suburban area.

With this more restricted data collection strategy, a response rate of 29.9% for the original survey was managed. Supplemented with the data from the basic question survey among the non-respondents, there is data on 52.2% of the targeted mothers.

To overcome the challenges of using the total design method, clear communication and agreements with school boards are necessary. When given the chance to explain the total design survey method and its benefits for data collection and the subsequent analysis, many school boards do see the necessity of allowing repeated contacts through their communication system for the quality of the data collection and resulting research reports. Offering a research report to the schools and day-care facilities also helped to gain their full co-operation. Having a high status researcher in the negotiations, such as a university professor, might also improve the goodwill of school boards. As the authority of a survey sponsor can affect the decision to participate – with more compliance for highly legitimate sponsors in society, such as universities (e.g. Groves et al., 1992; Stoop, 2012) – the decision of school boards to allow a specific design for a survey can also be expected to be influenced by such authority effects.

Schools may be reluctant to give too much additional work to teachers in terms of distributing letters and checking the signatures in the agendas of their students. Offering to distribute the surveys in each classroom worked in overcoming the objections of one school concerning the number of times their communication system could be used for the survey. With regard to the collection of the completed surveys, the pen-and-paper surveys were gathered through the communication systems of the schools and day-care organizations. Despite this again being a very economical way of retrieving surveys, using these communication systems can create an extra burden for the teachers and personnel, whose co-operation is needed for getting access to the target population. In the case of resistance from teachers or school boards, I would suggest that the collection of completed surveys is carried out by the researchers themselves. Alternatively, working with pre-paid and addressed return envelopes for posting pen-and-paper surveys or working with online surveys can also be options.

4. Conclusion and further recommendations

In times when privacy laws and lack of resources hinder survey data collection more and more, straightforward and economical alternatives to random sampling from official registers can provide a welcome alternative. Such alternatives are perhaps even more welcome in countries with no or limited official registers. With regard to surveying parents, this test case shows a positive experience of using schools and day-care facilities in Flanders as a sample frame. The widespread use of these institutions for children makes sampling parents in this way a very viable method. This sampling approach might also be an interesting alternative outside of Belgium, which of course depends on local school policies regarding the age at which mandatory education starts and how widespread the use of day-care facilities for babies and toddlers is in other countries.

As mentioned above, there are some challenges to this approach. However, many of the encountered challenges can be avoided or mediated. Firstly, I believe that better response rates are possible by making use of an authority or survey sponsor effect (e.g. Groves et al., 1992) on school boards and the heads of day-care organizations. This authority effect should be used to convince schools and day-care facilities to participate, as well as to ‘sell’ the total design survey method (TDSM) (Dillman, 1978; 2000).

With regard to TDSM, explaining the need for an elaborate survey design, including sending reminders at regular time intervals (in laymen’s terms), can help to gain the agreement of school boards and day-care managers. Offering them a research report based on the survey results further increased the willingness to co-operate. Given that the suggested approach of using the communication systems of these institutions requires the help of their personnel, the optimization of this co-operation is a necessity. With regard to surveys on work and family life, school boards, day-care managers and their personnel often enough have their own children and are confronted on a daily basis with how the parents of the children they guard juggle the conflicting demands of work and family. Therefore, stressing the scarcity of surveys on the specific topic and the social responsibility to help can also improve the willingness of school boards and day-care managers in the first stage to allow the optimal TDSM.

Working with a sample of anonymous respondents has the drawback that establishing the exact sample size requires some effort. The sample size of anonymous sample units can be determined with the help of pre-existing pupil lists (that include sibling constellations), the help of the children and their teachers in the schools, and the help of day-care personnel. This information was not perfect, as twelve mothers reported receiving multiple introductory letters. Nevertheless, there is a maximum limit of sample size. This leaves us with a more conservative estimate of the response rate than could have been obtained from random samples from a population register.

With regard to the coverage of the parents, the parents of the 2% of toddlers who did not attend school at the age of two and a half to three tended to more often have a migration background or lower socio-economic status (Hedebouw & Petermans, 2009). They also tended to make less use of day-care services. This might have caused a slight bias for socio-economic status and ethnic composition. However, as people with a migration background or lower socio-economic status tend to participate less than other social groups in surveys, such a bias tends to also occur in surveys with regular random sampling (see e.g. Stoop, 2005; 2010).

Other non-frequent users of day-care facilities are mothers who either stay at home to care for their children or who get help from their family. In Flanders, grandparents often help out with day care to enable young mothers to work (Hedebouw & Petermans, 2009). However, these mothers can still be sampled through any older children in the schools, implying that this sample probably mostly miss some first-time mothers who do not use day-care services. If the aim of a study is to investigate how women (or men) make the transition to first-time parenthood, it might be advisable to try to obtain contact data from the Flemish governmental agency monitoring young children and their health (the Kind & Gezin agency). This agency is under the direct authority of the Flemish Ministry of Welfare, Health and Family. However, being a government agency working with (information on) minor children, official permission is required in order to access their data. Such a strategy would, again, consume a great deal of time.

Despite some challenges, I believe that the strategy of convenience sampling among parents with children in schools or day-care services has significant benefits compared with long, costly procedures to obtain a random sample from the population register when targeting parents. As listings of schools and day-care services are freely available in Belgium, there are minimal costs involved in obtaining them. The official communication systems of the schools and day-care services allow another economical advantage, as postage fees are not required. Additionally, with the required weekly signing of the agendas by the parents to acknowledge they have received important letters, this communication system comes close to the reliability of registered post. Accordingly, this type of convenience sampling offers a straightforward sample frame, with an economically beneficial contact procedure, for sampling parents in Belgium. Depending on the mandatory nature of school for minors and the use of day-care services in other countries, this form of sampling could also be adapted for surveys outside of Belgium.