The Online Panels Benchmarking Study comprised three surveys based on probability samples of the Australian population and five surveys administered to members of non-probability online panels; altogether it includes the data from all eight surveys. The objectives of this study were to:
1. undertake the first Australian study comparing the accuracy of the results obtained from various surveys administered using probability based sampling methods and non-probability based sampling methods;
2. assess the usefulness of selected variables to calibrate non-probability online surveys with population benchmarks in an effort to reduce bias; and
3. compare the findings of this study to similar international studies.
The questionnaire administered to these samples, the Health, Wellbeing and Technology Survey, was designed by researchers at the Social Research Centre and included a wide range of demographic measures and questions about health, wellbeing and the use of technology. Data collection for all eight iterations of the Health, Wellbeing and Technology Survey was undertaken between October and December 2015 with varying fieldwork periods designed to accommodate the particular requirements of each survey. All the questions used to measure primary and secondary demographic characteristics and the substantive items were adapted from high quality Federal government surveys. These items were chosen because there were high quality population benchmarks available for these measures. This was a critical part of the overall research design as it enabled the accuracy of the estimates derived from the various probability and non-probability surveys to be compared against each other and against official population benchmarks.

Australian residential population aged 18 years and over (probability samples) / persons aged 18 years and over who were members of online panels (non-probability samples)

Date of Collection

Start date - Data Collection: 2015-10-19

End date - Data Collection: 2015-10-19

Time Period

Time Period - Start: 2015-10-19

Time Period - End: 2015-10-19

Geographic Coverage

National

Geographic Unit

State

Methodology

Content

Sampling Procedure

One-stage stratified or systematic random sample;Non-probability

Data Kind

Survey data

Response Rate

For all surveys, sample yield (the number of completed surveys as a proportion of all invitations) was calculated. The sample yields for the non-probability panels ranged from 2.6% to 15.4%. For the probability surveys the range was 9.8% to 26.2%.
AAPOR Response Rate 3 was calculated for the probability surveys only, in order to facilitate comparison with similar international studies. The calculation of Response Rate 3 relies on estimating the proportion of cases of unknown eligibility that may have been eligible for the survey and including this estimate in the denominator for the calculation of the survey response rate. Response rate for the ANU Poll (recruited) survey was 12.4% (see Note on Table 11 of the Technical Report for explanation). Response rates for the RDD survey and the ABS survey were 17.9% and 26.5%, respectively.
For more information, please see Technical Report Section 6.2.

Data Access

Content

Access Conditions

The depositor may be informed (by the archive) of use being made of the data, in order to comment on that use and make contact with colleagues of similar interests.

Deposit Information

Content

Depositor

Honey, Nikki

Deposit Date

2016-07-04

Version

Version 1.0

Other

Content

Unit of Analysis

Individual

Mode of Data Collection

Telephone interview, Web-based self-completion, Postal survey

Comment

The Online Panels Benchmarking Study comprised three surveys based on probability samples of the Australian population and five surveys administered to members of non-probability online panels; altogether it includes the data from all eight surveys. The objectives of this study were to:
1. undertake the first Australian study comparing the accuracy of the results obtained from various surveys administered using probability based sampling methods and non-probability based sampling methods;
2. assess the usefulness of selected variables to calibrate non-probability online surveys with population benchmarks in an effort to reduce bias; and
3. compare the findings of this study to similar international studies.
The questionnaire administered to these samples, the Health, Wellbeing and Technology Survey, was designed by researchers at the Social Research Centre and included a wide range of demographic measures and questions about health, wellbeing and the use of technology. Data collection for all eight iterations of the Health, Wellbeing and Technology Survey was undertaken between October and December 2015 with varying fieldwork periods designed to accommodate the particular requirements of each survey. All the questions used to measure primary and secondary demographic characteristics and the substantive items were adapted from high quality Federal government surveys. These items were chosen because there were high quality population benchmarks available for these measures. This was a critical part of the overall research design as it enabled the accuracy of the estimates derived from the various probability and non-probability surveys to be compared against each other and against official population benchmarks.

For weighting the probability surveys, a two-stage process was used. Firstly a design-weight (or pre-weight) was calculated in order to adjust for the unequal probability of sample members being selected into the sampling frame or invited to participate in the survey. Each respondent’s weight is the inverse of their probability of selection. After the design weight was calculated, it was then adjusted to reduce possible non-response error to create a final weight. Since crossclassification of benchmarks was not necessary, iterative proportional fitting (sometimes called rim weighting or raking) was applied for post stratification weighting. For weighting the non-probability surveys, only the second stage (post stratification) was applied.

ADA ID

au.edu.anu.ada.ddi.01329

Label

Online Panels Benchmarking Study, 2015

Language

en

Countries

Australia

Type of Research Instrument

Structured

ADA Processing Level

Level 3

Producer

The Social Research Centre

Producer abbreviation

SRC

ADA Study Level

Level 3

Version Date

2016-07-13

Version Responsibility

Australian Data Archive

Time Method

One-time cross-sectional study

Weighting

For weighting the probability surveys, a two-stage process was used. Firstly a design-weight (or pre-weight) was calculated in order to adjust for the unequal probability of sample members being selected into the sampling frame or invited to participate in the survey. Each respondent’s weight is the inverse of their probability of selection. After the design weight was calculated, it was then adjusted to reduce possible non-response error to create a final weight. Since crossclassification of benchmarks was not necessary, iterative proportional fitting (sometimes called rim weighting or raking) was applied for post stratification weighting. For weighting the non-probability surveys, only the second stage (post stratification) was applied.

Label: A2. Type of Internet connection - In some other way (including access through mobile phones or other mobile devices)

Variable Name: a2_3

Nesstar ID: au.edu.anu.ada.ddi.01329_V20

Invalid Cases: 0

Valid Cases: 4757

notes: (READ OUT)

Variable Position: 20

Question Text: When you are at home, in which, if any, of the following ways are you able to access the internet..?
- In some other way including Internet access through mobile phones or other mobile devices

Question Text: This section helps us to understand more about the lifestyle, health and wellbeing of Australians. Firstly, on a scale from 0 to 10, where zero means “not at all satisfied” and 10 means “completely satisfied”, overall, how satisfied are you with life as a whole these days?

Question Text: The following questions are about how you have been feeling over the past 4 weeks. In the past 4 weeks, about how often did you feel…? Would you say…?
- So depressed that nothing could cheer you up

Label: B7. Number of standard drinks usually consumed in typical day where alcohol consumed

Variable Name: b7

Nesstar ID: au.edu.anu.ada.ddi.01329_V48

Invalid Cases: 962

Valid Cases: 3795

notes: (READ OUT)

Variable Position: 48

Question Text: On a day that you (have/had) an alcoholic drink, how many standard drinks (do/did) you usually have? A standard drink is equal to 1 pot of full strength beer, 1 small glass of wine or 1 pub-size nip of spirits. Would it be…?

Question Text: We’re interested in whether your home is owned by you or members of your household. Is the dwelling that you live in owned outright, owned with a mortgage, being rented or is it occupied under some other sort of arrangement?

Question Text: And over the last 12 months, did you spend any time doing voluntary work through an organisation or group? Please note this does NOT include anything you do as part of part of paid employment, to qualify for a government benefit or any work done in a family business.

Question Text: To thank you for your time and responses we would like to send you compensation to the value of $10. To do so, we will need to collect your name (or an alias) and your contact details. This information will be used only for this purpose and would not be provided to any third party.
If you choose to redeem your $10 gift voucher via email you will need to be able to:
- Print a voucher sent to your email address OR
- View your email on your mobile phone to redeem your voucher Please select ONE of the following options: