2 Major steps in designing MICS sampleDefine objectivesKey indicatorsDesired level of precisionSubnational domains of estimationIdentify most appropriate sampling frameSample for another survey conducted recentlyMost recent census of population and housingDetermine sample size and allocationDetermine availability of previous MICS or DHS results to provide measures of sampling parametersMICS4 Survey Design Workshop

3 MICS4 Survey Design WorkshopSampling frameSampling frame should be nationally-representative and have complete coverage, with measures of size (households or population)Most countries conduct Census of Population and Housing every 10 yearsGenerally provides most effective sampling frame for household surveysSample for another survey conducted recentlyIn case of older frame, geographic areas with substantial changes, such as peri-urban in larges cities, may need to be updatedWhen no census is availableIdentify most complete geographic frame availableExample – Southern Sudan – list of villages from WHO immunization program with estimated populationMICS4 Survey Design Workshop

5 Indicators for sample size determinationSample size is different for each MICS indicator.Must choose a key indicator, since only one sample size can be used in MICSRecommendations for choosing key indicator:Choose from among main indicators of interest in your countryChoose the one which will yield largest sample sizeUsually for a single-year age groupUsually DPT, measles, polio or tuberculosis immunization - or birth weight below 2.5 kgExceptions: Do not choose infant or maternal mortality rates as the key indicators. Do not choose a low prevalence indicator that is desirably low (such as malnutrition prevalence).MICS4 Survey Design Workshop

6 MICS4 Survey Design WorkshopSample size formulawheren is the required sample size, expressed as number of households, for the KEY indicator,4 is factor to achieve 95 percent level of confidence,r is anticipated prevalence rate for key indicator,1.1 is factor to raise sample size by 10 percent for potential nonresponse,deff is shortened symbol for design effect,0.12r is margin of error to be tolerated, defined as 12 percent of r (12 percent thus represents the relative margin of error of r),p is proportion of total population that smallest group comprises, andis average household size.MICS4 Survey Design Workshop

9 Note on precision requirementsIn case of MICS2, precision requirements expressed in terms of acceptable margin of error (ME), which varied according to the size of the estimate (5% absolute error for high rate indicators or 3% for low rate indicators)For MICS3 and MICS4, this was simplified to a relative margin of error (RME) of 0.12Follow guidelines in sampling chapter carefully; avoid indicators with a high rateFinal criterion for acceptable precision: is the confidence interval useful?If confidence interval is too wide, estimate may not be usefulMICS4 Survey Design Workshop

10 Stratification and sample allocationStratification is the process of dividing the sampling frame into sub-groups (strata) of homogeneous (similar) PSUsAdvantages: better precision, flexible design, sub-national estimates for smaller domains (differential sampling rates)Reduced variance within stratum given similarity of unitsExample of stratification: region, urban/ruralExisting sampling frame, such as master sample, may have socioeconomic stratification for large citiesShould improve statistical efficiency of sample designGeographic domains defined as strataPossible to use variable sampling rates by domain to ensure sufficient sample size for eachMICS4 Survey Design Workshop

11 Implicit stratificationSort the sampling frame according to certain characters such as regions, urban-rural residence, sub-regions, districts, etc., then select a systematic pps sample.Ensures a representative sample for each subgroupAutomatically provides proportional allocation by size of subgroupMICS4 Survey Design Workshop

12 Allocation of sample to strataProportional allocationEffective for precision of estimates at the national levelEqual allocation to each domainUsed when each domain requires same level of precisionOptimum allocation – takes into account differential variance and costs by stratumFor example, variability may be higher in urban areas and enumeration costs may be higher in rural areas – use higher sampling rate for urban areasMICS4 Survey Design Workshop

13 Number of PSUs and cluster sizeSurvey costs depend not only on number of households but their distribution among primary sampling units (PSUs)Important to determine effective balance between number of sample PSUs and cluster sizeIn general, the more PSUs the better for reliability but the greater the cost (mostly costs of travel and listing)At national level, minimum of 300 to 400 PSUs should be selectedSubnational domains require larger samplesCluster size should be as small as practical for reliabilityExample: 8000 households selected in 400 PSUs of 20 sample households each is a much more reliable sample than 200 PSUs of 40 households each, but more expensiveMICS4 Survey Design Workshop

14 MICS4 Survey Design WorkshopDesign effectDeff - ratio of variance of estimate based on stratified multi-stage sample design and corresponding variance from simple random sample of same sizeMeasure of the relative efficiency of the sample designEffective stratification reduces the deffCluster sampling increases the deffDeft = square root of Deff, expressed as ratio of standard errorsGenerally presented in tables of standard errors for the DHSMICS4 Survey Design Workshop

16 MICS Sampling Option 1 – use an existing sampleDesign MICS as a rider to another survey if timely and feasibleUse sample from a previous survey and re-interview households for MICSOr, use old survey sample EAs and construct new listing of households to select for MICSOld sample must be probability-based, national in scopePossibilities – DHS, other national health survey, recent labour force surveyImportant: design parameters must be known (such as selection probability, stratification, etc.)MICS4 Survey Design Workshop

20 MICS4 Survey Design WorkshopDHS Method - Option 2Create “standard” segmentsDivide census population in each EA by 500 to determine number of standard segmentsMap sketch segments in each EAChoose 1 segment at randomList households in selected segment only (instead of entire EA)Purpose is to reduce listing workload to a manageable sizeMICS4 Survey Design Workshop

21 MICS Sampling Option 3 – use “compact clusters” with no listingModified segment, or cluster, design)Design new MICS sample based on prototypeTwo stages with census as frameUse of implicit stratification, systematic selection of census EAs at first stage with ppsPre-determine number of segments (measure of size) based on desired cluster sizeMap sketch segments in each EAChoose 1 segment at randomInterview all households in selected segmentMICS4 Survey Design Workshop

24 Common sampling option used by some countriesSelect EAs systematically with PPS, where measure of size is based on number of households (or population)In case of large EAs in sample, subdivide into standard segments, similar to Option 2Advantage: measures of size more exact, easier to implement a self-weighting design and control sample sizeMICS4 Survey Design Workshop

25 PPS systematic selection of PSUsSelection of PSUs with PPS provides a self-weighting sample when a fairly constant number of sample households selected in each PSU at second stageSystematic sampling of PSUs from a geographically ordered list ensures that the sample is geographically representative, with a proportional allocation to the different levels of geographyExamine template for PPS systematic samplingMICS4 Survey Design Workshop

26 Listing of households in sample segmentsImportance of new listing to represent current populationProblems with using previous listing (older than 1 year)Does not represent newer householdsDistribution of sample population by age group distorted, generally with higher median ageDifficulty of finding households in old listMICS4 Survey Design Workshop

27 Listing of households (continued)Common problems found in listing operationsProblem with quality of sketch maps – difficult to determine segment boundariesSometimes large differences found between number of households in frame (census) and number listedMICS4 Survey Design Workshop

28 Selection of sample households from listingSelection of households in the office following listing operationAdvantages – conducted by specialized staff, possible to avoid selection bias in the field, possible to control overall sample sizeDisadvantage – increased costs from having two field visitsSelection of households in fieldAdvantage – cost savings of having one integrated field operationDisadvantage - correct sampling may be difficult for field staff, selection may be biasedSelf-weighting samples – cluster sizes somewhat variableSelection of fixed number of sample households per clusterControls sample size, allowing weights to vary somewhat by EAUse of household selection table in fieldEasy to use, minimizes selection biasMICS4 Survey Design Workshop

29 Considerations for designing self-weighting samplesMain advantage of self-weighting sample is to simplify the estimation proceduresAlso effective for national-level estimatesDisadvantages of self-weighting samplesMay not be possible to obtain reliable estimates for smaller subnational groups, given proportional allocation of sampleDifficult to control overall sample sizeUse of SPSS and other software packages that automatically weight survey tables reduce advantages of self-weighting samplesMost countries are not using self-weighting samples for MICSPrefer selection of fixed number of households per EAMICS4 Survey Design Workshop

30 Subnational estimatesNumber of separate areas (domains) for which separate, equally reliable estimates are wanted affects sample sizeFor example, if 10 regional estimates are wanted, theoretically the sample should be increased by factor of 10As a compromise, larger sampling errors accepted for subnational estimatesOne proposal (by Dr. Vijay Verma) – increase national sample size by factor of D0.65, where D is the number of domainsResults in an average increase in the sampling errors for domain estimates by a factor of about 1.5Minimum number of PSUs required for each domain – for example, 30 clustersAllocation of sample to domainsEqual allocationModified proportional allocation, with a minimum and maximum number of sample PSUs per domainMICS4 Survey Design Workshop

31 Survey weighting proceduresAll analysis based on survey data must apply survey weights in order to prevent biased resultsSurvey weighting is design-specificOverall probability of selection has component from each sampling stage.Design weight is inverse of final probability of selectionNon-response must be taken into accountSeparate non-response adjustment for households, women age years and children under 5 yearsMICS4 Survey Design Workshop

32 Survey weighting proceduresFormulas for calculating weights depend on the exact sample design used in each countryDesign weights important for validating calculation of weights and coverage of frameWeighted total number of households by region, urban and rural strata should be compared to corresponding distribution from census data or projectionsNormalized weights – each weight is divided by the overall average weightUsing normalized weights, the weighted and unweighted total number of sample cases (households, women and children) are equalReview of templates for calculating weightsMICS4 Survey Design Workshop

34 MICS4 Survey Design WorkshopReducing biasAccuracy of survey results depends on both variance and bias (mostly from nonsampling errors)Bias should be minimized with quality control for all survey operationsBasic data quality determined during enumerationImportant to have good training and supervision in the fieldData capture should include 100% or sample verificationImportant to have quality control for editing and coding proceduresComputer consistency and range checksMICS4 Survey Design Workshop

35 MICS4 Survey Design WorkshopCountry example2008 Mozambique MICS3Use of existing surveySubsample of EAs from the other surveyShared listing with another surveyDifferent households selected for each surveyMICS4 Survey Design Workshop