2. Brief Information about Common Household Surveys
2.1 Background and Objectives of Selected Household Surveys
2.1.1 Multiple Indicator Cluster Survey (MICS)
The Multiple Indicator Cluster Survey is a household survey which was developed by UNICEF mainly to enable countries to fill in data gaps for monitoring the situation of children and women. It is capable of producing statistically sound, internationally comparable estimates of a wide range of indicators. MICS was originally developed in response to the World Summit for Children to measure progress towards an internationally agreed set of mid-decade goals. The first round of MICS was conducted in1995 in more than 60 countries. The second round, with approximately 65 surveys, was conducted in 2000.
The third round of MICS was carried out in more than 50 countries from 2005. It focused on providing a monitoring tool for the World Fit for Children, the Millennium Development Goals (MDGs), as well as for other major international commitments, such as the United Nations General Assembly Special Session (UNGASS) on HIV/AIDS and the Abuja targets for malaria. At least 21 MDG indicators have been collected in the current round of MICS, offering the largest single source of data for MDG monitoring.
Results from the surveys, including national reports, standard sets of tabulations and micro level datasets are available at UNICEF’s web site: www.childinfo.org.
2.1.2 Demographic and Health Survey (MEASURE DHS)
Since 1984, the Demographic and Health Survey (DHS) Project has provided technical assistance to more than 75 countries in conducting more than 200 demographic and health surveys to help advance global understanding of health and population trends in developing countries. In 1997, DHS became one of four components of the “Monitoring and Evaluation to Assess and Use Results” (MEASURE) Program.
The MEASURE DHS Project has gained a worldwide reputation for collecting and disseminating accurate, nationally representative data about health and population in developing countries. The project is implemented by Macro International Inc., which is funded by the United States Agency for International Development (USAID) with contributions from other donors such as UNICEF, UNFPA, WHO, UNAIDS.
Since October 2003, Macro International has partnered with four internationally experienced organisations – The Johns Hopkins Bloomberg School of Public Health/Center for Communication
Programs, Program for Appropriate Technology in Health (PATH), Blue Raster, and The Futures Institute – to expand access to, and use of, DHS data.
DHS surveys collect information about fertility, reproductive health, maternal health, child health, immunisation and survival, HIV/AIDS, maternal mortality, child mortality, malaria, and women and children’s nutrition. The strategic objective of MEASURE DHS is to improve upon and institutionalize the collection and use of data by host countries for program monitoring and evaluation and for policy development decisions.
2.1.3 LSMS – Living Standard Measurement Survey
LSMS was established by the Development Economics Research Group (DECRG) of the World Bank to explore ways of improving the quality of household data that are collected by statistical offices in developing countries. LSMS is a research project that was initiated in 1980 and carried out over several rounds in more than 30 countries. The program is designed to help policy makers identify how policies can be designed and improved to positively affect outcomes in health, education, economic activities, housing and utilities, etc..
Objectives of LSMS include:
- to improve the quality of household survey data.
- to increase the capacity of statistical institutes to perform household surveys.
- to improve the ability of statistical institutes to analyze household survey data for policy needs, and
- to provide policy makers with data that can be used to understand the determinants of observed social and economic outcomes.
LSMS provides users with actual household survey data for analysis and links to reports and research done using LSMS data.
2.1.4 LFS – Labour Force Survey
Labour Force Survey is one of the most common and most frequently collected household surveys. The recorded LFS was first conducted in 1940 in USA, 1960 in Australia, 1973 in United Kingdom, and so on. Currently LFS is conducted monthly in USA, and quarterly (four-times in a year) in Australia, New Zealand, United Kingdom and almost all countries in European Union.
Main objectives of LFS include:
- to capture labour market data, and
- to estimate unemployment rate in the country and regions.
In addition to generating official labour force statistics, data from the LFS are employed by academics and other researchers. In the UK, for example, the LFS has been used as a data source for research projects on topics such as female employment, economic returns to education, migration and ethnic minority groups.
On the other hand, datasets from labour force surveys are not included in this manual as examples. However, country EFA offices can get LFS data from the responsible government offices and/or agencies, for use in analysis.
2.1.5 Population Census
The oldest type of household survey with the broadest coverage is the “population census”. By international agreement, censuses collect data about the entire population in a specified area at a regularly marked time interval. Each person is asked questions about personal characteristics such as age, sex, marital status, education and employment status. Population censuses can therefore provides data about the number and composition of the entire population at a given point in time, and selected socio-economic and educational characteristics of households and individual persons in the country.
Since the census collects data from every household in the country, it can provide valuable information for policies and the planning of socio-economic development from national down to local administrative levels. Moreover, census is the main source and basis for constructing sampling frames for selecting households and population for other surveys.
Population censuses are carried out once every 10 years in most of the countries, or once every five years in some economically advanced countries. As such, the census is the most comprehensive source of demographic and socio-economic data for several countries.
Although the main objective of a census is to get reliable population data, the latest United Nations guidelines for preparing population censuses emphasize the need to collect data about literacy, school attendance, educational attainment, field of study and educational qualifications.
2.2 Structure and Contents of the Selected “Survey Questionnaires”
2.2.1 Multiple Indicators Cluster Survey (MICS)Questionnaire
MICS uses three main questionnaires in every survey:
(i) household questionnaire,
(ii) questionnaire for women aged 15-49, and
(iii) questionnaire for children under the age of 5.
The Household Questionnaire is comprised of questions about household characteristics, such as household listing, education, child labor, water and sanitation, salt iodisation, insecticide-treated mosquito nets (ITNs), and support to children orphaned or made vulnerable by HIV/AIDS, with optional modules for disability, child discipline, security of tenure and durability of housing, source and cost of supplies for ITNs, and maternal mortality.
B. Household Listing Form
C. Education Module
2.2.2 Model Household Questionnaire of MEASURE DHS
Although DHS surveys aim to collect data about fertility; reproductive, maternal and child health; immunization; survival and nutrition; maternal and child mortality; HIV/AIDS; and malaria, the key household questionnaire covers several additional questions about education and its differentials. The followings are relevant extracts from the DHS Model Household Questionnaire.
A. Household Identification
C. Listing of all Household Members – 2
2.2.3 Education and Training Related Sections in Living Standards Measurement Survey (LSMS) Questionnaire
LSMS is a comprehensive survey. Its questionnaire set contains (i) household and (ii) community and (iii) price questionnaires. The household questionnaire is over 100 pages long and covers 15 sections including education.
The education section of the LSMS questionnaires has three sub-sections on four pages as follows:
Ref: LSMS Working Paper 130 “Model Living Standards Measurement Study Survey Questionnaire for the Countries of the Former Soviet Union” by Raylynn Oliver.
2.2.4 Education and Related Questions in Labour Force Survey (LFS)
The LFS is a survey that collects rich information about labour force participation and work activities, unemployment and underemployment. The questionnaire also contains (i) literacy information about household population (generally, aged 5 and above), (ii) current participation (enrolment) in education, (iii) highest education level attained, and (iv) participation in technical/vocational training.
Data about education and related items from the Pakistan LFS(2008/09) are provided in the following table:
SECTION 4: HOUSEHOLD COMPOSITION AND DEMOGRAPHIC INFORMATION
And, the following presents education/training related questions of the Nepal LFS 2007/08:
Since no dataset is available, the labour force survey will be excluded from further discussion in the forthcoming sections.
2.2.5 Population and Housing Censuses
Population censuses gather data from every person in the country, and it is the most reliable source of population data. Household rosters, which are used in censuses, collect basic information about all household members, such as age, sex, marital status, education and literacy status, together with household characteristics, such as location and type of residence and the availability of services.
Example 1: The 2009 Population and Housing Census questionnaire in Viet Nam included the following questions to collect data about education and literacy status from the entire population. Combined with data about the age, sex, residence, migration and disability status–which were recorded in other questions – the literacy status, educational attainment and participation, and access to education, can be analyzed for various population groups.
Example 2: The 2005 Population and Housing Census from the Republic of Korea includes just two items about education in one question. Even with such limited data, the education and literacy status of population and schooling status of children can be studied by age, sex or residence, or any other relevant factor (variable).
Example 3: The General Population Census of the Kingdom of Cambodia (2008) contains questions about literacy, education and disability status in the main questionnaire.
2.3 Considerations on Sample Design
Populaton censuses collect data from all households in the study area (a region, or a territory or a country). Data are therefore collected from the entire household population. While collecting data during a population census, there might be some households that do not respond. But such non-responding households can be relatively few and generally negligible. Since the census collects data from the entire population, it does not require a sample design and the data and indicators derived from the census are the actual values of the population, not estimates from a sample of the population.
On the other hand, a household survey collects data from selected households in the area, and provides estimates (of the characteristics or indicators) for the entire household population in the area based on the sample households. The quality (accuracy of the estimates) and the usefulness of a household survey depend on the followings aspects.
i) Sampling method (about how the sample households are selected)
Common sampling methods include SRS (Simple Random Sampling), PPS (Probability Proportional to Size), cluster sampling, multi-stage sampling, and purposive sampling.
ii) Coverage (about whether the entire study area is covered by the survey)
To be representative of the entire population, all households must have an equal chance of being selected from all households in the area (country or region) using a random sampling method. Some household surveys select from the households with specific characteristics (e.g., poultry farmers) or from pre-assigned parts of the areas only (e.g. households beyond 3 mile radius from a school).
iii) Sample size (about how many households are selected) and allocation of samples (about how the sample households were allocated to different parts of the area)
iv) Data analysis – how to get estimates (values) of the key indicators, perceived standard errors of estimates, and by pre-determined level of disaggregation (e.g. by age, sex, grade, region/province/district/commune/village, socio-economic status, etc.).
These factors are all considered while designing the household survey sample and are usually included in the survey report. Secondary analysts and other persons who use the data need to know the sampling method and sample size before they can analyze the data.
- If the estimates are not calculated in line with the sampling method of the survey, the accuracy of the analysis will be lower.
- The sample and survey method also determines which weights (if any) should be applied to the data analysis.
- By understanding the coverage of the survey, the sample size and the level of disaggregation, data analysts understand the limitations of the survey including whether the desired disaggregation is appropriate at required degree of accuracy or not.
A survey was designed to get reliable estimates for adult literacy rates at the provincial level, disaggregated by sex. If the estimates of adult literacy rates were computed for adults who are living in remote areas with the lowest socio-economic status (lowest wealth index quintile) by district and by sex, the derived estimates will not be reliable. On the other hand, some surveys are designed to capture specific and rare events. In such a survey, sample size is large and thus sufficient to estimate common education indicators at lower disaggregated levels with an acceptable level of accuracy.
All the common household surveys described in this section use solid sample design, which is generally the same for all countries, and thus provides concrete survey methodology. Datasets obtained from those surveys and censuses are valid to use for various data analyses. For other surveys, the data analyst should check the sample design, including survey method and coverage, through the accompanying documents, such as survey report or service contract, and/or contact the person or organisations that conducted the survey before analyzing the data.
2.4 Understanding Survey Data Files and Availability of
Education Related Data
This section highlights the education-related variables in the main datasets of three common household surveys and sample outputs for selected variables.
Education Related Variables in MICS Sample Dataset
In MICS sample dataset, four SPSS data files are generated for: (i) household, (ii) individual household members, (iii) women aged 15-49, and (iv) children under five years of age. MICS datasets are shared with a wide range of users. The second data file, which is for all individual household members (or household listing – hl.sav), contains data about the education and literacy status of populations, including school-age children. The sample “hl.sav” data file contains 183 variables for 29,560 cases (persons). The following 21 variables are useful for analyzing education and literacy.
The following tables, which are useful for analyzing the schooling status of children aged 5-14, are derived from the sample data file “hl.sav”.