Module A2: Data Collection and Quality Control

3. A general introduction to data collection

3.1 What is data collection?

Data collection is a term used to describe a process of gathering (or obtaining) specific information about a phenomenon or an activity. Once collected, data can be stored in records or data- bases, analysed and used for purposes of monitoring or evaluation. The quality of data collected has a direct impact

upon the quality of analysis that can be performed using the data, which ultimately impacts upon the quality of decisions that can be made.

in the education sector, data collection through school censuses aims to obtain relevant, up-to-date and reliable data and information regarding the education system as a whole. Officers at all levels of the education administration including school personnel can use the collected data to understand the current state of education in the country, and to support sound policy- and decision-making. While organizing and conducting a school census, the following five principles of data collection should be observed:

Relevance:

the collected data should be relevant to the activity or phenomenon you intend to analyse and study.

Simplicity:

the collected data should be simple in concept and easy to measure.

Accuracy:

the collected data can accurately describe the activity, phenomenon, topic or issue you are studying.

Clarity:

the collected data should be clear, unambiguous, easy to interpret and easy to understand. Practicability: the collected data can be easily accessed and reliably used.

These principles may be summarized simply as the following ‘5-right principles’:

Get the right data: collect data which are relevant to the specific topic or issue.
For example, to better understand gender disparity in school, one must collect data on students separately for boys and girls.

Get the data right: collect data with precise definition and appropriate method of measurement.
For example, data on new entrants in Grade 1 must not include those who actually attended another school, dropped out, then enrolled in this school for the first time.

Get the data right away: get current and timely data.
For example, school censuses should be organized as close to the start of the school year as possible, once enrolment is complete and attendance has stabilized.

Get the data the right way: get data through a rigorous process which can guarantee data quality and ensure consistency.

Instructions about methods and data standards must be explained clearly. The people involved in data collection should be trained.

Get the right data management: collect reliable data which is guaranteed by good quality control conducted by relevant stakeholders.

It is important to involve all the stakeholders at different levels of the education system to check that the col- lected data are reliable and complete before they are processed, analysed and used.

There is often a tendency for educational administrators to want to collect more data and in greater detail, but then failing to make full use of the data they collected. This should be avoided because the more data you require the schools to supply, the less data you will actually get, because of the negative effects of an increased reporting burden on schools. Always respect the motto: ‘Do not collect data that will not be used.’ 

Activity 2

Reflect on data collection activities in which you participated, and answer the following questions:

  1. How did each of these data collection activities fulfill the key principles of relevance, simplicity, accuracy, clarity and practicability?
  2. To what extent were the 5-right principles above applied?
  3. What do you see as the difficulties in applying all these principles?

3.2 Sources of data – Where to get data related to education?

it is necessary to understand from where data and information can be collected for education planning and management. There can be five main sources of education data (see Figure 2).

3.2.1 Local educational institutions

Local educational institutions, which include formal institutions such as primary schools and second- ary schools, as well as non-formal institutions such as Community Learning Centres and community- based education programmes, are the key sources of data about education. Records kept in these institutions contain data about education in the local area and are particularly useful as the basis for reporting data in response to the annual school census.

3.2.2 Household information

Local households are a second key source of data about education. information regarding the social, economic and cultural background and the demographic characteristics of school-age children, youth and adults in the households are crucial for analysing factors that affect access and retention in education, and the impact of education.

3.2.3 Individual persons

individual persons refer to local people who are involved either directly or indirectly in education, such as students, teachers, principals, parents, local community leaders, etc. detailed information about individual students and teachers is considered to be the most micro-level information about the education system. As individuals, students and teachers can provide information about the state of access and participation in the education system, the quality of education, as well as other data about the educational environment and realities within the local area. Gathering data from these individuals can help to identify the reasons why some children are not enrolled, drop out of school, or attend class irregularly, and how education impacts on their life.

3.2.4 Local administration

The local government administration and other relevant local bodies may record and regularly update information about the local area and its population. Local administrative bodies may also keep information about public institutions, facilities and programmers, infrastructure, employment, social welfare and especially the disadvantaged population groups. Such information can help to identify ways to ‘reach the unreached’ which is key to the eFA goal of providing equitable access to quality education.

3.2.5 Other sources

There are other sources where useful data about education-related aspects of a local community may be obtained, such as:

  • Health centres and health workers: information about family health status, epidemics, hygiene, nutrition, and disabilities.
  • Police stations: information about juvenile delinquency, crime, security, etc.
  • Religious centres: information about disadvantaged local population groups, social events with educational implications, etc.
  • Local business entrepreneurs: information about the employment situation, skills in demand, and vocational training needs.
  • The media (printed/electronic): information about community events; social activities; etc.
  • NGOs and social workers: information about local development issues and disadvantaged people in the population
  • Community neighbourhood bodies: information about children who are not attending school.

This module focuses on the first data source: schools and school records. Readers must however remember the other data sources and know that these are also useful for monitoring progress toward achieving the EFA goals.

Activity 3

Please review the 5 sources of data above and, based on your own experience in your country, province or local area, answer the following questions:

  1. What kind of data about education can be collected from the local education institutions? For what purpose(s)? Please give examples.
  2. What kind of data about education can be collected from local households in your area? For what purpose(s)? Please give examples.
  3. What kind of data about education can be collected from local individuals? For what purpose(s)? Please give examples.
  4. What kind of data about education can be collected from the local administration? For what purpose(s)? Please give examples.
  5. Please identify some other sources of data about education in your country, province or local area. What kind of data about education can be collected from them?
  6. For what purpose(s)? Please give examples.

3.3 Process of data collection: How to collect data?

There are various approaches to collecting data, depending upon the type of data you want to collect, and the source of the data. The process of data collection must be systematic and based on well-defined procedures that are appropriate to the context within which the data are being collected. Questionnaire-based school census is the way that is most commonly used to collect information about education. This section presents a concise introduction to the process of collecting data using a school census questionnaire.

There are seven stages in the data collection process, as depicted in Figure 4.

Stage 1: identification of information needs

Before you design a questionnaire or collect data, you must first have a clear understanding of what kind of information is needed and will be utilized.9 You may recall that education stakeholders at various levels have different roles and responsibilities (see Figure 1 in Section 2), and they have different needs for information. Find out what information these different stakeholders need and how they intend to use the information. By comparing the information needs of various stakeholders, it is possible to identify common, core information that should be collected through the school census.

Stage 2: Translation of information needs into data categories

After identifying the core information that should be collected, the information should be prioritized and translated into specific data types and categories (for more information, see Section 4.2 below). These categories will provide the framework for designing the school census questionnaire. Make sure that these data categories are easy to understand, the data is easy to find, as this will help school staff to respond to the questionnaire. For example, more detailed data items under each main data category, such as the number of students by sex, grade or age, can be clearly defined and explained in practical terms.

Stage 3: design, testing of forms/questionnaires and revision

After selecting the data categories, the next step is to design the questionnaire that will be used to collect data.

A. Designing the school census questionnaire

i. Choosing an instrument

A data collection instrument is a tool for monitoring or measuring an activity, behaviour or phenom- ena. it can be used to measure status, progress, shortcomings, performance, achievement, attitudes, or other particular attributes of the objects to be analysed. For a school census, the objects we are measuring are the schools, students, teachers, and teaching/learning activities. Most school cen- suses use questionnaires as the data collection instrument. interviews, observations, group discus- sions can also be used to collect supplementary qualitative data.

ii. Design of the questionnaire

The questionnaire should include specific questions and tables that will enable the collection of data for each category of data. While designing a census questionnaire, please refer to the following tips.

TiPS

  • The questionnaire must be respondent-friendly. The questions are clear and unambiguous.
  • The layout of the form is simple and easy to follow. Most importantly, the design of the form should motivate the respondent to want to complete it from beginning to end.
  • The questions and explanations must be written in simple, clear language.
  • The questionnaire should not be too long, and the questions should be in a logical order.
  • Clear instructions about how to complete each part of the questionnaire must be provided, and the terms should be explained.
  • Make sure there is enough space to answer each question. Where appropriate, prompts for addi- tional information such as ‘Please specify’ or ‘Others’ may be added to enable the respondents to provide additional information which is not included in the choices provided.

iii. Concise instruction and explanations

Provide concise general instructions in the beginning about how to fill out the questionnaire. Additional instructions may be placed next to complicated questions or tables to explain the terms and how best to respond to them. Some parts of the questionnaire may change from year to year. it is important to indicate what has changed and explain the changes.

B. Pre-test and feedback

i. Pre-test

It is very important to try out the questionnaire with a representative sample of targeted respondents before distributing it to all respondents. A pre-test helps to see how the respon- dents understand and interpret the questions, and what kind of difficulties they face in trying to respond to each question. The questionnaire’s designers can then use the feedback from the pre- test to fine-tune the design and the questions that were misinterpreted or misunderstood by the respondents, or were too difficult to respond to.

In the initial pre-test, many of the questions may be presented as open-ended questions as this allows us to collect a wide range of possible responses. These responses to the open-ended ques- tions can then be used to define a core list of possible responses and to develop close-ended ques- tions with multiple choices.

ii. Feedback and finalization of the questionnaire

A thorough analysis of the feedback from the pre-test can help to finalize a fully operational question- naire. Based on the findings of the pre-test, ambiguities and difficulties in the questions and questionnaire design can be minimized by improving the phrasing of questions, response options, instructions and explanations. This will ultimately contribute to improving the ease of data collection and the quantity and quality of data collected.

Stage 4: Collecting the data

After incorporating the feedback and finalizing the questionnaire, it can be disseminated to all schools to begin full-scale data collection. Responsibilities now shift from the Ministry of education to the district education officers and local school managers. School managers, for example, should use the school records to prepare data and fill in the questionnaire, while district education officers would supervise and support the schools in such data collection processes, and ensure the com- pleted questionnaires are sent to higher levels on time.

Such cooperation in data collection can help to strengthen the relationship and mutual accountability between the district education officers and the managers of school they over- see. This cooperation also allows the district education officers to gather first-hand information on the concerns and needs of the schools in their area.

There are several points about the data collection process that require special attention.

a) Date of distribution and collection of the questionnaire

The timing of the school census can influence the quality of the data that are collected. it is therefore important to launch the census at a time when data are available and likely to be most representative across all schools in the country. For this reason, annual school censuses are often taken early in the school year at a time when enrolments have stabilized. The Ministry of education should consult with education officers in the field to determine when is the best date to launch the school census.

A deadline for the return of completed questionnaires must also be set. This deadline should allow enough time for the schools to prepare data and complete the questionnaire. it should also allow for extra delays in receiving the completed questionnaires from schools in remote areas. equally important is for the deadline to respect the schedules for processing and analyzing the data, and final release of the information.

b) School registry and list of respondents

in order to make sure the questionnaires have been sent to all the relevant respondents, the Ministry of education needs to establish a system to track the distribution of questionnaires and record receipt of the completed questionnaires. The school registry, which contains a comprehensive list of all schools and contact details of the school managers, should be updated for this purpose.

Prior to distribution of the school census questionnaire, every school should be contacted, either directly or through the district education office, to update and confirm their details. When the questionnaires are sent to a school, this can be recorded on the school registry. Likewise, when a school returns the completed questionnaire, the Ministry of education can record the receipt in the school registry.

c) Distribution of the questionnaire to the respondents

Once the launching date of the annual school census is fixed, the Ministry of education can distribute the questionnaires to all the schools that are listed in the school registry, either by mail or through administrative channels, such as through district education offices.

in many countries, the Ministry of education prints the questionnaires at the central level and then distributes them through educational offices in the field. The district education offices are often in charge of forwarding the questionnaire to all the relevant schools within the district (see Figure 5). in some countries, however, the Ministry of education decentralizes the printing and distribution of the school census questionnaires to the provincial department of education.

More and more, national ministries of education are disseminating the questionnaire using electronic media such as e-mail, Cd-ROM, or USB stick. in a few countries, the Ministry of education provides for direct, online data reporting through the Ministry of education’s website.

d) Completion of the questionnaire

School managers are responsible for completing the school census questionnaire accurately, clearly and in accordance with the questionnaire’s instructions. it is, therefore, important that the managers take time to read the general instructions and explanations before completing the questionnaire.

If some of the questions cannot be answered, or if some of the data reported are incomplete, appropriate footnotes and explanations must be given. At this stage, frequent communications between school personnel and education officers at district and local levels are crucial to ensure that any problems and issues are addressed, and all parts of the questionnaire are completed properly.

e) Collection

In some countries, the completed questionnaires are collected by the district education officers, and then sent to the provincial level for onward transmission to the central level. Often, a copy of the completed questionnaire is kept at the education office at each level. in other countries, the school sends the completed questionnaire directly to the Ministry of education, and provides a duplicate to the district and provincial education offices.

More recently, some countries have begun using internet-based technologies, such as e-mail or the World Wide Web to conduct school censuses. When using the internet, respondents can either return the electronic questionnaire file by e-mail, or be given an authorized log-on to the Ministry of education website in order to fill out the questionnaire online.

Stage 5: Follow-up reminders

During a school census, district education officers should maintain regular contact with all the school managers in their district to ensure that all schools submit the completed questionnaire on time. it is the district education officer’s responsibility to remind schools about the deadline for returning the completed questionnaire. The district education officer should actively track the reporting sta- tus of each school and assist them if they need help. By using computerized school databases, the Ministry of education can automatically track the reporting status of each school, and inform dis- trict education officers and school inspectors to follow-up in reminding and assisting those schools that have not returned the questionnaire on time.

Stage 6: data verification and rectification

data verification is the process of cross-checking the completed census questionnaires for complete- ness and accuracy. data verification is an important step in ensuring the data collected through the census is of high quality (see Section 6 for more details).

It is not uncommon for the schools to make errors or omissions while completing the questionnaire. Some of the errors can be detected directly by the school managers and corrected at the school level. Other errors and omissions can be identified by the district education officer and inspectors, or when comparing the data with those from other schools in the district. in such instances, the district education officer should immediately contact the school manager to clarify and rectify these errors.

Once the questionnaires have been returned to the Ministry of education, other errors and omissions can be identified while entering data into the databases and during initial data processing and data analysis. When a data error or omission is identified, the Ministry of Education should inform the relevant district education office to take action by contacting the school to obtain the correct or complete data. Because of their physical proximity to the local schools, educational officers at the district and local levels are responsible for ensuring quality data are reported by the schools to the Ministry of education.

During the data verification stage, three kinds of problems can arise. These are missing data, errors and inconsistencies. These are described briefly below, and further explained in Section 6:

Missing data – This occurs when there are questions that remain unanswered, or cells in tables that are not complete but which have no footnote to explain why the data are missing.

Errors – There can be obvious and not-so-obvious errors in the data. For example, when the total is not equal to the sum of the parts – like when the number of girl students is greater than the total number of students of both sexes, or when the total area of all classrooms is bigger than the reported total area of the school.

Inconsistencies – different values for the same activity or phenomena are reported in different parts of the questionnaire. For example, the number of female students in Grade 2 in the table about school enrolment is 300, whilst the same number in another table is 289.

Some of the ways to either prevent, or handle these data problems are presented in Section 6.

Stage 7: Processing and storage

After data verification and rectification, the answers to questions and the data in the tables can be coded and entered into a database for computerized processing. in countries or provinces that have many schools, it may be more efficient to enter, store and process the data using computers. The eMiS database can be designed to capture, process and store every data item that was requested in the school census questionnaire. The database should also incorporate features to automatically check the quality of data, calculate the indicators, and generate summary lists, tables and graphics.

in cases where there are missing or inaccurate data which cannot be corrected immediately, it will be necessary to create a backup of the database in preparation for future corrections. in countries or provinces that do not have computerized facilities to process and store the data, it will be necessary to employ professional staff who can properly manage these data in paper files.

Activity 4

Based on your own experience of working with annual school censuses, please review the seven stages of data collection outlined in this section, compare them with your own practices, and then answer the questions below:

For school managers and personnel:

  1. What are your roles and responsibilities with regard to the stages of data collection described above?
  2. What kind of difficulties have you encountered when completing the school census questionnaire at your school?
  3. How did you solve these problems when completing and returning the questionnaire?
  4. What caused missing data or errors? How did you fix or minimize such problems?

For district education officers:

  1. What are your roles and responsibilities with regard to the stages of data collection described above?
  2. What kind of difficulties have you encountered while trying to ensure that all the schools in your district complete and return the school census questionnaire on time?
  3. What do you think are the most important things to keep in mind during the seven stages of data collection? Why?

For central, provincial and regional education officers:

  1. What are your roles and responsibilities with regard to the stages of data collection described above?
  2. Based on your experience, what kinds of difficulties are often encountered when conducting a school census? How can you solve these problems? Please give examples.
  3. What do you think are the most important things to keep in mind during the seven stages Of data collection? Why?
  4. If you were to organize a school census, how would you go about it?

Comments are closed.