To be successful and useful, a school census must collect relevant, comprehensive and reliable data about every school. To do so, rigorous data collection methods using appropriate instruments must be adopted, accompanied by well-defined procedures and strict application of data quality norms and control practices.
Controlling data quality is essential throughout the entire data collection process. in this section, we look at the concept and practices of data quality control, and the roles and responsibilities of each level of the education administration.
When collecting data, it is important that the data collected are of high quality so that they can be reliably used as the basis to make sound decisions. To ensure data quality, data control measures must be applied at every stage of the data collection process, including when updating school records (see Section 6 in Module A1), creating data summaries from the records and completing the school census questionnaire at the school level. Additional data quality controls should be applied by district, provincial and central education officers, and when entering the data into computerized eMiS database as well as during data analysis, interpretation and use.
6.2 Sources of problems in data quality
Various problems can affect data quality and subsequent data analysis and use in decision-making. Five main sources of data quality problems can be identified as follows:
- Poor school records:non-existent,incomplete or inaccurate school records(seeModuleA1) can make it difficult for school managers to complete the school census questionnaire with reliable information. if a lot of schools have incomplete or inaccurate school records, it can impact the overall quality of the school census.For example: if a school does not keep records of students without textbook by subject, it cannot report the number of such students nor of the missing textbooks during the school census. The resulting national data will be incomplete and under-represent the real situation, thereby leading to wrong understanding and decisions regarding textbook production and distribution.
- Bad design of the school census questionnaire:inappropriate structure and presentation of the questions and tables, and missing or unclear explanations and instructions (see Sections 4.2 and 4.3), can cause errors while completing the school census questionnaire, if not also many omissions of important data.For example: A complex statistical table that presents multiple layers of headings and sub- headings that try to collect data on the number of students, repeaters, in-transfers, drop-outs by gender and age all in one table will be very difficult if not impossible to complete, which may cause the respondent to provide inaccurate or incomplete data.
- Lack of understanding of data terms,concepts and categories:Respondents fill in the questionnaire without reading and fully understanding the data concepts, terms and categories.For example: Respondents don’t understand they need to subtract the number of students who have repeated Grade 1 from those students who are enrolled in Grade 1 when asked to provide data about new entrants into Grade 1.
- Incorrect completion of questionnaire:Respondents enter either incorrect data or correct data but into the wrong cell, or have not completed all the essential cells of the questionnaire.For example:
- ThenumberoffemalenewentrantstoGrade1was343,but34wasfilledinthequestionnaire. This will affect information about the gender balance in this school.
- School managers did not fill all essential cells in the questionnaire using the correct symbols given in the beginning of the questionnaire (see example on p. 40).
- Inadequate and careless checking of the completed questionnaire: School managers, district education officers, inspectors and other stakeholders did not thoroughly check the completed questionnaires for errors and omissions.For example: The school manager and the district education officer checked to see whether all the questions and tables were completed, but failed to check for obvious data errors.
Data quality problems can be caused by heavy workload and unclear definitions of roles, responsi- bilities and tasks among those responsible for completing and verifying the school census question- naire. Poorly designed questionnaire, inadequate understanding of the instructions, intentional or unintentional misreporting, and many other factors can also affect data quality during school cen- suses. Having a thorough understanding of the factors that can affect data quality is essential.
6.3 data quality control during data collection
Data quality control can be done:
- Before and during school census data collection.
- During data entry and processing.
- When analysing, interpreting and using the data.
The following part introduces some general methods and practical tips for carrying out data quality control during the processes of data collection.
6.3.1 School record management
Quality data begins at the source. Standards and procedures about records management should be clearly defined and implemented in all schools. Schools must be required to maintain school records in a systematic and rigorous manner so information is available for regular school management and for use in responding to the annual school census questionnaire.
6.3.2 Design, pre-test and revision of the school census questionnaire
data quality control mechanisms should be built into the design of the census questionnaire and refined through the pre-test and subsequent revisions.
- Carefully investigate the socio-economic characteristics and aspects of culture that can affect school reporting and quality control practices, and consider how the questionnaire should be designed to minimize such problems.
- Design the school census questionnaire with clear structure, presentation and explanations, and concise instructions.
- Thoroughly revise the questionnaire based on the feedback gathered from the pre-test, before the finalized questionnaire is distributed to schools across the country.
6.3.3 Completing the questionnaire
- Respondents must read and fully understand the instructions for completing the school census questionnaire, and should ask district education officers for help when needed.
- School managers must carefully check and re-check the data for omissions and errors during and after completion of the questionnaire.
- District education officers and school inspectors should train relevant school personnel and maintain frequent communication with them to monitor progress while they complete the school census questionnaire, and to offer help with data or reporting problems.
6.3.4 Data verification and quality control
- Data quality checks and controls should be done as close to the data source as possible to make it easier for school managers and district education officers to trace data errors and omissions back to records and information at school.
- School managers should check for data omissions, errors in calculations, inconsistencies within and across tables, and for the incorrect use of special symbols (see next section for details).
- District education officers and inspectors should systematically check for late or missing responses, misunderstanding among school managers, and data omissions and errors (see next section for details).
- Training in data quality control procedures should be provided to education officers at the district and provincial levels to ensure they can check and control the quality of data from the schools.
6.3.5 Data quality control during data entry and processing
Additional data quality controls can be applied during data entry and processing.
- Automatic data verification mechanisms can be designed and incorporated into data entry systems using computers or internet-based online systems, to indicate data omissions, errors and inconsis- tencies. For example, the system may automatically add up all the values in rows and columns and cross-check with the reported totals.
- Once data entry is complete, calculating derived statistics such as percentages, ratios and rates can highlight missing data and errors. For example, it may signal errors when girls account for more than 100% of total enrolment, or when there are student-teacher ratios of more than 1000.
- When school managers enter data online, built-in automatic data verification systems can immedi- ately signal any data omissions and errors on-screen, so that corrections can be made immediately.
6.3.6 Data quality control during data analysis and interpretation
The processes of deriving indicators and analysing and interpreting the information from the school census data can help to highlight and identify other data anomalies.
- Calculation of education indicators and comparing them among provinces, districts and schools can reveal unlikely or illogical results that can be traced back to data or calculation errors.
- Further data errors and inconsistencies can be detected while interpreting the analytical results to draw conclusions.
- Having other persons to review the analytical results may help to identify data anomalies that were not obvious during the initial analysis.
- When disseminating information products and indicators, encourage the users to query the information or make comments. Standard procedures for referring queries back to the appropriate district education officer or school manager should be defined and implemented.
6.4 The role of stakeholders in data quality control for school census
Without serious data quality control, the data collection process can be like cooking food without tasting it. Stakeholders at different levels of the education administration can play specific roles in data quality control. These are described in more detail below.
6.4.1 Schools and school administrators
School, as the main source of school census data, can be considered the most important level for data quality control. At this level, attention may focus on two aspects:
School records must be created and updated in a systematic, complete and accurate manner. School records can be subject to frequent review by the responsible school personnel to ensure the data in the records are always accurate and up-to-date.
When preparing to respond to the school census, school managers and staff including teachers should retrieve the relevant school records, extract the correct information, and carefully tally the data in order to answer the questions and properly fill in the tables. during such preparations, school managers should ensure the data in the school records match the definitions of data in the school census (see Section 3 in Module A1 for details). When needed, the school manager should refer any errors or omissions to the school staff who are responsible for gathering and recording the data in the school records, so they can explain the omissions and rectify the errors.
Completing the school census questionnaire
Errors may occur when data are being entered into the school census questionnaire. This may occur for a number of reasons.
- There is a mismatch between the data requested and the data in the school records – This can occur if the school manager misunderstood the meaning of specific data categories in the school census questionnaire which are different in the school records, and used the wrong school data. School managers should always read the instructions and data definitions carefully before entering data, and consult district or local education officers if they need clarification or help.
- Mis-codingofdatainthequestionnaire-Schoolmanagersshouldbecarefulinensuringthey use the correct codes defined by the Ministry of education when entering data.
- Not checking responses-Schoolmanagersshouldcarefullycheckandre-checkthecompleted questionnaire to identify any remaining data errors or omissions before submitting it.
CHECK LIST FOR SCHOOL MANAGERS TO ENSURE DATA QUALITY:
- Check the school code
- Check the name of the school, full address, telephone number, etc.
- Ensure all pages and questions have been properly completed
- Calculate and re-calculate the vertical, horizontal and grand totals in tables to ensure the data are consistent
- Cross-check the totals between two related tables to verify consistency
- Check the balance between school income and expenditure
- Check any fields marked ‘data not available’ to confirm that the data are really unavailable and that reasons for the data being unavailable have been specified
- Check that any special symbols used in the questionnaire are consistent with the special symbols the Ministry of education has specified
For school managers, please compare the data quality control practices described above with the practices in your school, and answer the following questions:
- What role do you play in your school regarding data quality control?
- How do you ensure your school consistently reports high quality data during school censuses?
- What difficulties have you encountered with data quality control at your school? What caused these difficulties?
- Referring to the checklist above, what else should you check before you submit your completed questionnaire to a higher level?
6.4.2 District Education Officer (DEO)
The district education Officer is responsible for ensuring that all the schools in the district receive the school census questionnaire (or can access it online) and respond by the deadline. When requested, district education officers should be ready to provide assistance to schools to help them accurately complete the questionnaire.
After the schools, district education officers are the second most important level for data quality control. district education officers are normally familiar with the schools in their district so they are well-placed to make initial checks of the completed questionnaires submitted by the schools. Any data omissions, inconsistencies or errors identified at this level can be corrected easily by calling or visiting the school. Monitoring school records and helping schools to improve their school records management practices are also the responsibility of the district education officer. These tasks directly contribute to maintaining the quality of census data at the source.
While checking the completed school census questionnaire, the district education officers may apply the methods below:
- Coverage check
- Check the school list to ensure that all schools have returned the completed questionnaire
- Contact and remind the schools that have not responded
- Offer assistance to schools that have problems with completing the school census questionnaire
- Data check
- Are all the pages, questions and tables properly completed? Are the answers legible? Contact the school to complete any missing data.
- Have the special symbols been properly used to explain data limitations?
- Do the vertical and horizontal totals correspond to the sum of the detailed data?
- Are the same data consistent in different tables of the questionnaire?
- Are there illogical data (e.g. the number of female students is larger than the total of male+ female student population)?
- Use other innovative data checks.
- Feedback on questionnaire design
- Provide useful feedback to the Ministry of Education about difficulties encountered during school census so as to further improve the design of the questionnaire, instructions and the organization of the school census.