The previous sections show that many indicators can be used to monitor the six eFA goals and other aspects of the education system. This section explains how to select the most appropriate indicators for a given purpose, and how to ensure that the indicators are used to reliably explain the situation, issues and implications, as well as to guide decisions and actions.
7.1 indicator selection criteria and practices
We can use indicators to monitor the overall situation, to examine specific aspects and issues, and to consider the implications in making decisions. Often, more than one indicator are required to explain a certain phenomenon or issue. For example, we monitor participation in school through the use of a number of ratios including gross, net and age-specific enrolment ratios. Sometimes one indicator can also be used to depict several phenomena or issues. For example, we use the pupil-teacher ratio to check if there are enough teachers for the number of students, but this indicator can also be used as a proxy for quality of education.
The art of using indicators is not about calculating each time the full range of education indicators to monitor EFA, but to judiciously select the ones which can most clearly and effectively indicate the situation and highlight the issues. To be efficient, avoid selecting, calculating, analysing and interpreting too many indicators in order to obtain the same finding and conclusion as can a few well- selected indicators for the same phenomenon or issue.
Knowing how to select the most appropriate indicator for a specific purpose is important, because if the wrong indicators are chosen, they can lead us to confusing, misleading or even contradictory findings and conclusions. When we choose indicators, we need to ensure they are based on the right concept, definition, analytical objective, data type, data source and angle of interpretation. The following criteria can help us select more appropriate indicators.
- Relevant in concept to the aspect or issue to be examined
- Clear in defining the purpose and limitations
- Data are available and reliable
- Can be easily and rigorously calculated
- Can be easily presented and interpreted
- Can be easily understood and used to aid in planning and decision-making
In summary, good indicators are clear, reliable and SMART (Specific, Measurable, Achievable or Attainable, Result-oriented and Time-bound).
In practice, the steps to be taken to select education indicators include the following:
- Precisely identify what we want to monitor or examine;
- List down the questions that need answers;
- Identify the kind of indicators that can be used;
- Review the methodological basis and robustness of these indicators;
- Find out what data will be needed to derive each indicator, and where and how to get this data;
- Gather sample data and examine their quality and reliability;
- Perform a test calculation to produce a draft indicator to see if there are any difficulties or biases in the results;
- Analyse the indicator’s efficacy for explaining the phenomenon; and
- Select the indicator or indicators that are most feasible and appropriate for the purpose.
In this way, we can more efficiently select different indicators to monitor progress in achieving the six EFA goals and other aspects of education. Like a jigsaw puzzle, these indicators complement each other to constitute a more comprehensive picture. using the criteria and practices outlined above, we may identify and produce other additional indicators to fill the remaining gaps in the picture.
While we are using each selected indicator for monitoring EFA, we can continue to check its relevance, feasibility, reliability and usefulness. For the purpose of ‘reaching the unreached’, we may identify additional gaps, problems and issues in some disadvantaged areas or population. More detailed disaggregation and in-depth analysis of the selected indicators can help to better pinpoint the gaps and address the problems and issues. Often, new issues and questions can emerge during such detailed analysis, which may call for additional indicators to be calculated and analysed.
7.2 incorrect selection and misuse of education indicators
Errors in selecting and producing education indicators can lead to serious misunderstanding or even wrong decisions which could have serious consequences for the school or the education sector. Such errors can occur at any time during the indicator selection and production process, by:
- choosing the wrong issue or aspect
- asking the wrong questions
- using the wrong indicator concept, definition and methodology
- using the wrong data or wrong data source
- incorrect calculation and analysis
- poor presentation and interpretation of the results
- incorrect use of the indicator
A mistake at any one of these steps can lead to misunderstanding and poor decisions. For example, we may have identified correctly the issue and asked the right questions, but if either the methodology of the indicator was poorly defined, or the data we used were unreliable, or errors were made during calculation, analysis or interpretation, the resulting indicator will be incorrect and likely to cause misunderstandings and lead to poor if not wrong decisions.
Sometimes indicators may be misused. Misuse can occur by selecting indicators that are irrelevant to the issue under review, such as when the pupil-teacher ratio is used to assess the conditions of facili- ties in the school. Or in some cases the value of an indicator does not actually explain or highlight any issue, but is taken as being significant. Another form of misuse is to over-generalize by taking two indicators which have similar values for a district, and assuming they have the same values for all other districts as well.
Another example of misuse of education indicators occurs when comparing indicators created with different scales. When assessing students’ performance at a school, if the average score of a mathematics examination conducted for class A was 58 out of 100, and the results of another mathematics examination conducted for class B was also 58 per cent. can we conclude that both class A and B achieved the same outcomes by demonstrating that the students’ performance are the same in the two classes? if we do not know whether these two examinations asked the same questions, or whether the same teacher marked the papers, it will be impossible to give a clear-cut answer ‘YeS’ to the above question. For indicators to be comparable we must use the same scale, the same definition and the same way of measuring the indicators. in addition, all the other conditions and steps for deriving the indicator values must be same.
Review your own criteria and practices in selecting and using education indicators, and discuss with other colleagues and stakeholders in your school, district, province or country about what they know and do. Then, answer the following questions:
- What was agreed should be the criteria to be used in selecting education indicators? Which other specific criteria have also been used? Why?
- How should one go about selecting education indicators? What other steps can be added? Why?
- What other lessons have you and your colleagues learned when selecting and using education indicators?