Developing Kazakhstan's School Quality Assessment Platform

Developing a methodology and online platform to assess secondary education are part of ADB and the Kazakhstan government’s joint efforts to find innovative solutions for capacity development. Photo credit: ADB.

Share on:           

Published:

A prototype online platform was created based on a new method for analyzing the quality of secondary education.

Introduction

Quality of education is a key point in Kazakhstan’s strategic planning as cited in its Concept for the Development of Education of Kazakhstan for 2022–2026 and National Project Quality Education—Educated Nation.

Meanwhile, information on secondary education collected within the Kazakhstan national statistical reporting system are approaching big data scale. However, existing analysis of its education system is mostly based on a limited range of expert-selected factors and descriptive statistics presented in the annually published National Report on the State and Development of the Education System of the Republic of Kazakhstan. To date, reporting on the state of the education system is affected by the following factors:

(i) categorical indicators (qualitative variables such as information on teachers workload or teachers who have completed ICT courses) are excluded from the analysis;

(ii) selection of indicators largely depends on the subjective opinion and experience of Ministry of Education officers and education experts;

(iii) analysis is conducted at the national level and does not showcase the school-level picture; and

(iv) reporting is highly expensive and demands significant resources. As a result, decision-making is based on a limited set of indicators. There is a need to conduct in-depth data analysis using proven methods and research.

Lack of definitions (i.e., what are quality schools and secondary education) and clear measurements has led to weak understanding of quality assessment at the national, regional, and school levels.

At present, the quality of schools is not assessed using officially approved methodology. In practice, schools report the scores of their graduates obtained during the Unified National Testing—an exam students take to gain admission to higher education institutions and obtain state-funded scholarships. The result of the exam is viewed as an indicator of a school’s quality.

The main elements of the existing national quality assurance system include various standards and regulations related to educational organizations, educational content, textbooks, among others. Compliance with these requirements is mandatory for all stakeholders in the education system. However, the assessment of quality remains an unresolved issue for the country. The need to further elaborate on the quality concept and measurements of quality in secondary education was noted in OECD’s recommendations for Kazakhstan.

In a move to address these issues, a project under the Joint Government of Kazakhstan and the Asian Development Bank Knowledge and Experience Exchange Program developed a methodology to assess school quality and created a prototype automated system to demonstrate how it will work in practice.

Developing a Comprehensive Assessment

One of the country’s strengths is the National Educational Database—a digital information system of administrative educational statistics with detailed and legally secured data on educational organizations, enrollment, students, and staff. The data is largely used to inform decision-making, policy making, and planning by relevant authorities. Kazakhstan educational statistics cover a large amount of information on the material and technical provision of schools, teaching staff, and educational achievements of students from the Unified National Testing scores.

On the back of the National Educational Database, the Automated School Quality Assessment Platform project was developed. It is the first preparatory step toward creating a national secondary education quality assessment framework. It proposes a scientifically proven approach and methods based on data availability, strengths, and weaknesses. The project developed a methodology for a comprehensive assessment of the quality of schools using clear criteria and assessment indicators.

The scope of work also included the development of a prototype of an information system that allows various stakeholders to visualize and compare the country's public secondary schools in various quality aspects. With the availability of digital statistical data, the prototype platform incorporated automated analytical instruments based on the developed methodology.

The prototype system would provide benefits to: (i) Ministry of Education officials (i.e., Сommittee for Quality Assurance) who can review the schools, (ii) local education authorities who can use the data for decisions on funding (per capita and targeted), and (iii) parents who choose schools based on the system’s data.

Defining a School Quality Theoretical Model

In developing the methodology, a comprehensive study of scientific literature was conducted. The review showed a variety of approaches for conceptualization and understanding of education quality, which led to the conclusion that the multidimensional concept cannot be reduced to a single indicator or a set of indicators related to one educational domain (i.e., educational achievements of students, teachers’ qualifications, institutional, legislative, and regulatory settings, resources).

Desk review of data sources and national background also revealed a lack of data on teaching practice, classroom experience, satisfaction of official stakeholders, social interaction, and leadership. These circumstances limit the choice of existing models and approaches for school education quality assessment. But data on many indicators in the National Education Database and other information systems paved the way for an automated assessment of school quality.

Following the analysis of the information on the Kazakhstan educational context, the most acceptable option was the input–output model—a hybrid model of resource inputs and educational achievements outputs. This implies that the output: education quality (i.e., student educational achievements) results from the amount of input: resource provision. This theoretical model was used for further selection and validation of the selected variables in the school quality assessment.

Assessment Methodology

The assessment methodology is a combination of index scoring assessment approach and machine learning clustering method. The methodology implies collection of 114 selected indicators grouped into two indices measuring domains of material provision quality and school staff quality. These indices are combined into an aggregate index of the quality of school material and resource provision, which captures school quality in line with the theoretical input–output model.

The next stage of assessment involves clustering of schools based on the indicators and indices into three clusters of schools with low, medium, and high quality of resource and material provision.

The assessment methodology demonstrates the correlation between the assessment results and students’ academic achievement variables.

Open Interactive Platform Prototype

The School Quality Assessment Program (SQAP)—based on the developed methodology—is the prototype for the information system that would allow stakeholders to assess the country's public secondary schools. It is an interactive online platform developed to combine school resource provision quality and parental satisfaction assessment.

SQAP assesses the resource and material provision quality of every public school. It calculates the results based on two group indices (material/technical equipment and quality of teaching staff) and one aggregate index of resource and material provision.

Screenshot of the page for data visualization.

SQAP shows a visual of the assessment on a map where one can find any school and compare its assessment results to other schools, including separate data on the school indicators.

The prototype contains a dashboard that presents a regional analysis of summary data broken down into schools with low, medium, and high quality of resource and material provision.

SQAP also has a built-in data constructor that allows the user to upload the necessary information for certain parameters. In addition, it has a separate section describing the assessment methodology, and data codebook.

Moving Forward

Aside from the developed methodology, the prototype includes a parental opinion survey tool—an online questionnaire on parental satisfaction, which could be used as an additional index beyond the selected input–output model. This is implemented as a separate module within the SQAP platform and could expand the dimension of school quality.

On 1 March 2022, the project’s outputs were presented to the Ministry of Education’s Committee for Education Quality Assessment. The objective is for national legislation and regulatory documents to adopt the concept of secondary education quality proposed in the project. There will be discussions in intersectoral working groups, adjustments to the platform codes, and integration of the platform with existing statistics information systems. This way, the online platform and National Educational Database can share data. After the Ministry accomplishes the above steps, pilot testing of the platform will commence.

Figure 1: Methodology for automated school quality assessment

 

Miraim Atanayeva
Deputy President, JSC Information Analytic Center Kazakhstan

Miraim Atanayeva has more than 15 years’ experience in education analytics and social sciences. She works with the Kazakhstan government and international organizations on education, sustainable development, public policy and child protection. She is an expert in the digitization of educational indicators and contributed in the creation of Kazakhstan’s National Educational Database.

Follow Miraim Atanayeva on

Arystan Galiyev
Project Officer, Kazakhstan Resident Mission, Asian Development Bank

Arystan Galiyev oversees works with the Kazakhstan government and developing partners in education, agriculture, tourism, and environmental, social, and governance. He has more than 15 years of project management experience in various industries and national companies, including in foreign direct investment attraction.

Asian Development Bank (ADB)

The Asian Development Bank is committed to achieving a prosperous, inclusive, resilient, and sustainable Asia and the Pacific, while sustaining its efforts to eradicate extreme poverty. Established in 1966, it is owned by 68 members—49 from the region. Its main instruments for helping its developing member countries are policy dialogue, loans, equity investments, guarantees, grants, and technical assistance.

Follow Asian Development Bank (ADB) on
Leave your question or comment in the section below:
Disclaimer

The views expressed on this website are those of the authors and do not necessarily reflect the views and policies of the Asian Development Bank (ADB) or its Board of Governors or the governments they represent. ADB does not guarantee the accuracy of the data included in this publication and accepts no responsibility for any consequence of their use. By making any designation of or reference to a particular territory or geographic area, or by using the term “country” in this document, ADB does not intend to make any judgments as to the legal or other status of any territory or area.