Author
Literature

CHAPTER 6. EVALUATION OF THE EFFECTIVENESS OF PUBLIC POLICY

 

6.4 Problems of public policy assessment


Assessment of the effectiveness of public administration is a key tool for analyzing and improving the performance of individual officials and authorities, as well as the state apparatus as a whole. It can be considered simultaneously as a method, a procedure, and an indicator for determining the effectiveness of government institutions. The effectiveness of government agencies depends directly on the performance of their officials, which, in turn, is determined by the degree to which they meet both specific and universal performance indicators.

World experience shows that countries with well-organized institutions of power and a well-developed system of performance assessment demonstrate high quality of public administration. This is reflected in the achievement of significant achievements in various spheres of public life, the growth of national wealth and the improvement of the level of well-being of citizens.

The effectiveness of public administration can be measured based on the following relationships:

- result-cost: efficiency is higher when lower costs provide the same or higher result.

- resources-needs: the ratio between the available capabilities of the system and the goals set, which reflects the level of rational use of potential.

Despite the presence of large-scale resources in the state system, their actual use does not always correspond to potential opportunities. Therefore, when evaluating effectiveness, it is important to take into account not only the results achieved, but also the degree of implementation of available resources. This means that the assessment of the effectiveness of public administration is multi-layered and combined, including the analysis of both real and potential indicators.

The classification of public administration efficiency is particularly important, since it allows us to identify not only current achievements, but also reserves for further improvement of efficiency. Optimization of mechanisms for evaluating and managing resources contributes to a more rational use of the potential of the state system and improving the quality of its work.

When assessing the effectiveness of public administration, a number of existing features should be taken into account:

1) The problem of authorship of the result obtained: often, in the process of evaluating the results of work in any field, the result obtained is interpreted as a merit of the state, when it can be a consequence of the activities of a particular social system, for example, public organizations, business.

2) The problem of sources of management results: whether it is a consequence of regulatory legal acts that contain conditions that contribute to the development of the individual, business and other subjects or the activities of managers in their posts, their professional qualities.

3) The problem of the approach to evaluating effectiveness, due to the complexity of evaluating managerial effectiveness as such.

4) The problem of identifying resources and their use, which are at the disposal of the authorities[68].

In 2010, the Decree of the President of the Republic of Kazakhstan «On the System of annual assessment of the performance of central state and local executive bodies of regions, cities of republican significance, and the capital» was signed[69], which is the fundamental document for conducting the assessment.

There are three main stages in the development of Kazakhstan's System for assessing the effectiveness of state bodies' activities:

1) Stage 12010. Preparation of the legislative framework, development of a basic methodology, and conducting a pilot assessment of three state bodies: two central state bodies and one local executive body. A full-scale assessment of the effectiveness of state bodies 'activities has been carried out in Kazakhstan since 2011.

2) Stage 22011. Conduct a pilot evaluation of 40 state bodies: 24 central state bodies and 16 local executive bodies, and then refine the methodology based on the recommendations of foreign experts.

3) Stage 32012. Introduction of amendments and additions to the Decree of the President of the Republic of Kazakhstan dated 19.03.2010 No. 954 «On the System of annual performance assessment of central state and local executive bodies of regions, cities of republican significance, and the capital», conducting a full assessment of 39 state bodies: 23 central state bodies and 16 local executive bodies, and developing a draft five-year Concept development of the System for evaluating the performance of government agencies in 2012-2016[70].

Currently, the assessment system has become one of the key tools for improving the public administration system and increasing the country's competitiveness. It allows for a comprehensive analysis of the strengths and weaknesses in the activities of state authorities.

The system for evaluating the effectiveness of state bodies is aimed at a detailed analysis of their activities by decomposing the results of work into a number of specific, measurable indicators. These indicators make it possible to track the dynamics of efficiency annually and ensure comparability of various state structures – line ministries, agencies, regional akimats and other authorities. Regardless of the scope of competence and level of management, the key performance criteria for any government agency are the fulfillment of strategic goals, the quality of public services provided, the development of human resources and the successful implementation of reforms.

In addition, the evaluation system plays an important role not only in measuring and comparing the effectiveness of State bodies, but also in their progressive development. Regular analysis and monitoring of indicators contributes to the systematic improvement of their performance. For example, over the years of using the evaluation system, it has been possible to significantly reduce the time required to provide public services, increase the level of automation of processes, improve financial discipline and transparency of government structures. Additionally, the level of civic participation in the discussion of legislative initiatives has increased, as well as the burden on state employees has decreased due to the optimization of work processes.

According to the Decree, the evaluation system includes several key blocks aimed at a comprehensive analysis of the activities of state bodies.

1. The «Achieving goals» block.

The assessment in this section covers the level of implementation of strategic tasks, the efficiency of using budget funds, and the compliance of financial planning with the stated goals. At the central level, the relationship between budget and goals is analyzed, and at the local level, financial discipline is analyzed. Special attention is paid to the quality of strategic and budget planning: in addition to achieving the set goals and indicators, cases of their over-fulfillment, adjustment of planned values, and dynamics of actual implementation are evaluated. Since 2019, one of the evaluation criteria has been ensuring the availability of budget information through the publication of the Civil Budget, as well as involving the public in the budget formation process through the «public participation budget» mechanism.

2. «Interaction of the state body with individuals and legal entities» block.

This section evaluates the level of public services provided, the effectiveness of handling citizens ' requests, and the degree of openness of government agencies. The quality of public services is assessed by such parameters as compliance with deadlines and the list of required documents, the level of satisfaction of the population, as well as the level of automation of service delivery. Handling complaints includes analyzing the timeliness of responses to complaints and requests, the quality of their initial consideration, and the degree of satisfaction of applicants with the measures taken. The openness of state authorities is considered through their activity on digital platforms, such as Open Government portals, timely updating of information, and response to requests, comments, and suggestions from citizens.

3. «Organizational development of a state body» block.

This section is aimed at analyzing the internal processes of state bodies, including personnel management and the use of digital technologies. HR management is assessed by such indicators as satisfaction with working conditions of state employees (based on surveys), the level of staff turnover, compliance with the principles of meritocracy and gender equality. The use of information technologies is analyzed through the degree of automation of the functions of state bodies, the integration of information systems, the quality of filling the architectural portal with data about information resources, as well as the relevance and reliability of the posted information.

Mandatory requirements for operational assessment criteria and indicators are the following:

- criteria and indicators should be based on official statistics, government data, and other verifiable information;

- the method of collecting and processing the initial information should allow checking the correctness of the received data;

- the definition of the indicator should ensure unambiguous interpretation by both the evaluating and evaluated state bodies;

- obtaining the necessary data should be carried out with the minimum possible expenditure of time and resources;

- criteria and indicators should be determined based on the need for continuous accumulation of data and ensuring their comparability for individual periods;

- criteria and indicators should cover all activities of the state body and focus on the high-quality performance of all state functions.

In addition to the above criteria, criteria and indicators of effectiveness can be used as criteria and indicators used in international practice. Criteria and indicators should reflect directly the activities of the State body[71].

The process of evaluating the effectiveness of a State body is shown in Figure 6.4.1.

 

Figure 6.4.1. – Public policy effectiveness assessment process

 

DATA COLLECTION

Authorized evaluation bodies collect, process, and recheck the reporting information.

EVALUATION BY DIRECTIONS

Conclusions on the evaluation results are formed by directions.

DISTRIBUTION AND APPEALS

Conclusions are sent to government bodies. In case of disagreement with the evaluation results, the evaluated government bodies may submit an objection with supporting documents.

FINAL CONCLUSIONS BY EVALUATION BLOCKS

Final conclusions are formed by evaluation blocks, taking into account the results of the appeals.

EXPERT CONCLUSIONS ON EVALUATION RESULTS

The Presidential Administration forms expert conclusions based on the main evaluation results and systemic findings.

APPROVAL OF EVALUATION RESULTS

Evaluation results are submitted for review to the Head of State.

METHODOLOGY IMPROVEMENT AND MONITORING OF INSTRUCTIONS

Authorized evaluation bodies refine methodologies taking into account the proposals of government bodies. Monitoring of the implementation of instructions issued following the evaluation is carried out.

 

Analysis of the results of evaluating the effectiveness of government agencies in Kazakhstan revealed the strengths and weaknesses of this system.

The strengths of evaluating the effectiveness of government agenciesare the following factors:

- political support from the country's leadership;

- availability of alegislative base;

- an evolutionary approach.

- availability of pilot experience;

- professionalism of appraisers;

- application of assessment at both the central and local levels of government;

- comprehensive explanatory work in state bodies,mass media, etc.

Weaknesses in assessing the effectiveness of government agenciesare such factors as:

- strict focus on results;

- presence of a conflict of interest.

- non-residual institutionalization;

- insufficient efficiency of assessment procedures;

- insufficient level of dialogue between the evaluated andevaluating state bodies;

- weak involvement of non-governmental organizations (NGOs);

- lack of publication of the evaluation results in the mass media (mass media)[72].

Currently, the system for assessing the effectiveness of public administration in Kazakhstan is being improved and adapted to national conditions. Despite the progress made in the development of evaluation mechanisms, there are still unresolved issues related to the formation of criteria, methodology for conducting evaluation studies and their integration into the management decision-making process at all levels of government. Key challenges include the following:

- the use of «unmanaged» criteria. Some of the indicators used, such as the mortality rate of the population, infant mortality, life expectancy, as well as the degree of involvement of citizens in social and cultural processes, are only indirectly related to the results of public administration. In the short term, their dynamics may depend on a variety of factors beyond the control of government agencies, which makes them subjective and not always reliable in assessing the effectiveness of management decisions.

- insufficient attention to public assessment. In the existing systems of monitoring the effectiveness of public administration in Kazakhstan, indicators reflecting the perception of the activities of government bodies by the population are poorly represented. The inclusion of public opinion and the level of trust in state institutions in the evaluation criteria would make the system more objective and focused on the real needs of citizens;

- mismatch of indicators with regional specifics. Kazakhstan is a state with a high degree of territorial and socio-economic diversity. However, the same performance indicators for all regions do not take into account their specific conditions, which reduces the objectivity of assessments. For example, the criteria applicable to economically developed megacities may not reflect the realities of rural areas, and the estimated parameters relevant for industrial regions are not always suitable for agricultural regions.

- authorized state bodies recognize that the current assessment model has a long cycle of implementation, which is associated with the need to collect a significant amount of reporting and supporting information from the evaluated state bodies, as well as with subsequent verification of this data on the ground. As a result, the final analysis and summary of overall results for the past year can take up to 10 months, which reduces the relevance of conclusions and limits the ability to quickly respond to identified shortcomings. Additionally, the complexity of the process is due to the fact that most of the reporting data is provided by the evaluated bodies themselves, and only 2-3 days are allocated for their rechecking per state body. This significantly complicates the task of ensuring the accuracy of information, increasing the risk of inaccuracies and bias in the assessment.

To improve the effectiveness of the public administration assessment system in Kazakhstan, it is important not only to develop more flexible and differentiated criteria, but also to strengthen the role of public opinion, as well as to take into account regional characteristics when forming performance indicators of state bodies.

 


[68] Galiullin T. T. Modern Problems in Assessing the Effectiveness of Public Administration / T. T. Galiullin // Vestnik PAGS. – 2016. – No. 4 – Pp. 28–32. – URL: https://cyberleninka.ru/article/n/sovremennye-problemyotsenki-effektivnosti-gosudarstvennogo-upravleniya. Accessed on 25.03.2025.

[69] Presidential Decree of the Republic of Kazakhstan No. 954 dated March 19, 2010 “On the System of Annual Evaluation of the Effectiveness of Activities of Central Government and Local Executive Bodies of Regions, the City of Republican Significance, and the Capital” https://adilet.zan.kz/rus/docs/U100000954. Accessed on 25.03.2025.

[70] Dzhunusbekova G. A. Improvement of the System for Evaluating the Effectiveness of Public Authorities in the Republic of Kazakhstan.// Viešoji politika ir administravimas. – 2015. – T. 14. – Nr. 2. – С. 256.

[71] Presidential Decree of the Republic of Kazakhstan No. 954 dated March 19, 2010 “On the System of Annual Evaluation of the Effectiveness of Activities of Central Government and Local Executive Bodies of Regions, the City of Republican Significance, and the Capital” https://adilet.zan.kz/rus/docs/U100000954. Accessed on 25.03.2025.

[72] Walker R. M., Boyne G. A. Introduction: Determinants of Performance in Public Organizations. Public Administration. – Vol. 87. – No.3, 2009 (433–439).