During the last two decades, the challenges of measuring performance in both the public and private sectors have been widely discussed (Behn, 2003; Carter et al., 1992; Hood, 2006; Johnson and Kaplan, 1987; Kaplan and Norton, 2004; Neely, 1999; Pollitt and Bouckaert, 2004; Smith, 1995). Financial measures based on numbers generated by the traditional financial accounting system, such as profits, have been found to be not sufficient and out of date in measuring organizational performance (Curtis, 1985). For example, during that period, within NHS (Sehested, 2002), operations could not be performed due to lack of funding; there was widespread dissatisfaction with public services generally and the government was trying to reduce spending. Accordingly, organizations have implemented a number of non-financial measures to control their costs and improve their goods and services (Kidwell 2002). Performance measurement has evolved from merely producing accounting-related to more comprehensive information that contains both financial and non-financial information (Wilson et al., 2003).
With the increasing pressures of citizens and legislation that demand more responsibility in spending of public revenues and transparency in reporting on the achieved results, governments are demonstrating growing interest in the measurement of performance (Micheli, 2008). However, it is not easy and problematic to effectively and efficiently evaluate the performance in the public sector.
This essay aims to discuss the problems of performance evaluation in the public sector and some prospective techniques developed to improve them. The first part will briefly illustrate the performance measurement in private sector. Then, in the follow part, the roles and challenges of performance measurement in public sector will be discussed. And finally, a conclusion will be drawn from what had been argued.
Performance measurement in private sector
In the private sector, such as the profit-oriented organization, the principle measure of successful performance is profit and the majority of performance measurement activity is still based on the financial statements. From these statements which mainly include income statements, balance sheets and cash flow statements, we can easily know what resources were purchased, how they were used and the cost of them in each business organization. The reason to use profit as an indicator to evaluate how well a company performs is probably because within a competitive market, the prices of goods and services can be easily and relevantly valued.
However, some companies are not operating within a competitive market, so the profit itself is not sufficient to evaluate the organizations’ performance. In addition, such financial measurements contribute only little or nothing in helping an organization to achieve its strategic goals (Euske et al., 1993; Ghalayini et al., 1997; Jagdev et al., 1997; Kaplan and Norton, 1992; Nanni et al., 1992; Neely, 1995). In response, Balanced Scorecard (BSC) was developed to overcome this limitation by Norton and Kaplan (1992) which is a strategic performance management tool that links performance to strategy using both financial and non-financial performance measures and generally used by managers to guide future competitive success (Kaplan and Norton, 1996).
The roles and challenges of performance measurement in public sector
Like the private sector, public sector organizations around the world face pressure to improve service quality; lower their costs; become more accountable, customer-focused and responsive to stakeholders’ needs. There is an argument that performance evaluation processes within the public sector organizations can be improved through using the performance measurement techniques of private sector (Broadbent and Guthrie, 1992; Olson et al., 1998; Hoque and Moll, 2001), such as cost-volume-profit analysis, standard costing, and return on capital employed. However, since most of public sector organizations aim to provide a service for the service users rather than earn a profit and services provided free at the point of delivery and financed by taxation, the revenue cannot be used as a ideal measure to evaluate how well each service be received by its customers. Therefore, the amount of profit traditionally used as a performance measure in profit-oriented organizations is not suitable for the public sector. Instead, this type of organizations typically publishes non-financial output measures. In addition, due to that the values of goods and services in public sector are more difficult to identify and measure than the one in private sector, how to efficiently and effectively evaluate the performance in the public sector becomes problematic.
Elements of performance measurement in definitive governments
Measures of performance used in the public sector performance evaluation process include three basic elements: inputs (the resources consumed by governments and primarily be measured by using costs and non-financial measures), outputs (direct results of program activities and can be measured by using non-financial measures), and outcomes (broad results of program activities and be evaluated mainly by qualitative assessments such as questionnaires and interviews). The non-financial inputs, outputs and outcomes in this evaluation process are hierarchical; the lowest-level measures (inputs) are easier to measure but of limited relevance to the goal, while the highest-level measures (outputs) are more relevant but difficult to measure, so less reliable. For example, as for a secondary school, number of teachers as the input which is easier to measure but is not directly relevant to the goal of school; while the quality of school as the outcome is difficult to measure but is more relevant to the school’s objective.
Since public sector organizations lack clear objectives, which make it become problematic to link the inputs to the outputs and outcomes (Johnsen, 2000). As a good and service provider, the public sector will be faced with the difficulties of quantifying their main performance measures such as customer satisfaction, and quality of service (Jackson, 1990); which also mainly rely on human resources who have discretion over their efforts and hence need consistent monitoring and directing towards the organization’s goals (Neely et al., 1995). Moreover, public sector organizations are led by elected officials who are voted into office and are accountable to their voting stakeholders. These stakeholders may not be the consumers or end users of public services, such as environmental protection, tax collectors, motor vehicle registration, and immigration services (Battle, 1994).
Economy, Effectiveness and Efficiency
Since there are some other factors affecting the performance of public sector, it is difficult to establish the relationships between financial inputs and non-financial outputs and outcomes. Audit Commission (1986) emphasized two key measures of performance: efficiency and effectiveness. Service efficiency was defined as the “provision of specified volume and quality of service with the lowest level of resources capable of meeting that specification”. It is measured by the ratio: output/input and used to link inputs (usually measured by monetary items) with non-financial outputs (measured by either monetary or non-monetary items). The greater the ratio, the more efficient the organization is. Effectiveness can be defined as “providing the right services to enable the local authority to implement its policies and objectives.” It is only concerned with outputs and outcomes, and say nothing about how much an organization spend to achieve a certain objective. Hence, effectiveness contributes little or nothing in improving the performance of organizations, mainly because when there is no restriction on the budget almost all objectives can be satisfied. There is also a third element – economy, although included in efficiency, but specifically emphasized in the context of purchases from outside, defined as “the lowest possible cost consistent with the specified quality and quantity.” It only concerns inputs but no consideration about the organization’s objectives. Thus, it is meaningless if they operate alone. Sometimes a program could be very efficient but not effective, for example, you could do a wrong thing very well. Besides, it is necessary to judge these three elements together because you don’t want to spend public money on the cheapest inputs but not achieve the goals after all.
Challenges of performance measurement in public sector
There are various challenges public sector organizations must face when they evaluate their performance. The first issue is about the measurement of costs, such as when the full costs are not relevant within the organization, how to evaluate the performance. Additionally, a big part of costs are not directly linked with outputs and outcomes, so it must involve some other factors that affect the costs. Secondly, the non-financial output measures are less reliable than the financial ones mainly because just limited non-financial information can be controlled. Thirdly, although the non-financial measures may be easy to count and should be reliably measured, it is very difficult to establish the relationships between inputs, outputs and outcomes due to that most of the performance is being measured only once, such as students cannot be educated twice for the same thing, and also the ultimate objective is not clear at all. Hence, it is so common that there will have some unintended outcomes. Fourthly, non-financial output measures are not comparable between services and just focus on very specific characteristics. However, as we all know, the more specific the focus, the more useful the measurement is. In addition, it is hard to trade-off between the demand of complex and simple performance measures. Each type of measures has its own advantages and disadvantages. Last but not least, non-financial measures just focus on what the organizations can control, and concern nothing about the links to inputs and the ability to be audited which required by accounting.
Problems with establishing the performance measurement techniques
In order establish performance evaluation techniques in the public sector, social indicators, program-planning-budgeting systems (PPBS) and cost-benefit (C/B) analyses are mainly used. However, there exist many problems with them. For the social indicators, the meaning of statistics may be questioned because of the method of data collection and also these indicators are too general to be applied in specific areas. The main problem with the remaining two techniques is the measurement of outcome. For PPBS, it cannot convert the organization’s objectives into measurable outcomes, while for C/B analysis, non-monetary outcomes are impossible to transform into monetary ones. Therefore, these three techniques are not that useful in the real world (James E, Sorensen; Hugh D. Grove, 1977).
Methods for improving performance evaluation techniques
The problems with performance evaluation techniques including social indicators, PPBS and (C/B) analyses mentioned above places emphasis on the role of outcome measures. To solve those problems, monetary inputs should be related to non-monetary outcomes for specific programs in a cost analytic perspective. Cost-analytic techniques include approaches ranging from cost accounting and cost-finding for programs, units of services and episodes to techniques which link resource consumption to nonmonetary outcomes (James E, Sorensen; Hugh D. Grove, 1977).
There are two types of cost-analytic techniques, cost-outcome and cost-effectiveness (Quade, 1967; Goldman, 1967; Levin, 1974; Fishman, 1974; Yates, 1975). Cost-outcome refers to the programmatic resources consumed to achieve a change in a relative measure of performance, while Cost-effectiveness is the comparison of cost-outcomes to identify the most beneficial outcome to cost of programs, modalities or treatment techniques. According to James E, Sorensen and Hugh D. Grove (1977), cost-effectiveness analysis can be carried out, based on cost-outcome information.
Assessing outcomes turns out to be an even more challenging than costing for nonprofit services, because non-monetary outcome assessment is currently employed in nonprofit performance evaluation process. In profit-oriented organizations, performance measurement is mainly based on the profit figures. But in public sector organizations, there is no such comprehensive measure and there are few good ways of estimating whether additional inputs will generate coordinate extra outputs. The basic problem is that there is no meaningful measure of output, and outcome evaluations are influenced by time patterns, multiple outcomes, effects among different populations, simple vs. complex evaluations and research design issues. As the conceptual approach and role of outcome assessment becomes clearer, improvements in the quality of the output side of the cost-outcome approach can be expected to meet external and internal service accountability demands.
Conclusion
While performance measurement in private sector is almost wholly limited to financial measures, public sector mainly uses non-financial measures to evaluate. However, there exist some problems when using this type of measures. The basic one is the non-monetary outputs measurement. How to efficiently and effectively establish the causal relationships between financial inputs and non-financial outputs and outcomes has been widely argued. Three types of performance evaluation techniques, which are social indicators, PPBS and (C/B) analysis, are found to have some problems in outcome measurements. Cost-analytic techniques are developed by using cost-outcome and cost-effectiveness methods, to solve these problems and link the monetary inputs to non-monetary outputs and outcomes. However, the difficulties in the interpretation of cost-effectiveness measures cannot be underestimated and it is still not easy to use them in the real world. ReferenceAudit Commission (1986), ‘Making a Reality of Community Care’, London: HMSO.
Broadbent, J. and Guthrie (1992) ‘Changes in the Public Sector: A Review of Recent Alternative Accounting Research’, Accounting, Auditing & Accountability Journal, 5(2).
Battle B.F. (1994), ‘Going public: applying total quality management in the public sector’, The Journal of State Government, 67(2):16–23.
Behn, R. D. (2003), ‘Why measure performanceDifferent purposes require different indicators’, Public Administration Review, 63: 586-606.
Carter, N., Klein, R. and Day, P. (1992), How organisations measure success – The use of performance indicators in government. London: Routledge.
Hood, C. (2006), ‘Gaming in Target world: The targets approach to managing British public services’, Public Administration Review, 66(4): 515-521.
Hoque, Z and Moll, J (2001), ‘Public sector reform: Implications for accounting, accountability and performance of state-owned entities – an Australian perspective’, International Journal of Public Sector Management, 14(4): 304–26.
Jones, R. & Pendlebury, M. (2000), Public Sector Accounting (6th edition). UK: Prentice-Hall.
Kaplan, R. S. and Norton, D. P. (1992), “The balanced scorecard – Measures that drive Performance”, Harvard Business Review, January-February, pp.71-79.
Kaplan, R. S. and Norton, D. P. (2004), ‘Strategy maps: Converting intangible assets into tangible outcomes’, Boston: Harvard Business School Press.
Micheli, P. (2008), ‘Public sector performance: do we want to know more?’, KPMG’s Public Governance Institute magazine.
Neely, A. D. (1999), ‘The performance measurement revolution: Why now and where next’, International Journal of Operations and Production Management, 19: 205-228.
Pollitt C. and Bouckaert, G. (2004). Public Management Reform: A Comparative Analysis (2nd edition). Oxford: Oxford University Press.
Smith, P. (1995), ‘Performance indicators and outcome in the public-sector’, Public Money and Management, 15(4): 13-16.
Sorensen, I. E. and Grove, H. D. (1977) ‘Cost outcome and cost-effectiveness analysis: emerging nonprofit performance evaluation techniques’, The Accounting Review, 52(3): 658-675.
Sehested, K. (2002), “How new public management reforms challenge the roles of professionals”, International Journal of Public Administration, 25(12): 1513-37.
Wilson, C., Hagarty, D. and Gauthier, J. (2003), “Results using the balanced scorecard in the public sector”, Journal of Corporate Real Estate, 6 (1): 53-64.
