During the last twenty years, the challenges of performance measurement in both the public and private sectors have been widely discussed (Behn, 2003; Carter et al., 1992; Hood, 2006; Kaplan and Norton, 2004; Micheli and Neely, 2010; Pollitt and Bouckaert, 2004; Smith, 1995). Financial measures based on numbers generated by the traditional financial accounting system, such as profits, have been found to be not sufficient and out of date in measuring organizations’ performance (Curtis, 1985). For example, at that time because of no enough funding, NHS (Sehested, 2002) could not perform normally; people felt more dissatisfied with public services provided and government spending on these services were planned to be cut. Consequently, organizations have developed a number of non-financial measures to improve this situation (Kidwell et al., 2002). Also the information performance evaluation process produces has become more comprehensive (Dwight et al., 2009).
Nowadays, governments are expressing high interest in the performance evaluation (Micheli, 2008) with the increasing pressures caused by higher level demand for the accountability in the public sector, such as how do the public revenues (e.g. taxation) spend and report. However, it is not easy and problematic to effectively and efficiently evaluate the performance in the public sector.
The first two parts of this essay will briefly illustrate the background of performance measurement in both private and public sectors. Then in the third part, it aims to discuss the problems and challenges of performance evaluation in the public sector when non-financial measures are mainly used. And finally, a summary and recommendation for further research will be given.
2. Performance measurement in private sectorFor the private sector, such as the profit-oriented organization, the primary measure of performance is profit and the majority of performance measurement activities are still based on the financial statements. From these statements which mainly include income statements, balance sheets and cash flow statements, we can easily know what resources were purchased, how they were used and the cost of them in each business organization. The reason to use profit as an indicator to evaluate how well a company performs is probably because within a competitive market, the prices of goods and services can be easily and relevantly valued.
However, some companies actually are not operating within a competitive market, so the profit itself is not sufficient to evaluate the organizations’ performance. In addition, such financial measures are not that useful in helping a company to achieve its strategic objectives (Kaplan and Norton, 1992). In response, Balanced Scorecard (BSC) was developed by Kaplan and Norton (1992) to overcome this limitation which links performance to strategy using both financial and non-financial performance measures to guide future success.
3. Performance measurement in public sectorLike the private sector, public sector organizations around the world also face pressures to improve their service quality, lower the costs and become more responsible for stakeholders’ needs. Some scholars argue that in the public sector performance evaluation processes can be improved by using some specific techniques normally adopted in private sector (Broadbent and Guthrie, 1992; Hoque and Moll, 2001), such as cost-volume-profit analysis, standard cost accounting, and return on capital employed.
However, since most of public sector organizations aim to provide a service for the service users rather than earn a profit, and services are provided free at the point of delivery and financed by taxation, the revenue cannot be used as a ideal measure to evaluate how well each service be received by its customers (Sorensen and Grove, 1977). Therefore, the amount of profit traditionally used as a performance measure in profit-oriented organizations is not suitable for the public sector. Instead, this type of organizations typically publishes non-financial output measures. Nevertheless, the values of goods and services in public sector are more difficult to identify and measure than the one in private sector, how to efficiently and effectively evaluate the performance in the public sector becomes problematic.
3.1 Elements of performance measurement in public sector
Measures of performance used in the public sector performance evaluation process include three basic elements: inputs (the resources consumed and primarily be measured by using costs and non-financial measures), outputs (direct results of services and can be measured by using non-financial measures), and outcomes (ultimate results of services and can be evaluated mainly by qualitative assessments such as interviews). Within the evaluation process, inputs, outputs and outcomes are hierarchical; the lowest-level measures (inputs) are easier to measure but of limited relevance to the goal, while the highest-level measures (outputs) are more relevant but difficult to measure, so less reliable (Jones and Pendlebury, 2010). For example, as for a secondary school, number of teachers as the input which is easier to measure but is not directly relevant to the goal of school; while the quality of school as the outcome is difficult to measure but is more relevant to the school’s objective.
Since public sector organizations generally lack clear objectives, which makes them become problematic to link the inputs to the outputs and outcomes (Johnsen, 2000). In addition, those organizations will have some difficulties when quantifying the main performance measures such as customer satisfaction and quality of service (Jackson, 1990) during the process of performance evaluation. Hence, it is hard to avoid involving large amounts of discretion made by human resources.
3.2 Economy, Effectiveness and Efficiency
Audit Commission (1986) emphasized two key measures of performance: efficiency and effectiveness. Efficiency was defined as providing specified volume and quality of goods and services with the lowest level of resources. It is traditionally measured by the output/ input ratio and used to link inputs with non-financial output. The greater the ratio, the more efficient the organization is. Effectiveness refers to the provision of right services and goods which enables the organizations to meet their objectives. However, it is only concerned with outputs and outcomes, and says nothing about how much an organization spend to achieve a certain objective. Hence, effectiveness contributes little or nothing in improving the performance of organizations, mainly because when there is no restriction on the budgets and no control on the costs of inputs, almost all objectives can be satisfied. There is also a third element-economy which is emphasized in the context of purchases from outside and defined as providing specified quality and quantity with the lowest possible costs. It only concerns inputs but no consideration about the organization’s final objectives.
From the above discussion, we can clearly know that it is meaningless if these three measures operate separately. For example, sometimes a program could be very efficient but not effective and also you could do a wrong thing very well. Therefore, it is necessary to judge these three elements together because you don’t want to spend public money on the cheapest inputs but not achieve the goals after all.
3.3 Challenges of performance measurement in public sector
There are various challenges public sector organizations must face when they evaluate their performance (Jones and Pendlebury, 2010). Even though there are some exceptions, most public sector organizations are generally hard to provide stakeholders with credible and relevant information, let alone to use this information in the decision making process of budget.
The first issue is about the measurement of costs, such as when the full costs are not relevant within the organization, how to evaluate the performance. Additionally, a big part of costs are not directly linked with outputs and outcomes, so it must involve some other factors that affect the costs. Secondly, the non-financial output measures are less reliable than the financial ones mainly because just limited non-financial information can be controlled by organizations. Thirdly, although non-financial measures may be easy to count and should be reliably measured, it is very difficult to establish the causal relationships between inputs, outputs and outcomes due to that most of the performance is being measured only once, such as students normally do not wish to be educated twice for the same thing, and also the ultimate objective is not clear at all. Hence, it is so common that there will have some unintended outcomes which involve the interaction of many factors, planned and unplanned.
Fourthly, non-financial output measures are not comparable between services and just focus on very specific characteristics. However, as we all know, the more specific the focus, the more useful the measurement is. In addition, it is hard to trade-off between the demand of complex and simple performance measures by various levels of people, such as politicians and citizens. Each type of measures has its own advantages and disadvantages. Last but not least, non-financial measures just focus on what the organizations can control, and concern nothing about the links to inputs and the ability to be audited which required by accounting.
3.4 Problems with establishing the performance measurement techniques
In order establish performance evaluation techniques in the public sector, social indicators, program-planning-budgeting systems (PPBS) and cost-benefit (C/B) analyses are mainly used (Sorensen and Grove, 1977). However, there exist many problems with them. For the social indicators, the meaning of statistics may be questioned because of the method of data collection and also these indicators are too general to be applied in specific areas. The main problem with the remaining two techniques is the measurement of outcome. For PPBS, it cannot convert the organization’s objectives into measurable outcomes, while for C/B analyses, non-monetary outcomes are almost impossible to transform into monetary ones. Therefore, these three techniques are not that useful in the real world.
3.5 Methods for improving performance evaluation techniques
The problems with performance evaluation techniques including social indicators, PPBS and C/B analyses mentioned above place emphasis on the usefulness of outcome measurements. To solve those problems, financial inputs should be related to non-financial outcomes for specific activities from a cost analytic perspective. Techniques such as cost accounting and cost-finding for programs are used to link resource consumed to non-monetary outcomes (Sorensen and Grove, 1977).
There are two types of cost-analytic techniques, cost-outcome and cost-effectiveness (Sorensen and Grove, 1977). Cost-outcome can be defined as using specified resources consumed to reach a change in the ultimate objective compared with previous measure of performance, in order to improve the health symptoms or social performance, while cost-effectiveness can be defined as comparing cost-outcomes results to identify the best outcome with consideration of the costs. Also, this technique can be carried out on the basis of cost-outcomes information.
In profit-oriented organizations, decision making focuses mainly on the profit figures, while in public sector organizations, especially the non-profit organizations, there is no such comprehensive measure of output and outcome evaluations are generally affected by factors such as time patterns, multiple outcomes, effects among different populations and research design issues (Sorensen and Grove, 1977). In addition, it is difficult to find accurate measures of specific activities and then to estimate whether the change in inputs will influence the outputs. The reason for this is mainly because non-financial outcome measurements are typically adopted in non-profit performance measurement process. As long as the organization’s objective becomes clearer, the internal and external accountability demands in non-profit organizations will be met and the quality of services will be improved through using the cost-outcome and cost-effectiveness techniques.
4. ConclusionWhile performance measurement in private sector is almost wholly limited to financial measures, public sector mainly uses non-financial measures to evaluate. However, there exist some problems when using this type of measures. The basic one is the non-monetary outputs measurement. How to efficiently and effectively establish the causal relationships between financial inputs and non-financial outputs and outcomes has been widely argued. Three types of performance evaluation techniques, which are social indicators, PPBS and C/B analyses, are found to have some problems in outcome measurements. Cost-analytic techniques are developed by using cost-outcome and cost-effectiveness methods, to solve these problems and link the monetary inputs to non-monetary outputs and outcomes. However, the difficulties in interpreting cost-effectiveness measures should not be ignored. Because of this, it is still not easy to use them in the real world and there is a need for further research on this field.
5. References
Audit Commission (1986), ‘Making a Reality of Community Care’,London: HMSO.
Broadbent, J. and Guthrie (1992), ‘Changes in the Public Sector: A Review of Recent Alternative Accounting Research’, Accounting, Auditing & Accountability Journal, 5(2).
Behn, R. D. (2003), ‘Why measure performanceDifferent purposes require different indicators’, Public Administration Review, 63: 586-606.
Carter, N., Klein, R. and Day, P. (1992), How organisations measure success – The use of performance indicators in government.London: Routledge.
Curtis, Donald A. (1985), ‘The Modern Accounting System”, Financial Executive, January – February, pp. 81-93.
Dwight R., Phusavat K., Anussornnitisarn P., Helo P. (2009), ‘Performance measurement: roles and challenges’, Industrial Management & Data Systems, 109 (5): 646 – 664.
Fryer K., Antony J., OgdenS. (2009), ‘Performance management in the public sector’, International Journal of Public Sector Management, 22 (6): 478-498.
Greiling, D. (2005), ‘Performance measurement in the public sector: the German experience’, International Journal of Productivity and Performance Management, 54 (7): 551-567.
Hood, C. (2006), ‘Gaming in Target world: The targets approach to managing British public services’, Public Administration Review, 66(4): 515-521.
Hoque, Z. (2008), ‘Measuring and reporting public sector outputs/outcomes. Exploratory evidence from Australia’, International Journal of Public Sector Management, 21(5):468-493.
Hoque, Z and Moll, J (2001), ‘Public sector reform: Implications for accounting, accountability and performance of state-owned entities – an Australian perspective’, International Journal of Public Sector Management, 14(4): 304–26.
Jackson, P. (1990), ‘Measuring Performance in the Public sector”, Collected Papers from The Second Public Sector Conference of the Federation des Experts Comptables Europeens (FEE), pp. 11-22.
Jones, R. and Pendlebury, M. (2010), Public Sector Accounting (6th edition).UK: Prentice-Hall.
Johnsen, A. (2000), Public sector performance measurement uncertainty, ambiguity and epistemic communities (9th edition), Department of Accounting and Business Method, The University of Edinburgh, pp. 1-40.
Kaplan, R. S. and Norton, D. P. (1992), ‘The balanced scorecard – Measures that drive Performance’, Harvard Business Review, January-February, pp.71-79.
Kaplan, R. S. and Norton, D. P. (2004), ‘Strategy maps: Converting intangible assets into tangible outcomes’,Boston: Harvard Business School Press.
Kidwell, Linda A. et al. (2002), ‘New Management Techniques: An International Comparison’, The CPA Journal,New York: February, 72(2): 63.
Micheli, P. (2008), ‘Public sector performance: do we want to know more?’, KPMG’s Public Governance Institute magazine.
Micheli, P. and Neely, A. (2010), ‘Performance Measurement in the Public Sector in England: Searching for the Golden Thread’, Public Administration Review, 70: 591–600.
Propper C. and Wilson D. (2003), ‘The Use and Usefulness of Performance Measures in the Public Sector’, Oxford Review of Economic Policy, Oxford University Press, 19(2): 250-267.
Pollitt C. and Bouckaert, G. (2004), Public Management Reform: A Comparative Analysis (2nd edition).Oxford:OxfordUniversity Press.
Sehested, K. (2002), ‘How new public management reforms challenge the roles of professionals’, International Journal of Public Administration, 25(12): 1513-37.
Smith, P. (1995), ‘Performance indicators and outcome in the public-sector’, Public Money and Management, 15(4): 13-16.
Sorensen, I. E. and Grove, H. D. (1977), ‘Cost outcome and cost-effectiveness analysis: emerging nonprofit performance evaluation techniques’, The Accounting Review, 52(3): 658-675.
