Archive for the ‘Government’ Category

Professional Services Performance Management – performance criteria and measures

Monday, August 16th, 2010

Professional Services have some particularities that make them different from commercial activities in many respects, such as:

  • The way they relate with the customer: they see themselves / are seen as subject matter experts in an area, thus an equilibrium between their advice and the customer needs must always be maintained;
  • Internally, they might find it difficult to establish a culture and work practices that will reconcile both the more experienced (in many cases, the founders) and the more young and enthusiastic or to correctly assess the contribution and reward each individual within the organization.

Thus, there are at least two perspectives that the Professional Services firms must consider when addressing both their external and internal-focused performance issues.

On one hand, they have to attain the clients’ expectations in terms of quality, timeliness, adequacy of the solutions and services they provide (and these will often be formally measured by the customer himself).

On the other hand, they have to be able to attain internal alignment and correct evaluations of performance of the teams and individual consultants.

Regarding the first perspective, especially in the case of clients that are public authorities that resort to private consulting services, formal evaluation methodologies are being used.

In the public sector, it is considered that consultants are used to deliver a specific project or piece of research, also based on their recognition of the added value they can bring over a civil servant (UK Office of Government Commerce 2006).  The UK based Office of Government Commerce (OGC), together with Management Consultancies Association and the Institute of Management Consultancy developed a framework for the value criteria in working with Professional Services representatives, aimed at identifying and expressing the value expectations at the beginning of the assignment that will also provide the baseline against which delivery can be assessed in the post-assignment stage (OGC 2006). In this context, an important focus is put on the flow of the various stages in the project, from the business case (which identifies the value needed to be delivered), to the development of the specifications (explaining how the value will be created), followed by the delivery of the value and, finally, the measurement of the value delivered:

Source: OGC (2006)

The importance of having clear and correct performance standards for the Professional Services firms, mainly in the case of clients in the public sector, is outlined also by the fact that contracting them is not a matter of personal option of the public representative in charge, but it follows a specific public procurement procedure and it involves reporting to stakeholders on various levels, ultimately to the community itself, which is the final beneficiary (although not always directly) of the services.

Mike Cameron from Bozz Allen Hamilton (2004) suggested a three-levels Performance Work Statement (PWS) to be used in the procurement of Professional Services:

  • Performance Requirements (describes measurable results that must be attained)
  • Performance Standards (refers to the acceptable quality of these results)
  • Performance Measurement (describes how measurement will be done)

This framework must be able to answer two simple questions (Cameron 2004):

  • What problem are we trying to solve?
  • How will we know when we solved it?

Thus, Professional Services firms must be aware and have good knowledge of what are their clients’ requirements in terms of performance and further on translate these standards and criteria into how they deliver their work and how they can make an assessment of their own. This assessment will not only reflect the alignment to customer requirements, but also the internal competences (in terms of people, resources, processes etc.).

smartKPIs.com contains more than 100 popular Key Performance Indicators (KPIs) used in the Professional Services industry, for businesses in Consulting, Accounting Services, Recruitment & Employment, Engineering or Legal Practice. To explore the KPI examples, follow: http://www.smartkpis.com/kpi/industries/professional-services/.

References

Cameron, M (Bozz Allen Hamilton) 2004, Performance Criteria for Professional Services, Service Acquisition Center of Excellence Open Ceremony, November 18, 2004.

smartKPIs.com, Key Performance Indicator Examples in the Professional Services Industry, available at: http://www.smartkpis.com/kpi/industries/professional-services/ (accessed 13th July 2010).

UK Office of Government Commerce 2006, What value do consultants add?, available at: http://www.ogc.gov.uk/documents/PSF_Value_criteria_template_BA_13sep.pdf (accessed 13th August 2010).

R&D Investment and Performance around the world – a European Commision report

Sunday, July 11th, 2010

The Science, Technology and Competitiveness key figures report 2008/2009, released by the European Commission, presents a thorough analysis and overview of the evolution of  R&D investment and performance in Europe. The report is considered an attempt to monitor the progress and efficiency of implementing performance measures by the European research system.

Research is considered a key competitive asset in a global world as science, technology and patent applications are more widely distributed every year. Between 2000 and 2006 the investment in R&D increased in Europe by 14,8%, comparable with the US and Japan, where figures show a growth of 10.1 % and 21.9 % respectively.

Source: European Commission, 2008

The analysis of the participation shares in global R&D indicates that almost 80 % of researchers work outside the EU, 75 % of gross domestic expenditure on R&D (GERD) is executed in other world regions, and 69 % of patent applications are made outside the EU. This indicates a declining world share of GERD and patent applications, both for the US and for the European Union (European Commission, 2008).

Source: European Commission, 2008

The main conclusions of the report are:

  • Despite encouraging progress on increasing the amount of investment in R&D, the R&D intensity of EU-27 has remained unchanged, as countries with increasing R&D intensities do not have very high shares of EU-27 GDP.
  • Higher returns for private investment in R&D favor structural change, such as high-growth SMEs and higher demand and a single market for research-intensive products (European Commission, 2008).
  • European Research Area (ERA) integration is a key competitive factor for increasing the effectiveness of the European research system, due to its cost-effectiveness and framework conditions attractiveness.

The Key Performance Indicators used within this report are:

  • % R&D intensity (GERD as a % of GDP)
  • % Gross Domestic Expenditure on R&D (GERD – % Real Growth)
  • # Patents applications
  • # Researchers

For further examples of performance indicators, explore the R&D KPI examples section of the library of KPI examples available on smartkpis.com (smartKPIs.com, 2010).

References:

European Commission (2008), The Science, Technology and Competitiveness key figures report 2008/2009, available at http://ec.europa.eu/research/era/pdf/key-figures-report2008-2009_en.pdf (accessed 1 July 2010).

smartKPIs.com (2010), R&D KPI examples, available at http://www.smartkpis.com/kpi/functional-areas/knowledge-and-innovation/r-d/ (accessed 1 July 2010).

Industrial Production and Capacity Utilization Performance

Thursday, July 8th, 2010

The statistical release of the Federal Reserve, Industrial Production and Capacity Utilization: The 2010 Annual Revision, presents a thorough analysis of the industrial production (IP) index and the related performance measures of production capacity.

The industrial production index measures the real output of the manufacturing, mining, and electric and gas utilities industries. The base year considered within this revision is 2007, this lowering the level of the IP index for most periods. For monitoring this performance indicator, data was gathered on a monthly basis, from two main types  of sources:

  • output measured in physical units
  • data on inputs to the production process.

Total industrial capacity increased moderately from 2006 through 2009, but is expected to decline in 2010. The rates of change in both 2009 and 2010 have been revised up somewhat from earlier estimates, but revisions to earlier years were smaller (Federal Reserve, 2010).

Total industrial production, capacity and utilization (Federal Reserve, 2010)

The annual revision indicates an increased rate of capacity utilization for total industry by 0.7% point higher in the fourth quarter of 2007. At 71.1% , overall utilization in the fourth quarter of 2009 was 0.2% point below its previous estimate.

Capacity utilization (Federal Reserve, 2010)

The main conclusions of the annual revision are:

  • Manufacturing production contracted sharply in 2008 and 2009.
  • The production index for final products and nonindustrial supplies indicates moderate gains from 2005 through 2007 and then fell between 2008 and 2009.
  • In 2006 and 2007, the capacity utilization rate for total industry was situated above its long-run average of 80.6%, registered before between 1972 to 2009.

The Key Performance Indicators (KPIs) used within this review are:

For further examples of performance indicators, explore the Production KPI examples section of the library of KPI examples available on smartkpis.com (smartKPIs.com, 2010).

References:

Federal Reserve (2010), Industrial Production and Capacity Utilization: The 2010 Annual Revision, available at: http://www.federalreserve.gov/releases/g17/revisions/Current/g17rev.pdf (accessed 25 June 2010)

smartKPIs.com (2010), Production KPI examples, available at http://www.smartkpis.com/kpi/functional-areas/production-quality-management/ (accessed 25 June 2010)

Indicators Measuring the Equity of European Educational Systems

Friday, June 25th, 2010

Equity of the European Educational Systems. A Set of Indicators is a project that attempted to demonstrate the feasibility of building a set of indicators regarding the equity of education. The publication reports on a two-year period work on the issue of the equity of educational systems and covers three main parts:

  • The concepts of equality and equity, and the framework of indicators and its guiding principles.
  • A set of twenty-nine indicators on the equity of the educational systems, built in the context of this project and organized according to the framework.
  • An interpretation of the 29 indicators in an analytical interpretation of the equity indicators.

The European Group of Research on Equity of the Education Systems (GERESE) has been constituted in 2001 for developing a set of indicators for measuring the equity of education systems in the European Union Member States. The final report presents the 29 indicators developed as an informative tool to support decision-makers to redefine the educational politics.

The matrix of indicators

The matrix of indicators (GERESE, 2005)

The indicators presented are intended to provide input to the debate on justice in education, by offering some elements of response to the following questions:

  • What are the causes and the consequences of the individual educational inequalities ?
  • What is the importance of educational inequalities between girls and boys or between groups of different social, economic or national origins ? To what extent are they due to the societal context or rather due to the process of the educational system ?
  • How being below a minimum skill threshold can have important consequences for the individuals in and outside the school context ?

An approach to the fairness of European education systems

An approach to the fairness of European education systems (GERESE, 2005)

The table above shows that in some education systems, the inequalities in education are homogeneous, being pronounced (Germany and to a lesser extent Belgium) or small (Finland, Sweden, and to a less marked extent, Spain and Ireland) according to the three criteria at the same time:

  • inequalities between individuals (2)
  • inequalities between groups (3 and 4)
  • proportion below the threshold (5).

The two main overall conclusions stated in the report and emerged from this analysis:

  1. Education systems have clear differences in equity. The data does not confirm the premise according to which inequalities are low in countries where education has few external rewards.
  2. Some education systems that seem more (or less) fair than others on a large majority of the criteria, but for many the judgement of their fairness varies, sometimes considerably, depending on how we read the data. This study opted for a comparative and distinctive approach, pursuing a single principle of equity in the labyrinth of indicators, comparing equity, effectiveness and efficiency. (GERESE 2005)

References:

European Group of Research on Equity of the Education Systems (GERESE) 2005, Equity of the European Educational Systems. A Set of Indicators, available at http://www.okm.gov.hu/download.php?ctag=download&docID=296 (accessed 10 June 2010)

Working with initiatives in Performance Management – Check and Act vs. Study and Act

Saturday, June 12th, 2010

smartKPIs.com Performance Architect update 23/2010

In one of my previous updates I reviewed the history of the Deming cycle and its relevance for Performance Management. One of the most important benefits of managing performance in organizations is that it facilitates a structured process of improving the achieved results, which is the essence of performance.

Improvement doesn’t automatically derive from measurement. A robust process of analysis and decision making is required to facilitate suitable actions or initiatives. And to illustrate this process, Study as in the PDSA cycle is more meaningful than the Check as in the PDCA cycle.

The Performance Management case study presented the scenario of a non-profit organization interested in addressing childhood development issues. Some of the measures used were:

  • % Incidents of anemia
  • # Average scores on language and communication skills for toddlers
  • # Average scores for vocabulary tests

Using the traditional Plan – Do – Check – Act (PDCA) approach, the Check and Act phases would resume to gathering performance results data, reviewing it and taking actions to improve results. Generally the initiatives established as a result of this process would aim at doing more of the same thing. Improve efficiency or increase the volume of efforts.

However, a subtle change, that might appear superficial and technical to some, might mean more that it seems. Replacing Check with Study, shifts the emphasis from control and fixing the existing approach to learning and finding new ways to address the issue. For many years performance management has been associated with checking, inspecting, and controlling conformance. Performance Management for learning is a more balanced, mature approach to improvement.

In the case analyzed above, a review of the literature in the field and the latest research in the area of children health and development would reveal that the solution to the stagnation in achieving results might come from a surprising new direction. Under the title “Housing, Health, and Happiness” a new study published by the American Economic Journal: Economic Policy reveals that “replacing dirt floors with cement floors interrupts the transmission of parasitic infestations and should therefore reduce the incidence of both diarrhea and anemia. The reduction in anemia is expected to have positive effects on cognitive development” (Cattaneo et al, 2009).

The study, commissioned by the Mexican government, reveals the following results achieved during the experiment conducted in Mexico (UCBerkeleyNews, 2009):

  • 20.1% reduction in incidents of anemia
  • 30.2 percent higher score on the McArthur test (language and communication skills for toddlers ages 12 to 30 months)
  • 9% improvement in the scores obtained in the PPVT test (vocabulary tests for children ages 36 to 71 months)

When limiting themselves to checking the data and doing more of the same thing, organizations do not create the suitable conditions for leaning and integrating new ideas. Expanding the scope of inquiry from current approaches to researching new ones and investigating what happens in the field they operate in around the world, the improvement process benefits from using a more robust view on performance management, that emphasize the role of the study component.

In the case described above, reviewing recent research in the issue of health and early childhood development reveals a potential new approach that might just be the solution sought after. Setting up a new initiative that aims at replacing dirt floors with cement promises to a positive impact on the health and cognitive development of young children in the targeted community.

Study puts initiatives management in a new light.

Stay smart! Enjoy smartKPIs.com!

Aurel Brudan

Performance Architect,
www.smartKPIs.com


References

Cattaneo, Matias D., Sebastian Galiani, Paul J. Gertler, Sebastian Martinez, and Rocio Titiunik. 2009. “Housing, Health, and Happiness” American Economic Journal: Economic Policy, 1(1): 75–105. Note: a working paper version of the article is available at: http://www.stanford.edu/group/siepr/cgi-bin/siepr/?q=system/files/shared/pubs/papers/pdf/SCID367.pdf

UCBerkeleyNews, 2009, “Inexpensive flooring change improves child health in urban slums” available at: http://berkeley.edu/news/media/releases/2009/03/10_floors.shtml, accessed 05 June 2010.

Performance Management case study: Plan – Do – Check – Act (PDCA) in a non-profit organization

Saturday, June 5th, 2010

smartKPIs.com Performance Architect update 22/2010

Improving children’s quality of life in developing countries is today a priority of thousands of non-for-profit organizations. It is a difficult journey, influenced by many macro and microeconomic, political, social, cultural and religious factors. Many such efforts are structured in programs and projects. Monitoring not only their implementation, but also their impact is a requirement not only for tracking if they make a difference, but also for attracting new funding and other resources for future programs. Overall, many non-profit programs employ robust performance management systems to support the achievement of their purpose. Designing and using such systems is not as straightforward as it may seem.

Organisation

A non-profit organization.

Setting

The organization operates in both urban and rural regions, implementing programs and projects targeting specific health and early childhood development issues.

Mandate

Improve the health and education of children in at risk communities in developing countries.

Instruments

A performance management system is in place, linking objectives, performance indicators and initiatives.

Performance indicators

To monitor the achievement of this objective a set of performance measures can be established, targeting some of the specific issues to be addressed. For example:

% Incidents of anemia

# Average scores on language and communication skills for toddlers

# Average scores for vocabulary tests

Scenario

The organization is following the standard Deming cycle applied in a performance management context: Plan-Do-Check-Act (PDCA). Each year it formulates a plan of activities, specifying objectives, performance indicators and projects to be implemented. It monitors results every six months, when following an analysis of these results, review meetings take place. They generally result in a recalibration of initiatives and sometimes new ones are established. Several programs and projects are running at any time, aimed at raising awareness in the community of health and educational issues. Additional projects targeted specific issues such as improving the economic situation of the families in the community, better equipping the kindergarten / primary school and training the educators.

Some success was reflected by the reduction of the incidents of anemia and improvement in the scores.

However, after a while, the performance reports started to reflect a stabilization of results and no further improvements were achieved.

Questions

  • What changes to the existing portfolio of projects and programs should the organization make to improve results?
  • How should the organization alter the Performance Management System in use to facilitate better results?
  • What approach to stakeholder management should the organization take to facilitate sustainable changes in the community?

Stay smart! Enjoy smartKPIs.com!

Aurel Brudan

Performance Architect,
www.smartKPIs.com


Discuss the case in the smartKPIs.com Forum

This Performance Management Case Study is now available in the smartKPIs.com Forum, where members of the smartKPIs.com community are invited to contribute to the discussion (after logging in by using their registration details). New members are invited to join (for free) the smartKPIs.com community.

Performance Measurement case study – KPI examples in UK higher education

Friday, April 30th, 2010


What

The annual publication of “Performance indicators in higher education in the UK 2008/2009″ offers a brief presentation of the higher education institutions in the UK, in terms of performance and the indicators used to monitor it.

Who

The Higher Education Statistics Agency (HESA) – a private limited company which has formal agreements with government departments to provide the data which they require, and it is funded by subscription from all of the universities and higher education colleges throughout the United Kingdom.

Why

To provide reliable information on the nature and performance of the higher education sector in the UK and a consistent set of measures of this performance.

To contribute to a greater public accountability by the sector, as well as ensure that policy decisions can be made on the basis of consistent and reliable information.

How

Following the recommendations of the National Committee of Inquiry into Higher Education, the Government asked the funding councils to develop suitable indicators and benchmarks of performance in the higher education sector. The Performance Indicators Steering Group (PISG) was established, with membership drawn from government departments, the funding councils and representative bodies. Since 2002/03, HESA has published the Performance Indicators on behalf of Higher Education Funding Council for England (HEFCE) who published them previously.

Results

The set of Performance Indicators (PIs) was published in April 2010, by HESA is the twelfth in the series. The indicators are used for all publicly funded higher education institutions in the UK and they currently cover:

  • widening participation indicators
  • non-continuation rates (including projected outcomes)
  • module completion rates
  • research output

Performance indicator example

Source: HESA, 2010

Nationally, over 90% of 17 year-olds in full-time education attend schools or colleges in the state sector. 88.5% of young entrants to full-time first degree courses in 2008/09 had attended such schools, Chart 1 bellow shows the pattern.  (HESA, 2010)

Source: HESA, 2010

Reference

HESA 2010, “ Performance indicators in higher education in the UK 2008/2009”, available at http://www.hesa.ac.uk/index.php/content/view/1703/141/ (accessed 25 April 2010)

Monitoring ethnic representation in UK universities

Tuesday, April 27th, 2010

Recently launched report ‘Race into Higher Education’ presents a comprehensive review of current ethnic minorities in higher education. Based on detailed analysis of both the Office of National Statistics’ Labour Force Survey and the Higher Education Statistics Agency’s ‘HESA Student Record’, the report offers a support for  the Government policy makers, university boards and UK industry, to see how the current ethnically diverse generation will affect tomorrow’s  workforce.

According to the report, ethnic minority groups are better represented in UK universities than in the general population. Almost one in six (16.0%) of UK university students are from a Black Asian and Minority Ethnic (BAME) background. This is up from 8.3% in 1995-96, the year in which Business in the Community founded Race for Opportunity. This increase is virtually in line with the growth in the BAME population from 7.7% of 18 to 24 year olds in 1995-96 up to 14% in 2007-08. (Business in the Community, 2010)

Source: Business in the Community, 2010

British Indians continue to be the best represented ethnic minority group within UK universities in 2007-08 (3.3%) as they were in 1995-96 (2.1%). They were followed by Black or Black British Africans (3.2%) who have almost tripled their university presence in the last 12 years, and finally, mixed ethnicity students constituted 2.1% of the university population. (Business in the Community, 2010)

The “Race into Higher Education” report  presents the UK’s elite universities, regarding the students from ethnic minorities:Source: Business in the Community, 2010

Oxford and Cambridge recruit fewer ethnic minority students than the average for all other universities. However, the situation is different from one ethnic minority group to another.

The main performance indicator used within this report is % Ethnic minority students.

The report is available free of charge at: http://www.bitc.org.uk/workplace/diversity_and_inclusion/race/hesa_report.html

Additional resource:

smartKPIs.com (2010), “KPI examples for the Academic Education”, available at http://www.smartkpis.com/kpi/industries/education-training/academic-education/ (accessed 15 April 2010)

Reference:

Business in the Community (2010), “Race into Higher Education”, UK

Monitoring ethnic minorities employment after graduation

Monday, April 26th, 2010

Employment rate of  British students after graduation is another topic analyzed in the ‘Race into Higher Education’ report, which was mentioned also in a previous blog post.

The numbers presented in the report show that ethnic minorities are less likely to find employment after graduation, than their white counterparts: 56.3% of ethnic minority students who graduated in 2007-08 found work within a year, compared with 66% of white students. Both White and ethnic minority females are ahead of males in employment success by around four percentage points each. (Business in the Community, 2010)

Source: Business in the Community, 2010

According to Sandra Kerr, national campaign director at Race for Opportunity, “…only if more school leavers from ethnic minority backgrounds study at Oxford, Cambridge and other high achieving universities are we likely to see British ethnic minorities progress into senior management and key leadership positions.“(Business in the Community, 2010)

In order to support the increase in Higher Education for ethnic minorities and their chances to employment after graduation, the “Race into Higher Education” report makes the following recommendations:

  • British universities should monitor, report and improve on ethnic minority representation.
  • Increase awareness amongst ethnic minority school pupils of the importance of university choice.

The main performance indicators to support and monitor these initiatives are:

The report  is available free of charge at: http://www.bitc.org.uk/workplace/diversity_and_inclusion/race/hesa_report.html

Additional resource:

smartKPIs.com (2010), “KPI examples for the Academic Education”, available at http://www.smartkpis.com/kpi/industries/education-training/academic-education/ (accessed 15 April 2010)

Reference:

Business in the Community (2010), “Race into Higher Education”, UK

Performance Management matters – PricewaterhouseCoopers reports on Performance Management in the global economy

Friday, April 23rd, 2010

PricewaterhouseCoopers (PwC) conducted during 2008-2009 an exhaustive research regarding the importance of performance management in the context of a globalized economy. The research was lead by PwC Canada, in partnership with several academic and research institutions from Canada, Switzerland and Australia.

The methodology consisted of two stages: an online survey to more than 400 senior managers in both private and public sectors, followed by 12 in-depth interviews aiming to investigate several issues in a more complex perspective.

The purpose of the research was to provide answers to several core questions (PwC, 2009):

  • Do effective Performance Management practices contribute to driving superior performance?
  • Are these contributions the same in downturns as in good times?
  • Is performance management standardized or does it have to be customized for particular needs of organizations of different sizes and in different industries?

Source: PwC, 2009

Some of the most representative findings, were (PwC, 2009):

  • The identification of seven best practices that drive high performance and superior results, among which:
  1. having a holistic, broad approach (high performers focus on broad issues, such as brand loyalty, employee satisfaction, service quality, aspects that can create competitive advantage);
  2. creating linkages, alignment and integration of the measures to the key business drivers and cascading accountability at all organizational levels;
  3. adopting high value planning practices, such as vision, mission and value statements and environmental and social responsibility plans;
  4. developing advanced technology such as dashboards and business intelligence tools to complete the spreadsheets that are still predominant across respondents.
  • Performance management practices are equally essential in both recession and growth periods. As one argument, the interviews revealed that organizations with strong performance management programs can better manage costs strategically and, if the case, make difficult decisions in ways that minimize impact on the long-term.
  • Performance management in the public sector is not as different from the private sector as it might be believed. It was concluded that high performing public organizations are the ones that employ practices also identified in high performance private entities, such as linkage and alignment to strategy, use of advanced technology and use of value planning practices in the shape of vision, mission or values.

Canada was given a particular attention, being under the loop for comparison to the rest of the world. The finding was that Canadian organizations employ less robust performance management practices, but are better at overcoming potential barriers.

For complete overview of the research, please follow: http://www.pwc.com/ca/en/finance/performance-management/global-performance-management-research-study.jhtml

You can register for a full version of the report or directly download an executive summary.

Reference:

PricewaterhouseCoopers (2009), Performance management matters. Sustaining superior results in a global economy, CA.

The research was lead by PwC Canada, in partnership with several academic and research institutions from Canada, Switzerland and Australia. The methodology consisted of two stages: an online survey to more than 400 senior managers in both private and public sectors, followed by 12 in-depth interviews aiming to investigate several issues in a more complex perspective.