MEASURING SUCCESS FOR NONPROFIT ORGANIZATIONS

Leon Masiewicki, Director, Integrative Strategies 3D, LLC and Herb Rubenstein, President and Founder, Herb Rubenstein Consulting

Introduction

Non-profit organizations routinely publicize their accomplishments. Increased reach, meals served, number of clients, grant received, and other measures of activity define their good works. The United Way in many cities has required for years that the organizations that it funds provide numerical targets that they must achieve to continue being funded. Similarly, established organizations like National Arts Strategies, the pioneer in venture philanthropy in the 1970's, when it was known as National Arts Stabilization, and newer organizations like Venture Philanthropy Partners, are all requiring a new level of accountability and entrepreneurism from the nonprofits they fund. Quarterly targets based upon agreed upon measures of success are becoming the norm. Still, the nonprofit industry is just starting to enter the "Age of Accountability."

Nonprofits and Success Factors

Businesspeople have long been saying "what gets measured gets managed." Human Capital Capability, Inc. has built a business around "measuring what matters" in the business, government, education, and arts sectors. Hundreds of government agencies today do not have in place numerical measures by which to measure their success, even though there is a law that requires that they do. Thousands of nonprofits do not keep even the barest statistics of how many people they serve, what impact their service has on their target population, nor collect sufficient economic data to measure efficiency in their organizations. And, as Ken Boulding, the great economist, used to teach, "If there is money in it and it is not being done or done very well, then it must be very hard to do."

We agree that identifying success factors by nonprofits can be extraordinarily difficult. But it can and must be done by every nonprofit that hopes to survive in the 21st century. This article is based on our knowledge of many nonprofits. We conclude this article by proposing an analytical method for measuring success of nonprofits.

The Case of Teach for America

In the 2003 special issue of McKinsey Quarterly devoted to the issues of organization, Jerry Hauser, the Chief Operating Officer of Teach for America, traces the organization's path from an idea in 1989 to a $30 million operating budget with 3,400 teachers in public schools serving disadvantaged areas in 2003. Teach for America's stated goal is to grow to a $36 million organization with a 4,000-strong teacher corps.

As a society, we agree that public schools, particularly those serving low-income areas, are woefully underfunded and that the better trained and equipped teachers is an important goal in U.S. public education today. Therefore, we agree that, in principle, Teach for America's effort to put the brightest graduates of the best universities in the classrooms in impoverished areas is a worthwhile approach to help the students and schools in these areas.

However, how should we ideally measure Teach for America's success? First, of course, we should look at all of the costs of the Teach for America program. Publicly available data show that TfA is spending $18,000 in recruiting, training, and administrative costs for every teacher it places. In addition, the teachers' placed by TfA have their salaries, equal to the salaries of other new teachers, paid by the school districts and students loans are deferred. In addition TfA receives up to $9,450 over two years from AmeriCorps, which is funded by the Federal Government, to repay their student loans or pay for future education.

To evaluate the success of Teach for America or any other nonprofit, it is essential to understand its costs. For every "unit" of benefit provided, the organization must know exactly how much of its resources it has used and will use to produce this benefit. This cost, like all "cost curves" studied in microeconomics will show that as the number of units of benefits (service, goods) changes the total cost, and the average cost per unit will change as well.

While there has not been a comprehensive evaluation of all of TfA, the study of its Houston program reports an attempt to quantify the impact on the students who have TfA participants as their teachers. The executive summary of the study says:

"The differences between the average Teach for America teacher and the average non-Teach for America teacher, while always positive, are generally not statistically significant."

"We created ten statistical models to look at the difference between Teach for America and other teachers' contributions to student performance. In four models, Teach for America teachers, on average, produced significantly better student outcomes than non-Teach for America teachers. In two models, the average Teach for America contribution to student performance was positive but small. In four cases, the average contribution was positive and large, but the distribution around the average was not sufficiently different from the comparison group to be statistically significant."

Only after learning the true cost of a nonprofit's activities and the outcomes of spending these funds can one answer two important questions. First, what is the cost of success? Second, could this amount be better spent? This second question requires understanding not only of all of the alternative ways of achieving the desired outcomes, but also of the cost of achieving these results using different approaches.

Once we understand the costs and benefits of organizations like Teach for America and other programs to improve the educational results of impoverished youth, we can begin to allocate resources rationally, where they can be expected to have the greatest return or the biggest bang for the buck. While this sounds cold and calculating, this is the wave of the future; our society can no longer afford to put resources into nonprofits that produce less value than other organizations. Therefore, we think that nonprofit organizations like Teach for America would benefit from measuring their impact on the society against the resources they use in a very careful manner.

Is Evaluating Success Possible in the Nonprofit World?

If you run a nonprofit, especially a small one, you might be thinking, we cannot do this. We can not measure cost per unit of service. We can not measure the benefits of what we provide. And we can not find out what alternative ways our goods and services can be provided and measure their per unit costs either.

In the next section of this article, we show you how this can be done. We propose that every non-profit adopt the tool from the private sector called "effectiveness and efficiency analysis." This is a tried and true business tool with great applicability to the world of nonprofit organizations.

Effectiveness Analysis

Most nonprofits understand the objectives of their organization. They have budgets. They keep some statistics. They can count attendees, meals, track test scores, recidivism rates, salaries and career paths after training programs are delivered, beds filled in homeless shelters, calls made and prescriptions filled by the elderly. Even the IRS is now evaluating its outreach (the IRS calls them "sales programs") programs by how many tax returns it helps elderly people fill out every year. Few nonprofits, however, can either describe the economic impacts of their programs or measure how efficiently (cost per unit of service) they are delivering the services to fulfill their mission.

The Best Friends Foundation, which provides character education to adolescent girls, in addition to other programs, knows that society benefits from its work. Every year that its "clients" stay in school, delay pregnancy, refrain from drug use, increase their self-esteem, and serve as role models, mentors, and advisors to their fellow students who are not in the program, contributes to avoiding future social costs and enhances their clients ability to become productive citizens.

By tracking the per unit cost at different levels of "output" nonprofits can begin to develop carefully tailored strategies to reduce costs and improve outcomes. And until we have this information on all of the alternative programs aimed at achieving the same results, how can we allocate society's scarce resources efficiently to insure that the dollars invested are producing the most effective outcomes possible? Only having and understanding this information will tell us if the effort of any nonprofit is worth the cost. With charitable dollars and government dollars shrinking in the United States, all nonprofits must be able to make the case that its programs and services are worth the cost.

We assert that, for the most part, it is not very difficult to calculate the costs and benefits of most nonprofits' programs. Benefit calculations must always include "social cost reduction," if the objectives of a nonprofit include, for example, reduction of the incidence of welfare, recidivism, drug use, etc. By deploying the Effectiveness and Efficiency Analytical Tool described below, data can be collected and analyzed, and conclusions reported at a reasonable cost. The system outlined in this paper is a general guide to the development of an information system to measure, track, and analyze the effectiveness of nonprofit organizations with social missions.

Efficiency Analysis

Efficiency is as important for nonprofits as it is for businesses in this world where nonprofits and for- profits compete every day. Even though most nonprofits know their budgets and can account for every penny, they often do not have a system to let them know whether they are spending their money in the most efficient manner. Some efficiency issues are operational, such as organization of the ER at a hospital, or janitorial services at an educational institution. Some are strategic–they concern the focus of the organization's activities. Every institution is better equipped to handle some tasks than others and every institution can benefit from a better understanding of the activities it should undertake and the most efficient scale at which it should operate. Lack of such understanding usually results in mission creep, inefficient production of benefits due to lack of economics of scale, all of which can ultimately lead to waste or financial disaster.

Just as a rural health clinic should not attempt heart transplants, until telemedicine progresses greatly, the Texas Heart Institute should not set broken bones. The former will too often result in an undesirable outcome–loss of the patient and waste of the donor organ, while the latter will be prohibitively expensive, even if most heart surgeons can set most broken bones.

Nonprofits must examine the efficiencies of their activities from the perspective of their mission. This examination consists of answering two questions: Can we lower the cost per unit of our activities? Can others perform some of our activities more economically?

We can answer the first question by analyzing operations. In Teach for America's example above, for an efficiency analysis, we need to know exactly what is the breakdown of the $18,000 per teacher cost. To what extent is it recruiting? TfA interviews five to eight applicants for each placement. To what extent is it training? Training is necessary, but is TfA spending a reasonable amount? How much goes to the support of the placed teachers? Would this per unit cost go down dramatically if Teach for America placed 8,000 or 16,000 teachers rather than less than 4,000 teachers? Is there a demand for 8,000 or 16,000 Teach for America teachers?

It must be the goal of every nonprofit to become more efficient, to reduce costs per unit of service, per participant, and, most important, per unit of outcome, on an ongoing basis. This is called increasing efficiency or increasing productivity and is what drives private sector success. In the future, we predict, it will drive nonprofit success. The tools we describe in this paper are designed specifically to assist a nonprofit in being more efficient and more productive. Since all nonprofits use other people's money and receive tax advantages for doing so, they have a duty to increase their efficiency and productivity every year, if not every quarter.

Answering the second question, whether a nonprofit should be providing a given service or function, involves what businesses call competitive analysis. If clinics can perform certain procedures more cheaply, should a teaching hospital try to reduce providing these services to a minimum? Competitive analysis consists of finding out which other organizations, public, private, or even for-profit, perform similar tasks better, cheaper, and faster. If there are none, the institution providing these services in the most efficient manner should do well. If there are others who can provide similar quality service at a lower price or a higher quality service at the same price as the lower quality service provided by the nonprofit the choices are only to improve or cease activities, just like in business. Therefore, a competitive analysis is an essential component of the efficiency analysis because the competitors, or alternative providers, serve as a societal benchmark for measuring whether nonprofit institutions are efficient, compared to other organizations.

Factoring in Avoided Social Cost

Efficiency and effectiveness analysis in the non-profit world should take into account the full range of benefits that the nonprofit's programs and services provide. For example, we know that in fiscal year 2000, twenty billion dollars went to what we generally refer to as "welfare programs." 2.3 million family units and 6 million individuals benefited from AFDC and similar programs. Therefore, the average cost per family was approximately $8,300. Since some participants are on welfare for less than a full year, the cost of a full year of AFDC payments and related welfare benefits to a family in the program would be higher.

Therefore, if a nonprofit can show that it moves people from welfare to work, or is able to move some portion of its target population off a government program, then we all benefit by saving the social cost avoided by reducing the need for the government to provide a service. However, if it costs $100,000 or more per participant to achieve a welfare-to-work transition, then this analysis must call into question whether society is better off spending this much money to achieve this result. Of course, if a program keeps someone off welfare for 20 consecutive years, the cost may be worth it. In short, the funds spent by society, in the form of donations and tax breaks to run nonprofit programs, should be compared to the funds saved by society, in the form of reduced government costs, for example.

Measuring Return on Social Investment

To compare competing nonprofits' effectiveness and efficiency, we need to measure how much social good they generate by their economic activity. Businesses are using several such measures for gauging their success, and most of them have been modified to the nonprofit accounting world to measure various aspects of performance.

For example, a business's net profit, perhaps the most closely watched measure of performance, corresponds to "addition to endowment" for a nonprofit. However, addition to endowment is a good measure of how a nonprofit husbands its resources, but not of how effectively it spends funds to carry out its mission.

We would like to propose here a quantification of gain that a dollar donated to a nonprofit organization yields to society. We call it the Social Return Multiplier. The basic formula is simple:

SRM = VBS / Budget

Where SRM is the Social Return Multiplier and VBS is the Value of the Benefits to Society.

To calculate SRM for a fiscal year, for example, we need to quantify VBS, including all avoided social costs attributable to the nonprofit's activities during the year and divide it by the organization's budget, including administrative and fundraising expenses.

High-performing organizations will return more to society than they spend, therefore, their SRM will be larger than one. Nonprofits with SRM lower than one from year to year, like businesses unable to earn their cost of capital, are in trouble.

We can also calculate SRM over multiple periods. Some nonprofits, like those just starting their activities or changing their strategies, may experience a low SRM for some time. When they reach the cruising altitude, however, they ought to return to society more than they took earlier. Thus, even though their first-year multiplier might be a low 0.2, they must have a plan to raise their long-term RSM to more than one, which may mean that, for example, in the 10th year of operations, their RSM must be 2.5.Example of Effectiveness and Efficiency Analysis

The Joy of Sports Foundation employs activity-based costing. It has numerous programs and an accounting system to track the cost of each one. It keeps statistics on how many units of physical education it provides and how many students receive its other services, including sports and physical play to help children learn life skills and to develop their body, mind, and spirit. JSF was founded in Washington, DC, in 1989. Since then, it has served nearly 25,000 children. Today, its most important activity is the Healthy Kids Program, which is targeting the youth obesity problem in the United States. JSF is conducting a full-scale evaluation of the Healthy Kids Program from the perspective of the impact that this program has had during the first year of this three-year endeavor. Using both an efficiency analysis (cost per unit) and effectiveness model (impact per unit of service) JSF will be able to monitor and improve its programming. Most importantly, the efficiency and effectiveness analysis will empower JSF to scale its programs nationwide, appeal more successfully to prospective donors for larger donations and to set high standards for both effectiveness and efficiency within the organization. Then, as funders make choices in the future, JSF's competitors will also have to adopt this analytical approach in order to compete successfully with JSF for funds for youth development programs.

The Business Case for Efficiency and Effectiveness Analysis

We understand that it costs money, staff resources, time, and expertise to undertake effectiveness and efficiency analyses. How do we know it is worth it to organizations to deploy an effectiveness and efficiency analysis?

First, while the initial creation of the information system, data collection, analysis, and reporting systems, may be costly, we believe the annual operating costs of these systems can be kept to a minimum cost through the use of self-reporting, just IRS does in collecting data on incomes of individuals. Self-reporting systems can be developed to be reliable and inexpensive and can meet the needs of many nonprofits.

Second, using sampling techniques, nonprofits can estimate their benefit production across the entire population of those they serve. Sampling only a fraction of their recipients or program participants is cheap and, if done correctly, is as reliable as a full census.

Third, nonprofits will have to incur the cost of upgrading to activity-based costing in any event, just to keep up with their competition in the future. These costs are to be weighed against the tangible benefits that nonprofits will receive from measuring their success honestly and accurately. We believe the ability to measure the effectiveness and efficiency will pay returns many times over the cost.

We cannot overestimate how important measuring the effectiveness and efficiency of your organization is to promoting successful fundraising. These facts and figures "pre-sell" your organization to potential donors and give a strong indication to donors of the future value of their contribution. With lower tax rates and a continuing low rate of consumer confidence and high level of economic uncertainty, donors receive fewer tax breaks and tangible benefits from donating dollars to nonprofits, thus making the real cost of their donations increase. Today, with scandals at United Way, the Red Cross and other long-standing icons of stability and integrity in the nonprofit world donors need new, fact-based assurances that their dollars are making and will make a difference.

In addition, effectiveness and efficiency analysis will focus a nonprofit organization's management efforts on the right kinds of improvements, on delivering their services to the recipients and participants where they can make the greatest (most efficient) contribution. Such analysis can provide guidance to the nonprofit organization regarding what it can perform at the "best of class" level and what is can perform only on a mediocre level. It will also provide guidance on the critical issue of scale so that nonprofits, like for profits will be able to take advantage of economies of scale where they can be discovered.

Third, the efficiency and effectiveness analysis will ultimately affect the strategy of the organization. For example, it can help determine the need for or limits to expansion–the geographic reach and the types of clients the institution will accept and the selection of the client population. In sum, the society as a whole, nonprofit organizations and client populations will be better served with the efficiency and effectiveness information that we have identified in this white paper. Of course, such a system must be specific to each institution, its mission and its clientele. Once implemented, it will have enormous value to nonprofit organizations.

Conclusion

Consultants and management experts have been helping nonprofit institutions improve operations and fine-tune approaches to fund-raising for almost a century. Nonprofits will benefit greatly by deploying in a rigorous manner the efficiency and effectiveness analytical tool that we have outlined. This tool has been tested by the authors on the strategy and operations work at scores of companies in dozens of industries, from heavy manufacturing to software and from banking to education. The methodology presented here has the potential to make a real impact on nonprofit institutions and enhance their funding sources. It was written in Breakthough, Inc.: High Growth Strategies for Entrepreneurial Organizations (Rubenstein and Grundy, Financial Times/Prentice Hall, 1999) that businesses, nonprofits, and educational institutions all have to live by the same laws of commerce. The efficiency and effectiveness analysis will help them abide by these laws. The organizations just starting to embark, or who wish to embark, on this journey, will need a strong, multi-year commitment to this effort and they will need professional help. Those already beginning to use this analytical tool will find that this approach will constantly need tune-ups, as programs change their focus, as the institution's structure is reorganized, and as the nature of the competition changes.

We hope that this paper will help nonprofits better address the important issues surrounding efficiency and effectiveness.

Biographical Information

Leon Masiewicki is Director of Integrative Strategies 3D, LLC, a management consulting firm specializing in strategy development and strategic planning. His clients include government contractors, software developers, and medical technology companies. He holds a PhD in Mathematics from Columbia University and an MBA from Harvard. He has published articles on strategy and innovation. His email address is leonm@integrative-strategies.com and he can be reached at (301) 816-0130.

Herb Rubenstein is an attorney and the CEO of Herb Rubenstein Consulting, a leadership and management consulting firm. He is co-author of Breakthrough, Inc. – High Growth Strategies for Entrepreneurial Organizations (Prentice Hall/Financial Times, 1999). His email address is herb@herbrubenstein.com and he can be reached at (301) 718-4200 in Bethesda, Maryland or (202) 236-7626 in Washington, D.C.

back