Five Problems with How We Measure Social Change

For this month’s Nonprofit Millenial Bloggers Alliance post, we are tackling defining and measuring social impact.

I think we talk about social change so much that we underestimate what it takes to bring it to fruition.  Social change is inherently a societal concern with multiple layers and intersections, which makes defining it and measuring it difficult.  Can you really address health care without examining income, race, and gender?  Can you attribute a safer neighborhood to the efforts of a nonprofit or to changes in city legislature or both?

Given the complications in simply defining and examining social change, we need to stop assuming that the tools used to measure other objects—like profits—are adequate and effective.  In fact, many of the efforts I see for measuring our progress are short sighted, essentially boiling down to “Are we using donors money well?” Implied in this approach is that since donors are supporting our mission, if we please the donors, then we are fulfilling our mission.

While this approach is important for evaluating performance, it doesn’t tell us much about our progress towards social change.

• For one, it doesn’t put people in need at the center of the measurement: I understand the push to be objective by ignoring the voices of the people we serve and to focus on things we can easily count (How many get counseling?  How many complete applications?)  Yet how people are experiencing change, problems in our approach, and potential new challenges cannot be easily documented quantitatively.

• It assumes there cant be multiple interpretation of numbers: When I read data on how many inner city youth go off to college, I see that data as incomplete.  Why?  Because acceptance isn’t the issue– graduation is.  What’s the point of getting accepted if you don’t finish?  Yet for some acceptance is great, while others want to know the quality of schools, and others, like myself, want more long term data.  In other words, numbers are interpreted differently so no set of numbers is complete.

• It leaves little room for collaboration: If one sector had the answer to our most pressing problems, then guess what?  Only one sector would exist.  And if social change merely required someone with good intentions and loads of time, our problems would’ve been solved decades ago.  Essentially, how are we including and examining the various other forces that impact the well being of people we serve?

• It doesn’t include a long-term approach: Social change takes a while.  Providing immediate services to people is important but tracking changes in situations and circumstances won’t be captured in a yearly report.

• It doesn’t allow for critical thinking: Without focusing on processes how do we know if our definition of social change is appropriate?  How do we know if what we are measuring adequately captures progress?  Once we focus on gathering numbers without engaging critically with our own processes, we lock ourselves into methods that may be outdated or problematic.

This is not to say that change should not be measured.  It should and absolutely needs to.  The concern is that the current models don’t capture the entire picture and we need to develop a more holistic approach.  

Check out how some of the other nonprofit millenial bloggers view this issue:
Colleen, Does Writing a Check Equal Social Change?
Elizabeth, What is Social Impact?
Elisa, Measuring Social Impact
Tracey, The Meaning of Social Impact
James, Measuring Social Impact