The Observer Effect

12.09.2023

“Efficiency is doing things right; effectiveness is doing the right things” – Peter Drucker

Drucker’s famous saying is usually intended to remind us that it’s not enough to do things right, you also have to do the right things. However, in social services, we often think that we are doing the right things, but many times we don’t measure ourselves in a way that enables us to know whether we have indeed achieved our objectives.

Let’s take, for example, interventions designed to help families lift themselves out of poverty. Before the program starts and during the program, we measure its short-term effects.  But in most cases the goal is for the outcomes of the program to be retained for several years after the intervention. The effectiveness of most intervention programs is only measured during or immediately afterwards, while the long-term impact is rarely examined.  Short term measurement is important to let us know if we are going in the right direction, but it is not enough.  Without long term measurement, we cannot know if the intervention really succeeded.  In the case of relieving poverty, the family’s income 3 years after the end of the program is a far more significant measure of success than immediately after the intervention, but as stated, we do not usually measure this.

Sometimes, measurement is done using data and metrics that are readily accessible to the organization.  We tend to choose existing metrics and available data when we define how the organization’s success is to be measured.  This process is efficient but not “the right thing” (now we can read Drucker’s quote in his original intention).  It results in measuring products or outcomes that are not necessarily the main goals of the program.  A correct definition of metrics is the result of a process in which we first define how we perceive the success of the program (even though it may not be measurable). Once we have defined the ‘winning picture’ of the intervention, we must think which metrics (including those that are not available) will enable us to decide whether it has been achieved.  Sometimes we may discover that we have to develop these metrics, collect the data, and invest resources in the whole process.  But there is no other choice, if we really want to learn about the effectiveness of our activity – we have to invest the necessary resources.

Measuring impact is essential but may also lead to negative results.  I am not talking about failure to achieve a program’s objectives, but of the damage that may be caused by the measurement process itself.  One of the processes that measured quality in the health system produced an almost perfect result.  The program succeeded beyond all expectations.  However, after a qualitative examination of the process, it emerged that in order to comply with the defined metric, not only were other metrics neglected, but there was even some incorrect clinical activity that led to a decline in other aspects of performance that were not measured, and adversely affected the health of part of the population.

As the Jewish New Year (Rosh Hashana) approaches, we must stop for a moment and assess our measurement processes (in our professional lives and in our personal lives) and ask ourselves the following questions:

  1. Do our measurements cover the long-term, sustainable success of the program or just the outcomes during and immediately after the intervention?
  2. Have we defined what we want to achieve? Do our metrics measure these desired results?
  3. Are we aware of the negative outcomes of measurement? Which areas are we neglecting or sacrificing in order to achieve the desired results?

I invite myself and you to take the few months between Rosh Hashana and the end of 2023, as we prepare to set our targets in the work plan for 2024, to assess our measurement processes and think about how we can achieve better measurement and assessment of our impact.

Recent Posts

325 Days of War: Data Compilation of the Northern and Southern Communities

25.08.2024

These days mark 11 months since the outbreak of the war. Since October 7th, 1,635 Israelis have been murdered and

Read more >
  • Taub Center Staff

Homicides in Arab Society Continue to Climb

23.08.2024

Updated data on homicide rates in the Arab society up to May 2024 continue to paint a worrisome picture. In

Read more >
  • Taub Center Staff

Taub Center’s Environment and Health Initiative: Environmental and Resource Resilience at War

10.04.2024

Taub Center's Environment and Health Initiative researchers; Maya Sadeh, Prof. Nadav Davidovitch, and Prof. Maya Negev, delve into the environmental