The Call to Blue Marble Evaluation

Guest Blog by Glenn G. Page, Principal, SustainaMetrix

Over the weekend, I received a “wake-up” phone call with an opening line no parent wants to hear: “Dad I’m in a police car…”  With these words, my world changed! I wanted the facts - to be sure, but my raw instinct was to act, whatever it may be. I soon learned all was fine, a wheel had come off his car (long story) and a police detective just happened to be driving by and stopped to offer immediate assistance. But something about those first few words initiated a response mechanism that changed everything.  In an instant, I was called to action! I would have done anything to help, support, and provide whatever was needed, regardless. I was ready to go!  As I think about the crisis of global biodiversity, wealth inequality, climate disruptions, growth at all costs and rapidly increasing social, ecological and economic debt being transferred to future generations, I wonder why don’t we have the same collective response.

A form of community architecture has recently sprung up. The SDG Transformation Forum is a platform designed for action – not just thinking about the issues but doing something about them. It comes at a time when we know, without question, that our species is shaping the future of life support systems we depend upon. The SDG’s are probably the best expression of global governance we have, now codified as collective response to the mess we have been making. The Forum both appreciates and celebrates the SGD’s but also identifies that their pursuit needs a new mindset, different from that which created the current situation. In the name of goal attainment, the devil is in the details of how we do it. Attainment of all the laudable social and economic goals (SD Goals 1-11) without transforming the short-term focus of economic growth as the ultimate objective will prevent attainment of ecosystem goals of climate and life on land and below the water (SD Goals 13-15). "We are called to be architects of the future, not its victims. [The challenge is] to make the world work for 100% of humanity in the shortest possible time, with spontaneous cooperation and without ecological damage or disadvantage of anyone." –R. Buckminster Fuller

Following this edict from Bucky Fuller won’t be easy. For starters, we need to make sure our measurement of progress does not fall into the trap of quantitative analysis left only to the statisticians and methodologists. We envision a more inclusive and intuitive approach rooted in integration and synthesis across the goals, high degree of participation with holistic thinking that respects the non-linear, interrelated and complex realities that exist across scales.  This is messy work! Pushing us further, we need to ground thinking in values that undergird the SDG’s in the first place: to be architects of a transformative future! As Robert Picciotto wrote in a 2015 article on Development Evaluation in Transition: “in an uncertain world, where uncertainty prevails and change is the only constant, ethical values, more than pre-determined outcomes drives decision-making.”

Thus, we believe that evaluation and assessment itself must be transformed. That is an overarching philosophy that is guiding one of the working groups within the SDG Transformation Forum. The Assessment and Evaluation working group is gaining some real interest from folks who have developed stellar careers in the field of systems thinking, complexity theory and evaluation. Together we are recognizing major challenges if linear logic and the illusion of control dominates assessment and evaluation of SDG’s. Pathways to both sustainable growth and poverty reduction is not linear and context free, it depends on many factors. Notably surprises, unintended consequences, butterfly effects and disruptions all which requires paying attention to patters through feedback and adaptive learning.  This opens a window for scaling up the relatively recent growth of developmental evaluation and principles-focused evaluation to address issues of global systems change. As Picciotto notes in the article mentioned above, “this will require a more adventurous approach… well beyond the programme evaluation approach that has dominated it.” Zenda Ofir agrees! Zenda, a gifted writer, independent evaluator and former president of the African Evaluation Association and a member of this working group believes we need to evolve the profession, and in countries still known as the “Global South” may be fertile ground to jump start the process. There is a growing awareness of the need to think differently about sustainable development and its evaluation; the challenges are intensive and visible, and in need of drastic and urgent action and innovative solutions; and patterns of thinking and doing are les constrained. Think leap-frogging to cellphones; we need to seek the evaluation equivalent.

What started with a face-to-face meeting in early October in Scotland, our SDG Transformations working group has generated a DRAFT manifesto and two webinars and the early stages of an action agenda. What began as a small group of five people, has now engaged over a hundred people who are interested in what is needed to transform the nature of assessment and measurement to better respond to global systems change. And we believe this is only the beginning!

This work comes at a time when the field of evaluation is rapidly evolving. One of the working group members is Michael Quinn Patton who has built a distinguished career as a forefather in the field of evaluation pushing its envelope towards usefulness. Just now, if you were in a remote corner of the winter woods of Minnesota and you listen closely, you may just hear his keyboard chattering away as he writes the “Blue Marble” perspective. This book is not like the others as it is as much for global leaders as it is for evaluators. It’s as much about the design of our response to large-scale systems change as it is about measurement strategies. He is offering a systematic approach, rooted in evaluative thinking, principles focused evaluation and developmental evaluation. It’s about adaptive learning and collective response that requires both individual chops and collective competencies.

The name “Blue Marble” comes from a famous photo that was shot in 1972 during a formative time in our relationship with our home planet. The Cold War was red hot and the race to outer space was a critical symbol of success. During one of the Apollo missions, an unintended consequence occurred! A photo was taken of the earth itself, described as blue and white marble floating in the darkness of space that changed known as the “Blue Marble” shot that enabled all humans on earth to see our collective home as an integrated whole. It’s worth noting that several astronauts who saw the earth in this way have described it as a profound spiritual experience that forever changed their relationship with the earth (see The Home Planet published in 1988 by Addison-Wesley in New York and edited by Kevin Kelley).

We believe that the Blue Marble perspective is something we need for marking progress with SDG’s. Our working group is now at the earliest stages of developing an action strategy to define what may be needed to transform the process of assessment and measurement for SDGs. We believe that if the “evaluation system” is better understood, we can more effectively engage with it to understand major gaps, discrepancies, inefficiencies, hindrances and enablers to effectiveness. This is ambitious! One estimate puts the number of evaluators world-wide at 32,000. This number is dwarfed by the total number of accountants, auditors, management consultants and statisticians. The opportunity for high quality Blue Marble-type evaluation as a leadership function is upon us. This is consistent with the clarion call of Global Evaluation Agenda, drawn up in 2015 by EvalPartners and International Organization for Collaboration in Evaluation (IOCE) accepted in Bangkok, Thailand.  It will require transformative education of evaluation professionals, guiding principles rooted in competencies for the individual and for collective action. It will require serious rethinking of the role, promise and limitations of evaluation as practice in service of the world.  

Ultimately, if the global evaluation system is to serve the needs of people, ecosystems and the planet, we need to understand and help enable its evolution. A focus on SGD’s is a major starting point. We are all receiving “wake-up” calls every day.  For me, the real wake-up is not from a call from my son’s cell phone but from a world with a perfect storm of challenges and the slow the pace of collaborating across borders, across sectors, across disciplines toward shared global goals! The question remains: how are we responding?  If you are interested in transforming how we mark progress with SDG’s - come join us. 


Reflections from IPDET 2016

Michael and I recently spent two days with twenty international development practitioners and evaluators at IPDET in Ottawa. They were pioneers in our first two-day workshop on evaluating global systems change, the result of our six-month grant from Faster Forward Fund in support of Blue Marble Evaluators. We started our journey with this video, documenting human migration over time, followed by an icebreaker to get to know one another and share our own migration stories. Participants located themselves in the room based on where they were born and had to introduce themselves without saying the name of the country. Here are some photos and a video of the exercise:


 Blue Marble Evaluators describing the place where they were born without using country names. Camera is standing in North America, looking SE towards Australia. 

Blue Marble Evaluators describing the place where they were born without using country names. Camera is standing in North America, looking SE towards Australia. 

 Blue Marble Evaluators where they current live or work.

Blue Marble Evaluators where they current live or work.

With this exercise, we were challenging the primacy of national borders, which we believe often get in the way of systems change. In the words of Albert Einstein, "you can never solve a problem on the level on which it was created." Next we watched a video of the changing borders of Europe to drive home the idea that national borders are artificial and the result of war, colonialism. We also watched the DNA Journey, a powerful video by Momondo, which shows what our DNA reveals about where we come from. All of this was intended to re-frame our way of thinking from a global perspective. We ended this introduction with the Blue Marble Shot, the first complete photograph of the Earth taken from space in 1972. Each participant also receive a blue marble as a reminder to take the blue marble perspective into their work.

After this introduction and re-framing, Michael walked through the evolution of evaluation into an international field, culminating with 2015 as the International Year of Evaluation. While the focus on building evaluation at the country-level is important, we believe that the next frontier for evaluation is for exaluators to apply a global perspective to their work and build our capacity to evaluation global systems change efforts. At its core, evaluating global systems change involves looking across both sector and national boundaries. 

Over the course of two days, we explored what this might look like through case examples and exercises mapping global issues, diving into systems thinking and complexity concepts, and working with a real-world example from the Global Centre for Pluralism.  Check out the gallery below for some photos of this work.

There are many challenges to evaluating global systems change, but this workshop reinforced our belief that this approach is lacking and necessary, especially in the field of international development. We hope to increase both demand for Blue Marble Evaluators and the supply/availability of them. On one hand, there are already many global initiatives operating, but few have evaluators at the table to inform decision-making and encourage the use of systems thinking and complexity concepts in their evaluation approach. On the other hand, there are even more development projects being undertaken without a global perspective despite the fact that the issues they are addressing are undeniably global. We believe Blue Marble Evaluators can play a role in helping to re-frame these projects and design evlauations that capture the complexity and system-wide implications of this important work. 

Ultimately, what came out of this workshop was the recognition that evaluating global systems change requires Blue Marble Evaluators who understand the historical forces that have shaped the current world system, are committed to developing their cultural competence in order to effectively collaborate and communicate with diverse populations, and bring a unique combination of empathy, humility and the capacity and willingness to speak truth to power in highly politically-charged situations. The first cohort of IPDET Blue Marble Evaluators are tackling a range of complex issues from land rights in the Mekong delta to migration in West Africa, access to anti-retroviral drugs to human security. We hope this workshops laid the foundation for these pioneers to bring a blue marble perspective to their work. 

By Charmagne Campbell-Patton

MQP's Top Ten List of the Future of Evaluation

One June 8th, I gave the closing keynote address at the Canadian Evaluation Society's annual conference in Saint John's, Newfoundland, where I listed my top ten trends for the future of evaluation. They've since gotten some traction on twitter, but a few are out of order, so I thought I'd share my official list here for all to see: 

10. Integration of mixed methods

9. Increasing importance of data visualization

8. Increased us of social media for all aspects of evaluation

7. Learning from failure

6. Getting serious about unanticipated consequences and investigating unknown unknowns

5. Evaluation as the alpha trans-discipline (Scriven)

4. Integrated, holistic multi-disciplinary complexity-informed systems thinking

3. Real-time evaluation for a fast-paced world

2. Speaking truth to power

1. Global systems change evaluation

Click here to download the slides of my full keynote, "The Future of Evaluation: Beyond here there be dragons, or are those just icebergs?" 

Back to back book reviews, oh my!

A quick blog today to let you know that Michael and I have back to back book reviews in the latest edition of the upcoming edition of the American Journal of Evaluation. You can view them online now or wait for the printed edition to come out next month. We've also posted links to the reviews on our website for easy access, just visit our resources page. Michael reviewed a memoir by Ernie House called Cherry Street Alley, while I dove into an edited volume on Peace Education Evaluation. Let us know what you think!