Where nothing is clear, and everything keeps changing – new territories for evaluation

Greetings from the Faroe Islands.  In my efforts to develop new perspectives on evaluation I’m attending a conference on Communication, Collaboration and Relationships.  This is my annual dose of CPD – like many people from the UK, this is new territory for me and I couldn’t resist the location and the conference aims to ‘increase the motivation and the joy of learning, teaching, leading and serving’ and ‘bring public services into synchrony with emerging world conditions’.  One of the keynote speakers is Ken Gergen who amongst his many writings, articulates a vision of the researcher as an active agent in fashioning the future and research as a form of social action.

IMG_2817

This conference was nicely timed as I am hoping to write something to express the shifts in thinking about evaluation that I see emerging shoots of, as I go about my work in Scotland.  I’ve been revisiting the work of Michael Quinn Patton who is a well-known evaluator and former president of the American Evaluation Association.  He’s always an entertaining speaker and last year, he was in Scotland attending the Transformations 2017: Transformations in Practice Conference at the University of Dundee.  Most memorably he said, ‘we are familiar with systems thinking, but we haven’t used it in evaluation’ and went on to describe traditional evaluation as a ‘barrier to transformation’.

I suspect this would resonate with the people who attended the recent 2018 Fire Starter Festival event on Reigniting Evaluation.  Based on the premise that what you focus on becomes your reality, we deliberately focused on our experience of evaluation at its best.  There was a real appetite for evaluation – the generation and use of evidence –  to contribute to transformation within human services.  We know it can happen; we didn’t spend our time commiserating or outdoing each other with horror stories, instead we were future-focused – we identified what values and qualities we’d want to keep and what that would look like.  We talked about what we’d need to let go of and what would help us to discard those things we no longer want.  These aspirations are well expressed here and within these stories are many allusions to the unintended consequences of the way that we have traditionally thought about and practiced evaluation.

IMG_2819

There is no doubt of the desire that evaluation should be a tool for learning, to help us develop our practices, collaboration and performance – to do well in our work, to do even better and to learn our way through things when nothing is clear, and everything keeps changing – a bit like the weather in the Faroes.  When we are working with the complexity and messiness of communication, collaboration and relationships, when we are seeking to make the world a better place, we need some new ways of thinking about evidence, that treats evaluation itself as an intervention.   I often hear that ‘it’s all about relationships’  – it is clearly time to change and shift our focus to relationships, to our various and shared visions of a better future and even to play catch-up with ideas about complexity and systems thinking.    More follows soon and at #Faroes18

Posted in 5th generation evaluation, Appreciative inquiry, Co-production and new ways of working, Courses and events, How can evaluation be useful?, Living systems research, Research Impact, Stories, What is action research? | Comments Off on Where nothing is clear, and everything keeps changing – new territories for evaluation

“The cynicism that passes for sophistication”

I’m not sure when or how I first encountered Appreciative Inquiry.  But I do remember going to an Imagine Chicago conference in 2002 with a group of community activists from Edinburgh.  We had a thoroughly exciting time, meeting appreciative inquirers from all over the world in an explosion of energy and creativity.  At the time I was a research commissioner and manager working for a public agency in Scotland, with a brief to develop new funding for community action research.  I was an experienced social researcher myself, with a PhD and a keen interest in making an impact.

On the flight home I poured over the information and tried to make sense of it all – on one hand, I liked it and the energy of the people I’d met.  On the other, I was a little sceptical, especially about the idea of the focus on the positive and I was a bit stung by the idea that my more critical queries might indeed be what someone had called, the ‘cynicism that passes for sophistication.’ Ouch!

At the time I was immersed in the CARPP – Centre for Action Research in Professional Practice – programme at the University of Bath where, at times, I felt like I’d fallen off my ‘social researcher’ bike and was learning to ride it again.

bikecrash

To cut a long story short, I ultimately left my post and set up my own action research company, Research for Real.  I’ve been working for myself now for 15 years.  Originally, I was very influenced by the ‘Bath’ school of action research, that talked about first, second, and third person inquiry and ‘living life as inquiry’.  I also worked with Danny Burns and Susan Weil from SOLAR at the University of the West of England in Bristol and was excited by their approach to systemic action research, for example in the work for the British Red Cross.  These people continue to inspire me, as does Yoland Wadsworth from Australia who talks about human inquiry for living systems by building in research and evaluation.

You might say I dabbled in appreciative inquiry.  I found my own way of introducing it, particularly thinking about being more positive in evaluations and I ran some successful one-off events that helped people to understand their shared values and common goals.  My chance to immerse myself in it more fully came as part of the My Home Life team at the University of the West of Scotland and in some spin-offs from that work.  For about the last five years, I’ve been working alongside Belinda Dewar and her colleagues and have taken forward my own work using similar approaches in health and social care and in community development.

We’ve written about it and I think we’ve found way of integrating the generativity, imagination and attention to language of appreciative inquiry, with the focus on collaboration, experimenting and the practical orientation of action research.  I particularly like the future-making orientation to research as a form of social action, that both excites and incites further change.

So often, I am privileged to witness people getting excited again about the work they do, when they are asked to notice what they value, what matters to them or what works well. I like the focus on the ‘here and now’ – not starting off by imagining how we want things to be in some distant future, but rooting ourselves in what we already have and value, that we might too readily dismiss or take for granted.   We encourage people to talk to each other about how they feel about both the good aspects and those that are tricky and then, to work out, together, what kind of future might be possible.  And then to try something out, to test it, review and consider whether and how to sustain it.

So, yes, it has a bias – a real-life-centric bias.  It’s nurturing and emergent, and places relationship and conversations at the heart of change.   Now, I can’t imagine another way.

Posted in Appreciative inquiry, Co-production and new ways of working, How can evaluation be useful?, Leadership, Living systems research, Participatory research methods, Research Impact, What is action research? | Comments Off on “The cynicism that passes for sophistication”

Reigniting Evaluation – Unfolding Stories

Today about 24 people gathered in Edinburgh to explore what they value about evaluation, as part of the 2018 Fire Starter Festival.

2018-02-02 11.13.25Working with our own stories of experience of evaluation at it’s best, we identified what values and qualities we’d want to keep and what that would look like, going into the future. We talked about what we’d need to let go of and what would help us to discard those things we no longer want.

This collection of Unfolding Stories of the Future is the start of a series of conversations we are keen to continue.   Here’s a sample:

  • My word of the moment is engagement
  • It’s a bit radical but could we do away with the word ‘evaluation’?
  • I would like to think more about why we are evaluating – or prove or improve?
  • Let’s stop fruitless, box-ticking outcome evaluations
  • I’d be upset if this was just an interesting conversation and nothing changes when we leave the room
  • Let’s start valuing and understanding what we do well
  • Imagine if everyone valued and enjoyed evaluation

With a nod to the Year of Young People in Scotland, we felt that we wanted children and young people to understand the value and importance of reflection and learning, with others.  A few people signed up to be part of a small group keen to take this thinking forward – watch this space for further dialogue.  Please get in touch if you want to take part yourself.

Imagine if we made a difference – let’s try!

Posted in Appreciative inquiry, Co-production and new ways of working, Courses and events, How can evaluation be useful?, Leadership, Living systems research, Participatory research methods, Partnership working, Research Impact, Stories, What is action research? | Comments Off on Reigniting Evaluation – Unfolding Stories

When are you ever not prototyping?

I’m co-hosting an event at the forthcoming Fire Starter Festival.  We want to ignite conversations about learning-in-action to stimulate thinking about how evaluation practices might flourish to promote learning and innovation.

We put prototyping in the title as a deliberate provocation.  The term is used a lot – thanks in large part to the work of Ulab Scotland  Prototyping translates an idea or a concept into experimental action.  We like this idea of tentativeness, of uncertainty about whether or how something might work and for whom and of being open to being surprised or wrong.   In public programmes, we’re often very committed to cherished ideas and initiatives, especially when something seems so self-evidently ‘worthwhile’.  This word of warning from Gervase Bushe is helpful:

“In prototyping, it is important that people do not fall in love with an idea too early, so that they are still able to go to the stakeholder with an open mind and heart.  They need to be flexible and ready to hear the others’ experience of and reaction to the prototype.”

With this in mind, I’m especially interested in the idea of Living Labs – where ideas and innovations are tested out and refined in real-life contexts.  These experiments start in small, quiet ways, with practitioners who are working with real issues.  How can we make these practices more widespread?  Where does the evidence we already have about ‘what works’ sit alongside emerging new insights?

At the event, our intention is to focus on the future and to provoke discussion amongst us about how we can align our values of participation, collaboration and appreciation with our thinking and practice about evidence to recognise what is working well and to make the impact that is needed.  There are still a few places and we hope to appeal to anyone with an interest in how to create better public services.  Sign up here.

We also hope that this event will lead to a series of ‘Touch Papers’ to continue and broaden this dialogue and develop ideas about “5th generation evaluation” which will develop collaborative and appreciative new practice-oriented evaluation approaches and methods. Do get in touch if you want to contribute.

 

Posted in Appreciative inquiry, Co-production and new ways of working, Courses and events, How can evaluation be useful?, Leadership, Participatory research methods, Partnership working, Research Impact, Stories, What is action research? | Comments Off on When are you ever not prototyping?

Are we working well together?

I often ask myself ‘is this worthwhile work?’ And this guest blog for Space Unlimited is some part of a response; it describes a recent highlight of a stimulating week when I took part in a Space Unlimited Lasting Change event that brought young people and teachers from different schools together to explore how to engage and sustain young people to be genuine and active partners in learning.  They have a great deal to teach those of us interested in how to develop collaborative leadership practices that might produce better outcomes for people and communities.

Posted in Co-production and new ways of working, Leadership, Partnership working, Research Impact, Stories, What is action research? | Comments Off on Are we working well together?

Is this the best it can be?

I’ve very much enjoyed being involved in the development of this new toolkit for Creative Scotland. The ideas and approaches were developed from desk research and  action research, with seven pilot partnerships of artists, partners and participants using the tools  and sharing learning.  It was fascinating to be engrossed in the challenges of translating principles derived from research into best practice, into useful and practical tools that arts organisations and their partners would actually want to use.

This toolkit aims to open up conversation within partnerships about what is important and what can  be improved. It doesn’t try to define or limit an understanding of what ‘good work’ is.   Rather it aims to help those using the tools to openly discuss what they’re doing – asking themselves, and all those they work with, to think about whether they can do what they do in better ways. The goal is to encourage  a culture of reflection and continuous improvement.  In this respect it’s relevant for many partnerships situations where people need to develop mutual understandings of what they have come together to do.

Posted in Leadership, Participatory research methods, Partnership working, Research Impact, Stories, What is action research? | Comments Off on Is this the best it can be?

Participative transformation?

I have recently written a book review of Participative transformation learning and development in practising change, by Roger Klev and Morten Levin in the Journal Action Learning:  Research and Practice, 2015

They propose that given the challenges of the uncertainty in which we live, there is a need for learning processes that support and develop the practice of leading change, not by copying recipes or methods, but by being able to create collective reflections around our own and others’ experiences.

So, it’s highly relevant to discussions about developing a ‘Scottish Model’ of learning and change, leadership development and the challenges of health and social care integration.

I hope that their book and my appreciative critique of it can contribute.

The first 50 people who download it through this link will be able to do so for free.

http://bit.ly/1CQ14xa

 

Posted in Co-production and new ways of working, How can evaluation be useful?, Leadership, Partnership working, Research Impact, What is action research? | Comments Off on Participative transformation?

Calling time on ancedotal evidence….

I notice my irritation rising whenever someone refers to ‘anecdotal evidence’. As if people are saying the evidence is merely anecdotal. It’s unreliable and based on hearsay. It doesn’t count for much. And so often, such remarks are made in a situation where what we are actually discussing is people’s lived experience, of those who use public services or who work in them….Guest blog at the Alliance for Useful Evidence

Posted in Co-production and new ways of working, How can evaluation be useful?, Living systems research, Participatory research methods, Research Impact, Stories, What is action research? | Comments Off on Calling time on ancedotal evidence….

Caring conversations in action

Taking time to hear the stories that matter to matter pays off in the long run…it’s just the way we do things round here.7A

 

Asking for feedback can feel a bit embarrassing but we find the courage because it’s good to know what works well.

32A

 

 

We see inequalities sensitive practice as more than providing equality of care to clients.   We stay curious and find a way to ask what people need. 

28A

 

 

These are just three positive practice pointers that were developed in a recent Inequalities Sensitive Practice inquiry involved practitioners working in early years, homelessness and primary care mental health settings in North East Glasgow.

A strong message from the inquiry is that inequalities sensitive practice is about more than whether services are providing ‘equality of care’ to clients, patients or service users.  It is also about professional practice, the daily business of how staff interact with the people they work with – their clients and with each other.   Read more here about how using caring conversations helped to uncover the good practice under the radar.

Posted in Co-production and new ways of working, How can evaluation be useful?, Living systems research, Participatory research methods, Partnership working, Research Impact, Stories, What is action research? | Comments Off on Caring conversations in action

Research as if people were human

I’m delighted to be a guest blogger for IRISS this week.  This post talks about how stories energise and restore people’s connections with each other and what they care about, so motivating people to work together.  Action research – a truly humane approach to inquiry

Posted in Co-production and new ways of working, How can evaluation be useful?, Living systems research, Participatory research methods, Partnership working, Research Impact, Stories, What is action research? | Comments Off on Research as if people were human