Lessons from a Scottish road trip
By Pierina Dsouza, Senior Officer: Impact Learning & Policy with Tim Burton, Global Communications Lead
A couple of years ago we had family visiting us from India. We decided to go on a big family vacation in the car up to the Scottish Highlands. Three generations of our family were squashed into the car, snacks were distributed, children were distracted, and we set off.
We programmed the GPS and it took us wonderfully along the spine of the UK, right up to the Highlands. However, this is where the problems started.
The GPS was calling out instructions and, like the good city-dwellers we are, we followed them exactly through this rural paradise. Until, that is, we realised we were going round in circles. On the third occasion that the GPS asked us to take the same left that hadn’t existed the first two times, my husband’s auntie shouted at the GPS, ‘don’t listen to her! She has no idea what’s going on!’
After this outburst (and some howling laughter), we decided to ignore the poor woman in the GPS and start to follow our eyes and our common sense. I’m pleased to say we did eventually reach our destination and had a wonderful time – I’d highly recommend a holiday in Scotland!
Strategies are much like the GPS we used that day. We programme them with our destination before we start the journey, we follow them as diligently as we can, and we ideally arrive at our chosen destination on time and in one piece. A project strategy may feel very similar to that.
However, if we use a strategy without data, without monitoring, and without constant evaluation, we end up going round in circles and making little progress. Just as my family began to use our eyes and not just our GPS, monitoring through data provides us eyes to see beyond the strategy, to see whether we are in fact heading towards our desired destination.
Here are a few lessons that we can take from this road trip and can apply to strategies, whether they are for a local project or at a global level.
When we got lost, we suddenly became reliant on using our eyes and our old-school paper maps to get us where we needed to be. Everyone in the car, from the youngest to the oldest, had a pair of eyes and a unique perspective to help us recognise where we were, where we’d gone wrong before, and where we were going next.
The first thing to note about doing data well is that it requires resources: time, staff, and money. We must budget for this and we must use that budget wisely.
Once we have made the commitment to resourcing data and monitoring well, it is important to get everyone in the programme or on the project on board. Everyone involved needs to know WHY we collect data, HOW we collect data, and what level of data disaggregation is needed to give us a credible picture.
We must work with people to determine how to analyse the data to ensure it gives us a clear picture of progress towards our goals and does not lead us down a blind alley.
Whatever level of project or programme we are running, we are always reliant on the hard-working people on the ground to collect data for us. These are people who have competing demands on their time, who want to be dressing wounds, not filling in forms. Are they going to commit time to collecting data for us if they do not understand how it will make a difference to the project? Just because it is clear to me, does not mean it is clear to everyone else.
It is important everyone understands how the data is going to be used to help us all meet our goals. Find time and space to make this clear to everyone involved.
When we got lost, my family added paper maps, eyes, and common sense to the information we were getting from the GPS. We used more metrics.
Let us say you are running a livelihoods project. To assess the success of your project, you can ask a person what their household income is at the start of the project and at the end of the project. If they report an increase in household income that means your project has been a success, right? Not necessarily.
Photo Credit: Ricardo Franco
Their answer to the question ‘what is your household income?’ is highly subjective. There might be cultural, personal, or mathematical reasons why they do not answer the question accurately. They may just be telling you what they think you want to hear. It is not a reliable data point to inform the success of your project.
Instead, use multiple metrics by asking a variety of questions. You can start with their daily or monthly income, but you can also ask what goods the family owns. Do they have a radio, a television, a mobile phone, a vehicle, or a bigger home? What kind of food do they eat and how often?
Has their access to education or healthcare improved? Do they have any savings?
Do you see how these metrics begin to provide a much clearer picture of whether your project has successful led to economic empowerment? We must use multiple metrics for all projects, whether they relate to livelihoods, disability care, self-advocacy, or medical care.
When we were lost in our car, we decided not to rely on our own knowledge, but to ask some locals for help. We pulled into a garage and asked some people from the area if they knew how to get to our destination.
The Eye, Hand, Foot (EHF) score is an established monitoring tool for leprosy-related disability. The Leprosy Mission collects this data to monitor our global strategy.
The figures we were receiving appeared good on face value. We should have been happy, but other disability metrics were telling us a different story; ulcers, reactions, eye problems, and demand for prosthetics and protective footwear were on the rise.
The data was not making sense, so we needed some local knowledge to help us understand what was going on. As we delved deeper into the figures and spoke with our staff, a different picture emerged.
We saw that only 55-60 percent of MDT patients were being monitored for a change in disability. Many staff did not see the significance of monitoring EHF scores (again – the importance of getting everyone on board!). Patient records were being lost and staff were also not being trained. The situation was even worse for patients who had finished MDT, as many of them were not being followed up.
If we had relied only on the EHF scores to give us a picture of disability management, we would have been missing a big problem. Monitoring multiple data points allowed us to see there was an issue and we were then able to use some local knowledge to help us understand what was causing the problem.
Base your targets on the best available science and data. Arbitrary targets allow for hollow success stories. They may work for marketing, but they do not work for communities affected by leprosy.
