Academic | Triangle

Evidence that the Star accurately reflects change occurring during service provision

Posted on 24th October 202312th April 2024 by Outcomes Star

We are sometimes asked whether changes in Star readings actually reflect the changes that occur during service provision. We have three responses to this: the first two are based on the practices we have in place for the development of new Stars and for training and implementation of the Star. The third is to present the research evidence that the Star readings can be applied accurately and that readings correlate with other measures in the expected way.

Star Development
- New versions of the Star are created alongside managers and practitioners to ensure the Journey of Change captures key changes occurring for those using services.
- Pilot data is statistically analysed to check that the scales are sensitive enough to detect change.
- Service users and practitioners provide end-of-pilot feedback about the extent to which the Star captures the changes made.
Training and Implementation
- For the Star to accurately reflect change, practitioners should be well-trained with ongoing support to continue using it well. This is why training is mandatory, we provide free CPD for licensed trainers and encourage refresher training, regular supervision, and auditing.
Research Evidence
- Convergent validity: Star readings have been shown to correlate with other validated measures in our own, as well as in external peer-reviewed research.
- Predictive validity: Star readings, and change in Star readings, predict hard outcomes such as securing accommodation, employment, and school absenteeism
- Inter-rater reliability: different practitioners are able to consistently assign Star readings

We are keen to conduct further analyses of the relationship between Star readings and other measures, so please get in touch if you have linked data and are interested in us exploring it.

To read our three page briefing providing a more detailed version of the above, please download it here (PDF).

Why relational services matter

Posted on 21st June 202319th June 2023 by Donna

How do you deliver public services that are relational – services that recognise and respond to the unique needs and circumstances of each individual? That was the topic of the conference ‘Towards Relational Public Services’ that I have just had the pleasure to participate in.

Sam Game and I gave a workshop which explored how the Outcomes Star helps to make ‘Enabling Help’ a reality in practice. Working in a relational way is one of the six core principles of Enabling Help, Triangle’s blueprint for effective services.

Sam, who leads the implementation of the Parent and Baby Star in Health Visiting services in Warwickshire, described how using the Star has helped them work in a more relational, collaborative and strengths-based way. Parents have responded incredibly well. They are much clearer about the health visitor’s role and they love the way that Star helps them see what is going well as well as where they have needs. And it isn’t just the parents. It has also helped commissioners to understand and value the service.

Our workshop was one of more than 40 sessions which took place over two days at Newcastle Business School – part of Northumbria University.

For me the highlight was Mark Smith’s presentation on service transformation work in Gateshead. He described how they have implemented an approach to working with people with complex needs in which there are only two rules: ‘do no harm’ and ‘stay legal’. In this ‘liberated method’ case workers have a low caseload and there are no limits on how long they can work with someone or what kind of support can be provided. And case workers have budgets and the discretion to spend money to help solve pressing problems.

What I loved about this approach was that the bureaucracy was stripped out, giving workers the chance to do simple things that could make a big difference and avert much greater problems down the line.

Mark is part of a growing movement of managers, practitioners and academics who are challenging the orthodoxies of New Public Management (sometimes described as the 3 Ms – Markets, Metrics and Management). They are making the case that outcomes emerge from many interacting factors and therefore cannot be ‘delivered’ or dictated. In this complex environment, learning rather than prescription and control is the most effective management strategy. And it makes more sense for commissioners to take a relational approach and to hold service providers accountable for learning rather than targets. This new approach is called Human Learning Systems.

Toby Lowe, Visiting Professor of Public Management and the Centre for Public Impact closed the conference with a call to action – for all of us who want to see services that are more relational, responsive and effective to come together and work with a coalition of the willing to create a paradigm shift in public management. It is an exciting vision, and one that I think the Outcomes Star, a tool with flexible, relational working at its core, can play a role in realising.

Joy MacKeith, Co-creator of Outcomes Star

****

Watch Joy’s conversation with Toby Lowe about the synergies between Enabling Help and Human Learning Systems

Find out more about Enabling Help here

Find out more about Human Learning Systems here

Going for collaboration at the GoLab Conference

Posted on 26th September 20224th October 2022 by Donna

Hear Joy describe what Triangle has learnt about what works in service delivery in this podcast

Watch her conversation with HLS proponent Toby Lowe

In early September, I had a fascinating two days at the Government Outcomes Lab (GoLab) Social Outcomes Conference.

I learnt about Social Impact Bonds (SIBs) and other outcomes-based funding initiatives around the world, from France to Australia and Paraguay to Japan, but for me, the most interesting session was on the UK’s Life Chances Fund.

We heard from many of the initiatives receiving this funding, all of them doing incredible work in a range of sectors including vulnerable mothers, employment and mental health.

The strongest common theme was how this additional funding and its link to long-term outcomes drove meaningful collaboration between providers and commissioners. In these initiatives, all parties are working together to identify blocks and create an ecosystem of services that has the needs of the person being helped at the core. It was inspiring to hear about their work.

Interestingly there wasn’t very much talk about the payment mechanisms and the extent to which linking funding to the achievement of particular targets was a valuable part of the process. From behind-the-scenes conversations I have had at this conference in previous years, I know that this can be a sensitive subject. Projects can be reticent to raise this issue when funders are in the room.

There were, however, some interesting discussions about the sustainability of these programmes. Although they can demonstrate cost savings, these are not always savings to the organisation providing the funding. For example, the local authority provides the funding, but the health service makes the saving. And they are not always ‘cashable’. For example, re-offending rates are reduced, but that does not make it possible to close down a wing of that prison and thereby reduce costs, at least in the short term. This fact can threaten the sustainability of SIBs as the rationale depends on the logic of these cashable savings. Perhaps we need to look for other mechanisms to support collaboration.

I have attended this conference for the last four years. This year I sensed a growing recognition of the importance of a person-centred approach to service delivery and the need to create a collaborative service system to achieve this. And more and more people now have experience of seeing how this can deliver improvements in practice. What we need now is a way of making this way of working part of typical day-to-day practice – in commissioning and service management, rather than something that happens when there is an additional pot of money to incentivise and support it.

Here I think Human Learning Systems (HLS) has some of the answers. It is an approach with problem-solving and ‘doing what it takes’ rather than ‘doing by the book’ at its core. This was a point made at the event by Lee Whitehead of Manchester Metropole University. He studied many SIBs and found that those that followed the HLS principles were more likely to succeed. Gary Wallace of Plymouth also talked about how they are taking the kind of relational approach that both HLS and Triangle recommend outside of the context of a SIB.

I’ve been exploring HLS and the overlaps with Triangle’s Enabling Help principles with Toby Lowe, one of its leading proponents. You can watch our conversation here.

And for a fuller description of what Triangle has learnt from twenty years of creating and supporting the Outcomes Stars in practice, listen to this episode of Next Stage Radicals where I describe the main messages of our Enabling Help report.

Also, look out for October’s edition where Mark Smith of Gateshead Council will be talking about how they are transforming their service delivery – informed by HLS and working with the Outcomes Star^TM.

I will certainly be back at GoLab in 2023. I hope there will be even more space next year to hear from people like Mark who are finding ways to implement the person-centred and collaborative approaches, that are so important to Triangle and the people who use our Outcomes Stars– with or without a SIB.

Further reading: Dr Anna Good shares her thoughts on #SOC22

Musings on the importance of trusting relationships, professionals’ emotional labour, and the desire to improve delivery through data.

GoLab – Social Outcomes Conference 2022

Posted on 26th September 20224th October 2022 by Donna

One presentation that particularly resonated with me was from Reji Ikeda at the Ministry of Justice in Japan. Talking about ‘value measurement for offender rehabilitation’, he emphasised the importance of the quality of the relationship and the ‘emotional labour of professionals and volunteers’ on rehabilitation outcomes. He also spoke about how poor occupational well-being can negatively impact these relationships. I couldn’t resist mentioning the Justice Star as an excellent tool for developing trusting relationships and boosting staff morale in this context.

Joy’s recent report Enabling help puts human relationships at the heart of effective service delivery, so this is at the forefront of my mind when thinking about the Star. The Enabling help report highlights the problems with borrowing from approaches such as the scientific and economic paradigms in social provision. Although not focused on the Star, the innovative suite of Stars offers a better, more enabling approach. I thought of this when the keynote presenter, Julie Battliana, stated that ‘agitation is alone is not enough to make change happen, innovation is required’.

Another theme that I noticed running throughout the conference was the desire to use data to improve service delivery. I felt this was encouraging in a conference often focused on meeting targets and payment by results. Miika Vuorinen, a Chief of Evaluation from Finland, commented that one way to know if social outcome frameworks are working is to see if they are being integrated into the decision-making processes. Closer to home, Stan Gilmour from Thames Valley Police gave a powerful presentation, emphasising ethical decisions based on using data to allocate resources and plan social impact.

It’s possible I was looking out for these positive signs, as I feel passionate about encouraging and supporting good use of Star data. By ‘good use’ of data, I mean those closest to the creation of Star data (practitioners and managers) interpreting the data and using the learning to encourage changes needed to maximise the outcomes for those being supported.

Further reading: Going for collaboration at the GoLab Conference

Triangle co-founder and Outcomes Star co-author Joy MacKeith reflects on the theme of collaboration at this year’s Social Outcomes Conference and points to Human Learning Systems as an effective way to make this the norm in service delivery.

The Family Star Plus within the Supporting Families Programme (2022-25)

Posted on 27th June 202229th June 2022 by Donna

All local authorities delivering Supporting Families must have an approved Outcome Plan setting out their indicators of eligibility with associated family level outcomes against headline objectives set out in the programme guidance.

Since the start of the supporting families programme (formerly called the ‘Troubled families programme’), many local authorities have been using the Family Star Plus as part of evidencing outcomes within their approved Outcomes Plans. This version of the Star was developed specifically to be aligned with the programme’s objectives.

We are currently putting together guidance to show how the Family Star Plus fits the increased list of headline outcomes in the ‘Supporting Families Outcomes Framework 2022-25’, which will be in effect from 3 October 2022:

The new framework identifies ‘practitioner assessments and contacts’ or ‘Practitioner and/or self-assessment’ as valid data sources as part of the evidence within all of the ten headline outcomes, and the Outcomes Star, while collaboratively completed, fits within this category.

‘Validated outcome measures’ are also suggested as evidence for meeting some outcomes. The validation of the Family Star Plus as an outcomes measure, including evidence of ‘inter-rater reliability’ and ‘predictive validity’ is reported in a recent peer-reviewed journal article (Good & MacKeith, 2021).

We will publish the 2^nd Edition of the Family Star Plus shortly as well as guidance on how to use the Family Star Plus (2^nd Ed.) within the new Supporting Families framework.

Please contact our Research Analyst, Dr Anna Good (anna@triangleconsulting.co.uk) if you would like to find out more.

New ‘How to’ guide explains how the Outcomes Star can turn Human Learning Systems principles into practice

Posted on 16th June 202220th June 2022 by Outcomes Star

Since the publication of our Enabling Help report last year, we have been working with Toby Lowe and colleagues at the Centre for Public Impact to articulate how the Outcomes Star can operationalise learning as a management strategy and other core components of the Human Learning Systems (HLS) paradigm.

Both HLS and Enabling help focus on moving on from the failings of New Public Management towards an alternative vision of supporting ‘human freedom and flourishing’ through creating flexible, relational, compassionate service delivery systems in which the role of measuring is to support learning for individuals, organisations and places.

The Star as a tool for turning HLS and Enabling Help principles into practice

We are delighted that the Star is one of the tools identified in the Centre for Public Impact’s recently launched guide: Human Learning Systems: A practical guide for the curious, commissioned by Healthcare Improvement Scotland and Iriss.

The guide highlights three different ways that the Outcomes Star can be used to put the principles of Human Learning Systems into practice:

Person level: the Star empowers people to take an overview of where they are, understand their life as a system, choose their goals and work out the next steps to achieve them
Organisation/place level: the Star enables meaningful purpose-aligned reflection and learning about what is and isn’t working within services, in order to better support those they serve
Connected learning cycles: the Star offers a way of collective sense-making between different actors within or across organisations, for example within a multidisciplinary team supporting an individual, or between commissioners and service providers

Read the Outcomes Star case studies included within the guide.

The synergy between the Star and human learning systems

HLS started as an analysis of the problems within traditional approaches to public service management. It has gone on to articulate an alternative framework and then look for practical ways of bringing that vision to life.

The Outcomes Star, in contrast, started as a practical tool for service providers to evidence that they are making a difference in a way that really reflects and supports what they do. From there we articulated the vision of service delivery and public management that was implicit in the tool in our Enabling Help report.

The convergence of these approaches, despite their different starting points, speaks volumes. The time has come for this shift in thinking and practice. We look forward to furthering collaboration with the HLS movement and others who share this vision.

Could your use of the Star inspire others?
Are you working with the Star as part of a wider HLS approach?
Please get in touch with our Research analyst, Dr Anna Good (anna@triangleconsulting.co.uk), as we are keen to create and publicise more case studies.

Human Learning Systems – A practical guide to doing public management differently

Posted on 16th June 202216th June 2022 by Donna

Last year, we worked with several organisations to release an e-book – Human Learning Systems: Public Service for the Real World. This had one key message: if we want to achieve real outcomes in the world, then we need to do public management differently. That is, we need to plan and organise public service in a radically different way.

Back in 2016-17, when a few of us first began to explore this idea, this was a distinctly heretical position to take. The people that spoke to us for our first report did so on condition of anonymity. Fast forward five years. Now, not only do we have a good picture of what an alternative approach to public management looks like, we have over 50 case studies of that in practice from around the world. And those who have been at the vanguard of this practice, like Plymouth Council, are winning national awards for doing so.

At the same time, across Scotland’s health and social care system, there is a growing appetite to commission differently, to better focus on creating the conditions for people to achieve the outcomes that matter most to them. That’s why Healthcare Improvement Scotland and Iriss commissioned the Centre for Public Impact to create a practical guide for organisations who want to use a Human Learning Systems approach to public management.

We’re excited that the guide, Human Learning Systems: A practical guide for the curious, is now ready for organisations to use. But, before we ask you to dive in, we wanted to share a little about the journey we’ve been on so far, the need for this resource, and what you can expect.

Recognising the complexity of our lives

Human Learning Systems begins with the recognition that real outcomes are created by the unique combination of actors and factors, and the relationships and interactions between them, that make up a person’s life.

The exact combination of relationships between actors and factors that create a desirable outcome in one person’s life will be different to another person’s. And, it will change over time, as our lives evolve, and the world changes.

Learning as a Management Strategy

Therefore, if we care about outcomes, we need a different approach to public management which recognises this complexity. We need Learning as Management Strategy. This is what our new guide to Human Learning Systems helps organisations get to grips with. Our guide enables those engaged in any aspect of social action or public service to organise themselves to have the best chance of creating genuine outcomes in the complex environments of people’s lives.

Learning as Management Strategy changes the focus of what can be planned and managed. Rather than trying to meticulously pre-plan and organise a programme of activity in someone’s life, it enables public facing workers to learn together with the people they serve. They can learn about the unique nature of each person’s life as a complex system – the key relationships and the driving factors in their life – and how all of those things interact. And then, when they have learnt together, it enables the workers and those being served, to explore and experiment together around what helps to produce better and different outcomes in people’s lives.

All of this exploration and experimentation generates challenges for how systems at other scales are planned and managed – how teams, organisations, places and nations work. And so the managers at these scales need to undertake the same processes of learning and experimentation – to ensure that the action-learning which supports the creation of real outcomes happens rigorously, efficiently and effectively.

Human Learning Systems: A practical guide for the curious

All of these action-learning processes can be planned and managed.

Our new guide is designed to help people understand the practical actions they will likely need to take to plan, organise and undertake these action-learning activities.

The guide incorporates case study examples, written by Andy Brogan from Easier Inc, and Mark Smith from Gateshead Council, to illustrate the applications of the framework in real life. It draws on the richness of their experiences to explore the detailed methods they use, the obstacles they encountered, and how they responded to those obstacles, all while placing learning at the heart of their approach.

The guide was also tested with public sector and voluntary organisations across Scotland who supported us in further iterating and developing it so it can support experimentation in different contexts.

How can you help?

This is just the beginning. We’re keen to keep testing and iterating the guide and keep asking the question: “what help do people need to adopt a Human Learning Systems approach?” We’ll be seeking feedback from everyone who downloads it – was it useful? How could it be better? Do write to us and tell us how you’re using the guide. With your help, we can truly create something extraordinary.

Explore the guide

We’ve worked with partners to develop a practical new guide to help you apply the Human Learning Systems approach to your work. Download the guide to begin your journey towards more human public services.

This article was originally published by the Centre for Public Impact.

Written by:

Toby Lowe Visiting Professor in Public Management

View biography

Chandrima Padmanabhan Senior Associate, Europe

View biography

Making change visible: The Outcomes Star captures important achievements that could be missed by focusing on hard outcomes

Posted on 27th January 20222nd February 2022 by Donna

As the creators of a suite of measures capturing distance travelled towards ‘hard outcomes’ we are sometimes asked whether there is evidence that Star readings correlate with or predict outcomes such as offending or employment. In some cases, we hear there is resistance to using the Star and instead commissioners, managers or funders are only interested in how many service users have ticked the box of meeting these hard outcomes. This misses out on capturing important achievements, ignores the role of internal change in maintaining concrete achievements and disincentives working with those most in need of support.

This briefing describes some of the evidence we have of the ‘predictive validity’ of the Star – that it does in fact predict outcomes such as school attendance, employment, training and accommodation status. This includes findings reported in two articles recently published in peer-reviewed journals.

In it, we also explain the value of the Outcomes Star in measuring the full journey leading up to and including changes in behaviour or circumstances.

The author of this briefing, Dr Anna Good, draws on her expertise in behaviour change theory to summarise the strong evidence base supporting the importance of the changes assessed by the Star. It is clear from the research literature (and our extensive experience of working with service providers), that early steps on the Star’s ‘Journey of Change’ such as acknowledging problems and accepting help are often essential to subsequent change in hard outcomes. Moreover, change in skills, confidence and beliefs are often key factors in the maintenance of life-changing improvements.

****

Please download our new briefing, ‘The Outcomes Star captures important achievements that could be missed by focusing on hard outcomes’. If you would like more information or support about the use of Star data, please get in touch with us at info@triangleconsulting.co.uk or +44 (0) 207 272 8765.

The Outcomes Star as a management information tool

Posted on 29th December 202110th January 2022 by Donna

Learning from Star data at all levels of the organisation

Over three-quarters of Outcomes Star users in our client survey said Star data reports were ‘useful for learning how their service was ‘doing’ and ‘helpful in managing or developing the ‘service’. Indeed, Star data can provide meaningful management information at all levels, from a service manager reviewing a single ‘worker’s caseload to a senior management team reviewing data aggregated across services.

Alongside other data (e.g. satisfaction surveys, output and process data), Star data reports, such as those available from our upgraded Star Online System, allow organisations to ask increasingly focused questions about what is happening with the people they support.

Managers can gain essential insights by looking at differences in starting points and change across outcome areas, client groups, and service settings. Because these insights are likely to be greatest when compared against prior expectations, Triangle has produced resources to support ‘Star data ‘forecasting’.

Learning from Initial Star readings

The distribution of first Star readings provides a valuable overview of people’s needs coming into the service. Star readings can be compared against expectations to ensure that service users are entering the service appropriately and are offered suitable interventions.

An excellent example of the use of first readings is in Staffordshire County Council, where they look at start readings to see if the families are at the right level of service. In our interview with the Commissioning Manager at the time, she told us that “if we have families in our Family Intervention service that have readings of five, I look a bit deeper to see if we’re really using our resources correctly”.

Learning from change in Star readings

Movement in Star readings for each outcome area also provides an opportunity to learn where things are going well and when further exploration of service delivery may be warranted.

For example, if one service shows different outcomes to another service, this is a starting point for further investigation:

Is there other evidence that one service facilitates better outcomes than another?
Are there reasons why one service might be supporting people better than another?
Is the service user profile different in the different services?
Is practice significantly different in that service, and might there be lessons for other services?

A more in-depth analysis of the movement from each Journey of Change stage is also possible, offering more significant potential for learning than typical numerical outcome scales. Managers can explore which stage transitions are happening frequently and where there may be blockages to making other transitions. For example, a service may be very good at helping service users to begin accepting help but struggle more with moving them towards greater self-reliance, limiting the progress currently being made. Specific changes to service delivery might then need to be developed.

*****

Please download our new briefing, ‘The Outcomes Star as a management information tool‘ for more detail on how Star data reports can be used to improve service delivery. If you would like more information or support about the use of Star data, please get in touch with us at info@triangleconsulting.co.uk or +44 (0) 207 272 8765.

How the ideas driving social provision are steering service delivery off course

Posted on 19th July 20219th November 2021 by Donna

Triangle, the social enterprise behind the widely used Outcomes Star tools, is calling for a paradigm shift in social provision. In a new report to be published in September 2021, co-founder and Star co-author Joy MacKeith argues that at its heart service delivery is about meeting human needs and changing behaviour. Everything we know about how change happens points to the importance of relationships, trust and connection. Research also shows that services must be holistic and tailored to each person. But the ideas currently driving social provision steer the focus away from relationships and flexibility and onto procedures, markets, targets and standardisation. They break service delivery down into parts rather than focusing on the whole system.

The report presents an alternative vision – an enabling approach to service delivery. Called Enabling Help, this alternative puts the focus of the service delivery system on the service user, rather than the helper, the service or the intervention. Enabling Help builds relationships, trust and hope, develops skills and capabilities, is holistic, responsive and tailored to each individual person.

It also paints a picture of what it means to make Enabling Help a reality in practice. At the front-line it means moving to a collaborative approach rather than telling and directing. For managers it means changing the emphasis from managing procedures to enabling front-line workers to deliver relationship oriented, collaborative, flexible, problem-solving services. For commissioners it means shifting the focus from numbers to narratives – co-learning with service providers about what works.

‘Enabling Help’ builds on Triangle’s twenty years’ experience of helping organisations to support and measure change for people receiving services. Working with over one hundred collaborating organisations including local and national charities, housing associations, grant-making trusts, local authorities and NHS trusts, has provided a unique insight into what works when supporting change and building well-being and potential. And training and supporting over one thousand organisations to use the Outcomes Star in practice has highlighted what can get in the way of delivering what works. This report pulls all this learning together and identifies the real reasons why people being helped get stuck in services and the people delivering the help feel frustrated and de-motivated.

The report calls on all those involved in service delivery from front-line workers, to managers, commissioners, researchers and policy-makers to embrace this new set of ideas and put relationships, responsiveness and learning at the heart of everything they do.