It’s often the case when we are busy, that we don’t take the time to reflect on or celebrate our successes. I am therefore enthusiastic to be taking the opportunity to do just that in this blog, sharing our methods for this project, and how we went about delivering the work.
While our data and analytics team had delivered service demand forecasting work to Adult Social Care (ASC) customers previously, I first got involved in mid-2021. The ask from our Finance colleagues felt like an exciting opportunity to produce forecasts to inform budget setting for ASC following significant changes in demand post Covid-19. Forecasting is the process of making future predictions, typically used to inform decision making, and here we were aiming to predict future demand on ASC services, based on historic demand.
Our data were approximately four years of historic monthly hours or package demand for various ASC services. We used time series forecasting models and initially tested several different models for each service area before choosing the best models based on accuracy and fit. However, there were over 20 forecasts to do, and because we wanted to make the process easily repeatable, I used ‘R’ to build a loop which would batch forecast all our datasets in one go. In this loop (after much discussion!) we settled on using Arima models. We found them to be a good fit for the majority of our data, and they are flexible as you can manually change components where necessary. This was the first time we had done this and a great learning opportunity.
Forecasting residential care demand was a particular challenge, due to the large drop-off in care packages for older people during the Covid-19 pandemic. The initial forecast predicted numbers of packages continuing to sharply decline. I discussed this with our customer, who agreed this was not a realistic scenario as there will always be a minimum level of need for residential care. We adjusted the forecasting model, and this left us with a plateau in demand, and not a nosedive! This is a good example of forecasting being a mixture of art and science - where we also need to use common sense and intuition when we forecast. Just because a model runs and gives an output, doesn’t mean you should use the output if you know it paints an unlikely scenario.
The outcomes of the forecasting work were shared with finance customers by involving them in regular discussions about the forecasts and encouraging conversations about how future work programmes may affect ASC demand. I was also very honest about my own confidence in forecasting so far forwards with recent Covid fluctuations to consider in our data, so we stuck with projecting forward 18 months. Regularly discussing model outputs, particularly the perpetual analyst’s queries over model fit and accuracy also really helped to quality assure our work within the team. The forecasts were a success, building confidence in the accuracy of our work by ensuring transparency throughout on how we got to our evidence base to inform their budget setting process.
In April 2022 we were asked to do the forecasting work a second time. We continued conversations, working closely with our customers to agree our high-level aims for the forecasting work, both short- and longer term. We agreed this time to try to account for savings being made by active work programmes in some ASC service areas, and for some delays in timely reporting in our own data. This didn’t mean changing everything, but we did need to adjust certain forecasts to reflect more accurate data and a true picture of what was going on with ASC service demand. Our sister team (Performance and Business Intelligence) provided us with information about accuracy in data reporting, and Finance Business Partners provided us with data indicating where changes in demand were being influenced by active ASC programmes. The outcome was a suite of forecasts that everyone felt happy with, delivered to an agreed deadline, and confidence in our outputs thanks to our collaboration, whilst maintaining the integrity of our work.
Reflecting on this experience, I take a lot of satisfaction from having a good working rapport with our customers built on trust and honesty, helping to build data literacy and understanding of forecasting as an approach. As a team we progressed our skills to include batch forecasting and we are now working towards fully automating this process! Our next steps are to continue this work and to try some scenario modelling with our customers, and as a team, to use this experience as a great example of best practice when we are engaging with future customers.
1 comment
Comment by Jeanette Young posted on
Really interesting process