My honest review: Govt's new Data Strategy
TL;DR - it's a strategy with a plan and metrics, but they only appear tangentially related to the stated objectives. Doh!
At the end of last year, the Government, released a new Data and Digital Strategy to ensure the APS is keeping up with technology and adopting leading-edge practices.
They also released an implementation plan and a metrics framework with the (laudable) intent of being able to measure the success of the Strategy.
If you haven't seen it, the Strategy has five missions :-
Delivering for all people and business
Simple and seamless services
Government for the future
Trusted and secure
Data and digital foundations
I'll focus on the last three as they are direct enablers for the first two.
Under those 3 missions are 8 outcomes:
Adopt emerging technologies
Modernise investment approaches
Build and maintain trust
Modernise legislation
Connect data, digital and cyber security
Manage data as a valuable national asset
Build a data and digital-capable APS
Grow APS maturity
And tied to those outcomes were 7 measures of success:
% of digital investment proposals in-scope of the Investment Oversight Framework (IOF) which meet the requirements of the DTA’s Benefits Management Policy as assessed by the DTA in a financial year (new metric).
% people who trust government services (61% in June 2023)
Entity security maturity rating (83% in 2022)
% of current APS workforce in data and digital jobs (10.8% in Dec. 2022)
% of women in data and digital roles in the APS workforce (41% in Dec. 2022)
% APS employee satisfaction (74% in June 2023)
Overall entity Data Maturity Rating (new metric)
In order to assess whether they are meaningful metrics, let's conduct a "thought experiment" - could those metrics go up and still not deliver any meaningful improvement in the associated outcomes? If they could, they're not suitable measures of success.
The first metric is based on the proportion of proposals in-scope of the DTA's IOF that also meet its Benefits Management Policy. This metric would improve with a reduction in the denominator - that is, a less ambitious Government agenda for digital transformation could result in a reduction the number of proposals included in the measure, and the metric might go up just because the Government reduces its digital ambition and has less projects assessed.
A better metric might be the proportion of the Government's budget allocated to projects which meet this particular hurdle.
The next metric - trust in Government services - is incredibly broad and the Government's own research shows it is driven by an incredible array of factors, the greatest two by a large margin being "satisfaction with life" and "trust in other people" (see pp.13). Neither of those are directly related to the Government's Digital or Data capabilities.
Again something more specific such as net promoter score across Government services seems like it might be more meaningful.
The third - entity security ratings - does seem like a reasonable way to measure the Government's overall capacity to deliver services in a trusted and secure manner, and the most recent assessment released this month also shows this metric is improving. Bravo.
The next three metrics all relate to APS workforce characteristics such as proportion of the overall workforce in data and digital roles.
The latest numbers (to June 2023) show a decrease in the Data and Digital intensity of the APS workforce: by almost 0.4% (from 10.17% in December 2022 to 9.8% in June 2023)[1] - overall the APS workforce increased by almost 8,000 roles over those six months but the lower proportion of those new roles in Data and Digital job families brought the overall average down (and there was no change in overall diversity of those new jobs).
APS job satisfaction - the metrics framework suggests this measure is useful because it is correlated with customer satisfaction, engagement and productivity - I'm not an expert in this space, but it certainly seems to be a rather tangential metric to measure the success of these objectives.
If the intent is to measure APS capability, perhaps some form of measure of productivity in the Digital and Data space would be appropriate.
The last metric (Data Maturity Rating) is new and I could not find any published details but I will be waiting to see how specific the rating system is.
Overall, the initial set of metrics published by the Government seem very broad and only indirectly related to the linked objectives.
So what?
The importance of metrics as way of measuring impact is that they confirm (or not) whether a strategy is meeting its objectives - without precise and meaningful metrics there's significant risk of wasting resources and failing to achieve success.
I struggle to see how such an ambitious strategy designed to improve the Government's data and digital capabilities will succeed without improvement in the data and metrics used to drive its success.
I worry that there is a real risk that some of these measures will distract the Government and in 2030 we'll be in the same position we are today - sliding further down the global digital rankings.
Felix's Review - 2 stars ⭐⭐
Footnotes
[1] The Government's metrics document states "10.8% of APS Jobs (December 2022)" but they do not specify which job families they include so I'm not sure where 10.8% comes from... My own analysis shows this number at 10.17% in December 2022 based on the proportion of the APS in the 'Data and Research', 'ICT and Digital Solutions' and 'Information and Knowledge Management' job families.