The case for adaptive and end-to-end policy management

The APS Reform Agenda provides a rare window of opportunity for addressing structural and systemic issues in the APS, so why not explore how might we transform the way policy is designed, delivered and managed end to end?

Why should we reform how we do policy? Simple. Because the gap between policy design and delivery has become the biggest barrier to delivering good public services and policy outcomes and is a challenge most public servants experience daily, directly or indirectly.

This gap wasn’t always the case, with policy design and delivery separated as part of the New Public Management reforms in the ’90s. When you also consider the accelerating rate of change, increasing cadence of emergencies, and the massive speed and scale of new technologies, you could argue that end-to-end policy reform is our most urgent problem to solve.

Policy teams globally have been exploring new design methods like human-centred design, test-driven iteration (agile), and multi-disciplinary teams that get policy end users in the room (eg, NSW Policy Lab). There has also been an increased focus on improving policy evaluation across the world (eg, the Australian Centre for Evaluation). In both cases, I’m delighted to see innovative approaches being normalised across the policy profession, but it has become obvious that improving design and/or evaluation is still far from sufficient to drive better (or more humane) policy outcomes in an ever-changing world. It is not only the current systemic inability to detect and respond to unintended consequences that emerge but the lack of policy agility that perpetuates issues even long after they might be identified.

Below I outline four current challenges for policy management and a couple of potential solutions, as something of a discussion starter

Current policy problems

Problem 1) The separation of (and mutual incomprehension between) policy design, delivery and the public

The lack of multi-disciplinary policy design, combined with a set-and-forget approach to policy, combined with delivery teams being left to interpret policy instructions without support, combined with a gap and interpretation inconsistency between policy modelling systems and policy delivery systems, all combined with a lack of feedback loops in improving policy over time, has led to a series of black holes throughout the process. Tweaking the process as it currently stands will not fix the black holes. We need a more holistic model for policy design, delivery and management.

A cartoon of a policy team celebrating because they completed their policy, and handed over the policy instructions to a picture of a black hole, all the while wondering what it would be like to see it through. The policy instructions are caught by an implementation team who knows the policy design team has moved on, so do their best to interpret and implement their understanding. The impacts of the delivery are lost in a black hole as well, where the people affected by the policy can have their lives literally ruined, and eventually, an evaluation team asks “Why didn’t they just evaluate earlier?”.

There is also a significant gap with the public. From the start, there is usually a lack of diversity in expertise and experience in shaping a policy, and once an intervention is decided and rolled out, the people affected by policies have limited means to give feedback. Engaging the public early and often, and then providing clear feedback loops would help policies be better designed and improved over time.

Problem 2) The lack of real-time monitoring of intended AND unintended impacts

The laudable efforts to improve policy evaluation are great, but formal evaluations usually have two limitations that could be better addressed with other mechanisms. Firstly, formal evaluations often tend to be positivist, in that they look for “Has this initiative delivered what it said it would”, and aren’t often driven or set up to explore and understand unintended impacts, such as human or environmental patterns that emerged as a result of a new policy interacting in a complex domain.

Secondly, formal evaluations are usually a point-in-time assessment, rather than real-time monitoring of policy impacts. Evaluation teams are not connected to the day-to-day delivery of policy interventions, creating a timeliness challenge in mitigating issues that are identified. Evolving and improving policy evaluation methods will create greater understanding, but perhaps too little, too late for those affected in between. Real-time monitoring of intended and unintended impacts would nicely complement formal evaluation methods, while also providing a timely trigger if anything trended in the wrong direction.

Problem 3) A systemic inability to iterate policy in response to impact, feedback or change

Policies are often designed by a policy team, and then handed over to implementation, so that the policy team can move on to the next policy priority, creating a systemic inability to iterate policies as the real impacts are felt in delivery.

It doesn’t matter how collaborative or inclusive you are in designing a policy, there will always be perpetual change in the environment, and unintended impacts to mitigate. We need to take the lessons from the creation of “Continuous Integration and Continuous Delivery” (CI/CD) pipelines in service delivery, to create a “CI/CD Policy” approach which would manage policy design and delivery as part of the one continuum, drawing upon continuous feedback loops, monitoring and measurement of policy and human impacts to inform and iterate policies and the respective interventions.

This would not only help policies to maximise the realisation of policy intent in a rapidly changing world but would also provide the means to proactively identify and manage policy impacts (positive and negative) as they emerge.

Problem 4) Inconsistency in policy literacy and practice across the sector

Last, but not least, is the inconsistent definition, context and practice of “policy” across the sector, creating confusion and real issues of authority, decision making and accountability. Unfortunately today, many of the “policy guides” currently available limit themselves to Government Policy development, which has led to the common but dangerous assumption that Government Policies are the highest authority and that the peak of good public service is to simply advise the Government.

To my mind, there are three highest level and fundamental categories of “policy”:

1. Foundational Policies: the constitution, legislation and regulations that provide the context, framing and highest legal authorities and accountabilities of a department;
2. Government Policies: the directions of the Government of the day via the respective Ministers, which is subject to foundational policy limitations; and
3. Operational Policies: these cover all the operational, whole-of-government, department-defined rules and delegated policies, which are subject to both the government and foundational policy directions, but are the authoritative domain of secretaries.

The diagram below provides a useful reference on the hierarchy of authority of different policy types, as well as a guide to decision-making involved in each. This should help public servants realise that different actors are needed for change to different policy types and that even ministerial directions are constrained by the Foundational Policies above. It also should provide public servants more understanding as to what decision-making is actually within their delegated authority, such as operational policies.

This diagram shows five types of policies, starting at the highest authority with the Constitution, which is only changed by the people (public) via referenda, then Legislation (inc regulations) which is changed by the Parliament via Bills/Acts, followed by Gov Policy (Big P) which the Government changes via Ministerial directives, then Operational policies followed by the Department Secretary via departmental approvals, and finally internal operational decision making (implementation, program planning, delivery, etc) which department executives determine via internal delegations and processes.

Potential solutions

Solution #1: Adaptive policy management

So what might adaptive policy management look like? Well, let’s start with what the characteristics for delivering great policy and human outcomes might look like, and then we can reverse engineer an ideal policy operating model we could work towards.

Perhaps policymaking could be more of a team sport:

All teams involved in policy work together to co-design the policy intent, instructions, and success criteria and to pre-test some interventions. All teams work together to design, deliver and continuously improve all policy interventions, using shared policy tools, data, systems and methods, with impact monitored (intended and unintended) and evaluations triggered as required. CC-BY: Pia Andrews, 2023

Below is a high-level potential approach to the policy lifecycle, where policies are designed and delivered collaboratively, with shared policy infrastructure and real impacts monitored,  escalated and fed into policy improvements over time, with formal evaluations able to be triggered when things go terribly wrong, not years later. Policymakers could, for instance, establish a theory of change between the vision/outcomes and the actions being taken, to ensure the indicators and measures are connected to and represented in delivery from the start. If all policies required a purpose statement, it would help implementers to ensure the delivery was aligned with the purpose and intent of the policies.

A diagram of a policy journey, from defining purpose, then outreach, the definition of success, options, and trials to a decision point, followed by the establishment of interventions, then a cycle of test/design-deliver-management, which continues till close or policy change.

In this model, there are only two phases in the policy lifecycle:

• Policy purpose and authority — collaboratively developing the overarching policy purpose/intent, definition of success, and exploring options with a wide range of stakeholders, experts and those affected by the policy, including options testing, with a clear definition of the measurable change(s) that should result, and the problem or opportunity the policy is trying to address. This all leads to a decision point, which varies according to the policy type above.
• Policy interventions design & delivery — this includes the end-to-end co-design and management of all related policy interventions, including the program(s), services, grants, rules/legislation/regulation, or operational policy development. Policy interventions are continuously monitored individually and at a portfolio level for intended and unintended impacts, constantly improved and iterated based on feedback loops, and improvements are fed where relevant back into iterating overarching policies based on evidence and expertise.

Any form of policy could follow this model. Whether constitutional reform, legislation/regulation reform, advice/options to government, whole of government policies or operational policies, the intended outcome can be better realised through being a little more test-driven, participatory, multidisciplinary, iterative and through managing the whole policy lifecycle as an end to end approach with real-time and continuous improvements to interventions (like services, regulations, etc), while also continuously monitoring for policy impact that can feed into policy improvements.

Proposals for reforming how policy is done are often — understandably — met with concerns at “slowing things down”. But if you look at the full journey of policy today, policy intent realisation is already quite slow. If we had a more end-to-end and test-driven approach, we’d get better policies designed that are easier and faster to implement, which would dramatically shorten the time to realise policy intent, even if it means a little more time upfront.

Solution #2: A focus and expansion of policy professionalism in the APS

We need to not only teach what all types of “good” public policy look like but also create a culture of continuous learning and improvement for policy professionals. Perhaps we could start by complementing the excellent digital, data, HR and strategy professions coordinated by the APSC, with a “Policy Profession”?

But we also need to teach public service craft to all public servants, including what a healthy, politically neutral and evidence-based approach to public administration looks like, and why we aren’t achieving it as a norm across the sector. For instance, we need to have clear and consistent guidance on how to engage with Ministerial offices appropriately, so that everyone can maintain the integrity, dignity and trustworthiness expected of our public institutions. We also need clear guidance on how to promote an open APS that engages appropriately and regularly with the community, something which will hopefully be addressed in the APS Reform Agenda proposed Charter of Partnerships and Engagement.

All public servants should be confident to maintain real and long-term stewardship of public good, above and beyond day-to-day pressures or policy objectives, and also be knowledgeable of their foundational policy accountabilities, which are found in the constitution and relevant legislation and regulations. For instance, I have been surprised and somewhat horrified to hear people talk about how AI is a problem in government because it isn’t regulated, seemingly unaware that all government systems, regardless of the technology, are subject to Administrative Law, the Privacy Act, PGPA and many other foundational policies (leg/reg). We have many checks and balances we can use to ensure good governance, we just need to be aware of and apply them more consistently across the whole sector. For example, here is a paper where I documented the “special context of government” and then applied that special context to the use of AI in government. It resulted in a holistic approach that is complementary to the concept and practice of responsible government. When everyone has a shared and common understanding of the special context and responsibilities of public service, we have a good chance to get shared and high-integrity approaches to everything we design, deliver and administer in the public sector.

Solution #3: Shared and end-to-end “Policy Infrastructure”

Given how long this post has become, I’ll share more on this concept in a subsequent post, but here’s a teaser: Basically, whilst different teams have different tools, including distinct and separate interpretations of policy, we’ll continue to see an interpretation gap, and a lack of end to end policy visibility, which impedes end to end policy management.

The model above includes the following elements, aligned to the broad temporal phases of policy delivery:

To support test-driven policy ideation and announcements (pink):

• Public engagement tools to explore, co-design & test policy options, both initially (new policies) & ongoing (continuous improvement to policies and policy interventions).
• Linked and integrated admin data for research, policy modelling & patterns monitoring, best hosted by an independent, highly trusted entity, like the ABS.
• Case law and gazettes as a utility to use for analysis and to test new ideas.
• Publicly available modelling tools for testing and exploring policy change.

To support test-driven policy design, development & drafting (purple):

• Consistently applied the Human Impact Measurement Framework used across government, including for new policy proposals and for monitoring.
• Public repository to share policy tools, government models, measurement frameworks, synthetic population data, etc.

To support the Parliamentary publishing and visibility (aqua):

• A linked data representation of the administrative orders to automate reporting, accountability, auditing, security, and access to streamline MOGs.
• Publicly available Policy as code (intended outcomes, legislation, models, defined target group) available at legislation.gov.au
• Policy catalogue where all operational and Government policies can be discovered, along with measures and transparent reporting of progress.

To support policy implementation (green):

• A “Citizen’s ledger” to record all decisions with traceable explanations, for auditing & citizen access
• Policy test suite to validate the legality of system outputs in gov services & regulated entities.

To support policy compliance, iteration & improvement over time (yellow):

• Open Feedback loops for public and staff about policies & services, to drive continuous improvement and to identify and mitigate harm.
• Continuous monitoring of policy & human impacts, including dark patterns & quality of life indicators, alongside usual systems monitoring, to ensure adverse impacts are identified early and often.
• Escalation and policy iteration mechanisms to ensure issues detected are acted upon at portfolio and whole of gov levels.

What do you think?

What are the challenges you see, and what do you think needs to be done to improve policy management end to end? How might the APS Reform agenda help drive change, and how can we all do our part to improve things? How could we better deliver policy outcomes and better public and community outcomes? How can we close the gap between policy and delivery? Would love to hear your thoughts and examples!

I want to acknowledge all the people who are contributing to this work, especially the folks involved in the Intended and Unintended Impact of Social Policy research project which is worth keeping an eye on!

UPCOMING EVENT:

Pia Andrews will be speaking at The Mandarin’s upcoming conference: Rebuilding trust and integrity in the Australian Public Service, 22 February 2024, QT Canberra