ADM systems can increase the speed and efficiency of decision-making processes, but they can also reduce accountability and transparency around how decisions have been made, and how the law has been applied. There is also a risk that ADM systems are assumed to be free from error, even where it is not possible to check how the system has reached a particular decision.  

Many of the decisions made every day by government departments, and which affect all of us in some way, involve making judgment calls about whether those decisions comply with the law, and whether they are fair and reasonable. ADM systems are not always designed to be able to exercise this kind of judgment, and where that is the case, their decisions may not always be correct or fair.  

What the Rule of Law Report found  

It is not always possible for humans to see how an ADM system has reached a decision, and whether that decision is correct. One of the main reasons for this is that some types of ADM systems use algorithms which work in mysterious ways: we are able to see their inputs and outputs, but we cannot see how one becomes the other.  

These are called ‘black boxes’ – and they can be problematic when they are used by Government departments. They make it impossible to understand:  

• how and why a decision has been made;  
• whether those decisions followed the law;  
• what other factors influenced the decisions.  

This makes it harder for individuals to understand whether decisions which impact them are in fact fair and lawful, and to then challenge those decisions.  

What can be done to address these challenges to the rule of law   

It is important to have clear rules and best practices for the safe, ethical and responsible use of ADM systems, and our report recommends current and future governments do further work in this space.  

It is also important to avoid using black-box ADM systems within Government departments, especially when making a decision that could have a significant impact on someone (for example, decisions relating to penalties, benefit entitlements, and visas).  

Government departments must also have systems and processes in place to ensure:  

• there is human oversight of decisions made by ADM systems;  
• anyone who is affected by a decision made by an ADM system has the ability to ask a human to review or reconsider that decision.