When should a decision remain open to reconsideration?
Your application has been denied.
Why?
You did not meet the criteria.
Which criteria?
The decision was made automatically.
I think some of the information is wrong.
The decision is final.
Maybe the decision was correct.
Maybe the criteria were reasonable.
Maybe every piece of information was accurate.
Maybe another review would produce exactly the same answer.
The troubling part is not necessarily the no.
It is that there appears to be nowhere left for the question to go.
Sometimes the Computer Should Decide
There are good reasons to automate decisions.
A computer does not get tired halfway through the day.
It does not have a bad morning.
It does not favor one applicant because their story was more persuasive.
It does not become impatient because the line is long.
It can process enormous numbers of cases according to the same criteria.
That consistency matters.
Imagine two people in identical circumstances receiving different answers because they happened to reach different employees.
Or one person receiving an exception because they knew whom to call while another person, in the same situation, did not.
Human discretion can correct unfairness.
It can also create it.
Rules can protect us from arbitrary judgment.
Automation can make those rules faster, more consistent, and easier to apply at scale.
In some circumstances, automated systems may eventually make decisions more accurately and more fairly than people do.
We should welcome that possibility.
The question is not whether computers should ever be allowed to say no.
Sometimes they should.
The harder question begins when the no may be wrong.
Consistency Is Not Infallibility
Consistency is one of automation’s greatest strengths.
But consistency tells us that the same process was followed.
It does not tell us that the process was right.
A good rule applied a million times can create enormous value.
A bad rule applied a million times can reproduce the same problem with extraordinary efficiency.
A system can be consistently right.
It can also be consistently wrong.
Automation may not have created the problem.
The problem may have existed in the rule long before anyone wrote the software.
But automation changes the scale.
A mistake made by one person may affect a handful of decisions.
A mistake built into a system can affect thousands or millions.
The better we become at automating decisions, the more important it becomes to recognize when the thing being automated deserves another look.
Before the Calculation Came the Judgment
Imagine a system that automatically approves applicants who score 700 or higher.
Someone scores 701.
Yes.
Someone scores 699.
No.
The computer applies the rule perfectly.
But somewhere before the calculation began, someone decided that 700 should be the threshold.
There may be excellent reasons for that number.
Evidence may support it.
Regulation may require it.
Years of experience may show that it works well.
The point is not that the threshold is arbitrary.
The point is that the precision of the calculation can make us forget that judgment existed before the calculation began.
Technology can execute a rule with extraordinary precision.
That does not make the rule a law of nature.
We need rules.
Without them, decisions can become unpredictable and arbitrary.
But rules are created for circumstances we can imagine.
Life eventually produces circumstances we did not.
What Might Be Wrong?
When someone questions a decision, they are not necessarily asking for a favor.
They may simply believe something important was missed.
Sometimes the information is wrong.
A payment was recorded incorrectly.
A record belongs to someone else.
A fact used in the decision is simply untrue.
Sometimes the information is accurate but incomplete.
The system knows what it was designed to know.
It simply does not know something relevant.
And sometimes the information is correct and the system works exactly as designed.
Yet the result raises a different question:
Does the rule itself need another look?
A challenge does not prove that the original decision was wrong.
But it can tell us where to look for the possibility of error.
In the information.
In what is missing.
Or in the system itself.
You Cannot Question What You Cannot Understand
Suppose someone receives:
Your application was denied.
Why?
You did not meet the criteria.
Which criteria?
We cannot tell you.
What exactly are they supposed to challenge?
People do not need the source code behind every automated decision.
They do not need to understand every calculation.
And there may be legitimate reasons some information cannot be disclosed.
Privacy matters.
Security matters.
Fraud prevention matters.
But when a consequential decision affects someone, perhaps they should understand enough about what happened to recognize whether something important may be wrong.
Explanation does not require complete transparency.
Sometimes it simply means giving someone enough information to understand the decision that affected them.
Asking the Same Question Twice
Imagine receiving a button:
Appeal Decision.
You click it.
The same information enters the same system.
The same rules are applied.
The same answer returns.
No.
Was the decision reconsidered?
Or did we simply ask the same question twice?
Meaningful review requires the possibility that something the first process missed can actually be considered.
Corrected information.
Missing context.
A different interpretation.
Independent review.
Perhaps even a problem in the rule itself.
The review must also be capable of affecting the outcome.
That does not mean it must change the outcome.
The answer may still be no.
The ability to question a decision is not the right to receive the answer we want.
It is the ability to say:
There is something relevant here that you may have missed.
Somewhere We Can Actually Reach
An appeal can exist on paper and still be practically unreachable.
Perhaps it requires hours of paperwork.
Technical expertise.
Multiple phone calls.
Knowledge of an obscure process.
A website someone cannot use.
Or more time than many people can afford to give.
Then everyone may technically have the same right to appeal while only some people can realistically exercise it.
That matters.
Meaningful recourse should not become a privilege reserved for people with the most money, knowledge, time, connections, or persistence.
A process does not have to be effortless.
But when a decision carries significant consequences, there should be a path for questioning it that ordinary people can realistically use.
A Human Is Not a Magic Ingredient
The obvious answer to automated decisions is often:
Put a human in the loop.
Sometimes that may be exactly right.
But a human can rubber-stamp the machine.
A human can be rushed.
A human can misunderstand the situation.
A human can introduce biases the automated process was designed to reduce.
Human beings make mistakes too.
So perhaps the enduring requirement is not that a human must always make the final decision.
It is that consequential decisions remain meaningfully reviewable.
Today that may involve another person.
Tomorrow it may involve an independent system, a specialized review process, a combination of people and technology, or something we have not yet invented.
The mechanism can change.
What matters is that when something important may have been missed, there remains somewhere for that information to go.
When One Challenge Becomes a Pattern
Recourse does more than protect the individual.
It can teach the system.
One person says the information was wrong.
That may reveal one mistake.
A thousand people say the same thing.
Now there may be a pattern.
One person encounters a circumstance the designers never anticipated.
That may be unusual.
Thousands encounter it.
Perhaps it is no longer unusual.
A challenge can correct a decision.
A pattern of challenges can correct a system.
And technology could help us do this better than before.
The same tools capable of processing millions of decisions may help organizations identify recurring errors, common reversals, unexpected patterns, and rules producing outcomes their designers never intended.
The technology that helps scale decision-making can also help scale accountability.
Responsibility Cannot Disappear Into the System
Many of us have heard some version of:
The system won’t let me.
Sometimes that is completely true.
The person we are speaking with may have no authority to change the outcome.
But then the question becomes:
Who does?
If an organization chooses a system, gives its decisions authority, and benefits from its efficiency, responsibility cannot disappear when something goes wrong.
A decision can be automated.
Responsibility cannot disappear with it.
And responsibility does not only mean finding someone to blame.
It means someone remains responsible for monitoring the system.
Correcting mistakes.
Looking for patterns.
Improving the process.
And deciding when the process itself needs to change.
Eventually, No Must Mean No
There must also be an end.
A person cannot appeal forever simply because they dislike the answer.
Institutions need to function.
Rules need authority.
Decisions sometimes need to become final.
Meaningful recourse does not mean infinite recourse.
It means there was a genuine opportunity to raise relevant information, correct meaningful errors, and have the decision appropriately reconsidered.
After that, the answer may still be no.
And the no may be entirely justified.
The goal is not to make authority powerless.
It is to make authority correctable.
The Confidence to Decide, the Humility to Reconsider
The computer may say no.
Sometimes it should.
Automation can make decisions faster, more consistently, and perhaps sometimes more fairly than we can.
We should not give that up simply because machines can make mistakes.
Human beings make mistakes too.
Perhaps the deeper question is not whether the decision came from a person or a machine.
It is whether the system making the decision recognizes the possibility that it may have missed something.
The information can be wrong.
Something important can be absent.
A rule can produce an outcome its designers never intended.
A trustworthy system does not need to promise that every challenge will change the answer.
It needs the confidence to decide.
And the humility to remain correctable.
That humility does not weaken authority.
It may be part of what makes authority worthy of trust.
Because the ability to question a decision is not the ability to demand a different answer.
It is the ability to bring forward something the decision may not have seen.
Sometimes nothing was missed.
The original decision should stand.
Sometimes one decision was wrong.
Sometimes thousands of individual challenges reveal that something larger is wrong.
No system will always know which one it is the first time.
Perhaps that is why a good decision-making system needs more than the ability to say yes or no.
It needs somewhere for doubt to go.
Somewhere a mistake can become visible.
Somewhere new information can be heard.
Somewhere a question can still reach the authority capable of answering it.
And sometimes all of that begins with one person saying:
Something here isn’t right.
Here is what you may have missed.
Please look again.