Clarity
Vivid imagery
Historical perspective
Imagine this: you apply for a loan, and the banking system denies your request. The reason? «The machine learning model has identified a high risk.» What does that even mean? No one explains. You feel like you're standing before a judge who delivers a verdict but stays silent on the motives. This is the main problem with modern artificial intelligence – it can make decisions but can’t explain them in a way that makes us trust them.
The Illusion of Transparency: When More Information Means Less Understanding
Major tech companies have already realized the issue and are building «explainable AI» tools. SHAP, LIME, counterfactual analysis – these acronyms sound like spells from the world of data. But there's a catch: they show us how the algorithm works, but they don't explain why we should trust it.
It's like a doctor diagnosing you and showing you an X-ray, saying, «See these shadows? That's what determines your condition.» Technically, it's correct, but you still don't understand what's wrong with you. We confuse transparency with understanding – a classic mistake not only in IT but also in economics.
Remember how in 2008, banks «transparently» published highly complex reports on their mortgage products? All the numbers were out in the open, but the essence of the risk remained hidden. The result is known to all. With artificial intelligence, we risk repeating the same mistake, but on a much larger scale.
The Second Trap: One Explanation for All
Another misconception is the belief that one good explanation fits everyone. It’s like assuming you can explain where babies come from to both children and adults in the same way. A credit analyst with twenty years of experience and a first-time loan applicant need different types of explanations. The first needs statistical patterns; the second needs simple, real-life analogies.
But most AI systems work on a «one-size-fits-all» principle. As a result, experts get overly simplistic explanations and lose trust in the system, while regular users drown in technical details and also don't trust the algorithm. It's a paradox: the more we try to explain, the less we're trusted.
The Cognitive Bridge: How Psychology Explains Explanations
This is where psychology comes to the rescue. As early as the 2000s, American scientist Bertram Malle realized that people don't explain behavior chaotically but according to specific cognitive schemas. We have internal «templates» for understanding actions – and these templates can be used when creating explainable AI systems.
Malle identified five main ways we explain behavior: through knowledge and memories, situation modeling, pattern-finding, direct recall, and rationalization. But even more important is his fundamental distinction: we explain what we see (observable) and what is hidden from our eyes (unobservable) differently. We also perceive intentional actions and random events differently.
Imagine two people at a party. One accidentally drops a glass – we explain it as clumsiness or a slippery floor. The other deliberately throws a glass on the floor – here, we look for motives: anger, a desire to provoke, or a display of temper. The same thing happens with artificial intelligence. When a system makes a decision, we subconsciously decide: was it an «accident» or an «intention»? And depending on the answer, we expect different types of explanations.
Four Types of Behavior and Trust
By combining observability with intentionality, we get four categories of behavior:
Actions – what we see and what appears to be intentional. For example, a recommendation system shows you a specific movie. You see the result and assume the algorithm «wanted» to suggest exactly that. This requires explanations like «the system chose this movie because...»
Behaviors – visible but random events. The service gave a strange recommendation that doesn't fit your taste. This is perceived as a glitch, and the explanation should acknowledge uncertainty: «sometimes the system makes a mistake because of...»
Intended Thoughts – hidden but purposeful processes. The system analyzes your data to identify new patterns. We don't see the process itself, but we believe the algorithm «thinks» logically. The explanation focuses on the methodology: «the system uses this approach...»
Experiences – hidden and unpredictable processes. A generative model creates text or an image. The result is unique, the process is opaque, and the outcome is unpredictable. This requires explanations through analogies with human creativity: «like an artist drawing inspiration from...»
In practice, the two extremes are most common: actions and experiences. These are critically important for building trust. Actions require logical, cause-and-effect explanations. Experiences are best explained through examples, analogies, and context.
Credit Scoring: When AI Plays the Role of a Colleague
Let's take a specific example – a credit decision. A bank employee works with a system for evaluating borrowers. For them, the algorithm is a colleague, a more experienced analyst who helps make decisions. The employee perceives the system's work as an action: meaningful, intentional, and based on logic.
In this case, three types of explanations are effective:
Historical explanations: «Borrowers with similar characteristics have repaid their loans on time in 73% of cases.» This shows how past experience influences the current decision.
Causal explanations: «The main risk factors are a low income relative to the requested amount and an unstable credit history.» This answers the question, «Why this specific outcome?»
Conditional explanations: «By increasing the down payment to 30%, the risk drops to an acceptable level.» This shows what can be changed to get a positive decision.
And here's what does NOT work in this context: abstract charts of feature importance, statistical correlations without a link to real cases, and probabilistic assessments without explaining their meaning. Such explanations may be technically correct, but they don't align with how a credit analyst is used to thinking about risk.
Interestingly, the same system for the borrower themselves should work differently. The customer doesn't see the algorithm as a colleague – for them, it's more like «expertise.» This requires simpler explanations using analogies: «Your credit profile is similar to the profile of someone who...» or «Imagine the bank is a cautious friend who is lending money...»
Document Analysis: When AI Explores the Unknown
A completely different case is using large language models to analyze documents. A lawyer uploads a contract and asks the system to find potential risks. There is no single «correct» answer that can be checked in a textbook. The system analyzes the document just as a person would – through interpretation, comparison with experience, and searching for analogies.
This is the category of experience: the result is unpredictable, the process is creative, and the outcome depends on a multitude of factors. The user doesn't expect the system to provide a logical justification like «the system chose this risk because the importance coefficient is 0.84.» Instead, they need explanations through sources and context.
The most effective method here is direct recall. The system should show which specific parts of the document it relied on: «Clause 3.2 mentions a condition that could be interpreted as...» or «The wording in the warranty section is reminiscent of a case from 'Renault' vs. 'Lyon Credit Bank'...»
The key difference from credit scoring is that here, it’s not patterns that are important, but sources. The user must be able to «trace the path» from the system's conclusions back to the original text. This creates the feeling that the AI isn't just making things up but is truly «reading» the documents as carefully as an experienced lawyer would.
Personalizing Explanations: One Algorithm, Many Faces
The most interesting discovery of this approach is that the same algorithm can be perceived differently depending on the user's experience. A novice sees a credit scoring system as an experience – a mysterious process that they have to take on faith. An expert sees it as an action – a logical sequence that can be analyzed and critiqued.
This means that explanations must adapt not only to the task but also to the user's qualifications. A junior credit analyst needs educational explanations with examples: «Borrowers with less than a year of work experience are usually considered riskier because...» An experienced analyst needs only a heads-up on the statistical patterns: «Data from the last five years shows a 0.67 correlation between work experience and the probability of default.»
The technical implementation of this principle requires creating user profiles and adaptive interfaces. The system should «remember» what types of explanations a specific user prefers and gradually adjust its communication style. It's like a good teacher adapting to their audience: explaining complex concepts to first-year students using simple analogies, while giving graduate students scientific definitions straight away.
Trust as a Social Contract
But behind all the technical solutions lies a deeper problem: trust in artificial intelligence is a social, not a technical, phenomenon. We trust AI not because we understand the algorithms, but because the system behaves predictably and honestly.
Think about how you trust a doctor. You don't demand that the doctor explain the molecular mechanism of every prescribed medication, do you? Trust is built through professional behavior: the doctor asks the right questions, considers your medical history, explains risks in simple terms, and admits the limits of their knowledge.
The same goes for AI. Users begin to trust a system when it:
- Acknowledges uncertainty where it exists
- Shows the sources of its conclusions
- Explains the limitations of its capabilities
- Behaves consistently in similar situations
- «Remembers» previous interactions and learns from mistakes
Practical Recommendations: How to Build Explainable AI
This research leads to several concrete recommendations for creating systems that people truly trust:
For «action» tasks (credit scoring, medical diagnostics, risk assessment):
- Use historical examples and statistical patterns
- Show the key decision factors in order of importance
- Explain what needs to be changed to get a different result
- Avoid abstract correlations without practical meaning
For «experience» tasks (document analysis, content generation, data exploration):
- Always cite sources and grounds for conclusions
- Use analogies with human thought processes
- Acknowledge the boundaries of result reliability
- Provide alternative interpretations
For user adaptation:
- Create user profiles that account for their expertise
- Offer different levels of detail in explanations
- Remember communication style preferences
- Allow users to choose the format of explanations
The Future of Explainable AI: From Tech to Psychology
This research signals an important shift in understanding the problem of explainability. We're moving from the question, «How do we technically explain the algorithm's work?» to «How do we psychologically build trust between a human and a machine?»
The next stage of development is creating systems that automatically choose the style of explanation depending on the task and the user. Imagine an AI assistant that can switch between the roles of an «experienced colleague», a «careful researcher», and a «patient teacher» depending on the situation. Such systems won't just be technically accurate – they'll be socially intelligent.
But the most important takeaway is that explainability isn't a technological problem that can be solved with a new algorithm. It's a fundamental question about how humans and machines will collaborate in the future. And the answer lies not in code, but in understanding how the human mind works.
Money exists only because we believe in it – and trust in artificial intelligence works on the same principle. We don't trust the algorithms; we trust our understanding that these algorithms are doing what we expect them to do. And until we learn to shape that understanding, the most perfect technical explanations will remain useless.
