Assessing the Suitability of a Problem for AI: A Framework for High-Fidelity AI

Artificial Intelligence (AI) has transformed numerous industries by automating tasks that once required human cognition. However, not all problems are equally suited to AI implementation. The suitability of a problem for AI depends on its complexity, entropy, and ambiguity (CEA), the difficulty of data capture, and the feasibility of designing an objective function—a function that defines success for AI systems.

This essay proposes a methodology for assessing AI suitability, introducing the concept of High-Fidelity AI (HFAI)—AI that can consistently perform a task within a specified range of desirable outcomes. By comparing human competency and AI performance across a standard distribution, we illustrate how AI surpasses humans in structured, low-entropy environments but struggles in edge cases that require adaptability in ambiguous or unpredictable conditions.

The Problem Assessment Framework: Complexity, Entropy, and Ambiguity

To determine whether a problem is suitable for AI, we evaluate it based on three fundamental properties:

1.1
Complexity: The Amount of Structured Information to Process

• High complexity means a problem has many interdependent variables and a large solution space.
• AI excels in high-complexity, structured environments if the problem can be well defined mathematically.

Example: AI-powered drug discovery analyzes billions of molecular interactions, a task infeasible for human researchers at scale.

1.2
Entropy: The Degree of Unpredictability

• Entropy refers to randomness or uncertainty in a system.
• AI performs well in low-entropy environments where historical data can model future behavior.

Example: AI can accurately forecast warehouse inventory needs based on seasonal sales trends. However, it struggles to predict consumer behavior shifts due to unexpected global events (e.g., a pandemic).

1.3
Ambiguity: The Number of Possible Interpretations of Data

AI struggles when information can have multiple valid meanings, especially if context is critical.

Example: AI summarizing a legal contract is feasible because legal language is structured. However, AI interpreting poetic metaphor remains challenging due to its subjective nature.

Thus, problems best suited for AI are those with high complexity but low entropy and ambiguity.

The Role of Data Capture and Objective Functions

Even if a problem has the right CEA profile, AI implementation is limited by data availability and the ability to define an objective function.

2.1
Data Capture: The Challenge of Information Acquisition

AI requires high-quality, representative data. If data collection is difficult, incomplete, or biased, AI performance suffers.

• Easy Data Capture Example: AI analyzing customer purchase history to recommend products. Data is structured, extensive, and labeled.
• Difficult Data Capture Example: AI diagnosing rare medical conditions where only a few hundred cases exist globally. AI cannot learn without sufficient examples.

2.2
Objective Functions: Defining Success for AI

AI optimizes for an objective function—a mathematical rule that defines “success” for the system. If success is ambiguous or shifting, AI struggles.

• Well-Defined Objective Function: AI playing chess. Success is easily quantified as winning the game.
• Poorly-Defined Objective Function: AI writing a novel. What makes a novel “good”? Different audiences prefer different styles, and no singular metric defines literary quality.

High-Fidelity AI (HFAI): Defining AI Competency

High-Fidelity AI (HFAI) is AI that can perform a task consistently within a predefined range of desirable outcomes. Unlike general AI, HFAI does not aim for human-like cognition but instead optimizes for precision and reliability within a specific domain.

3.1
Defining High-Fidelity AI Performance

HFAI must meet the following criteria:

• Consistency – The AI produces stable, repeatable results.
• Defined Boundaries – The problem scope is well-constrained.
• Tolerance for Edge Cases – The AI is robust against minor variations but fails gracefully in extreme cases.

3.2
Examples of High-Fidelity AI vs. Non-HFAI

Task, High-Fidelity AI (HFAI) feasibility, reasoning:

• Automated Loan Approval
  Feasible — clearly defined risk factors, structured data.
• AI Writing a Book
  Not feasible — subjective quality, no singular success metric.
• AI Predicting Stock Market Trends
  Partially feasible — high complexity but also high entropy.
• AI Detecting Defective Products on an Assembly Line
  Feasible — low entropy, minimal ambiguity.

The key takeaway is that AI competency is not a universal metric—it is problem-specific.

AI vs. Human Competency: A Standard Distribution Model

Human and AI performance can be understood using a standard distribution of task complexity, entropy, and ambiguity.

This plot illustrates the standard distribution of AI vs. human competency across varying levels of task complexity, entropy, and ambiguity. AI excels in structured, low-entropy tasks but declines rapidly in ambiguous environments, whereas human competency spans a broader range, maintaining effectiveness even in high-ambiguity scenarios.

4.1
The Standard Distribution of AI Competency

1. Center of Distribution: Highly Structured, Predictable Tasks (Low Entropy, Low Ambiguity)
2. • AI significantly outperforms humans in tasks like chess, fraud detection, or medical image analysis.
   • These tasks have clear rules, large datasets, and objective success criteria.
3. Moving Toward the Edges: Increasing Entropy and Ambiguity
4. • AI loses competency faster than humans.
   • Humans have heuristic reasoning, allowing them to handle ambiguous or novel situations without prior experience.
   • Example: AI processing job applications may struggle with candidates who have unconventional career paths.
5. Extreme Edge Cases: Highly Ambiguous or Unpredictable Situations
6. • AI performs poorly, while humans maintain some level of competency.
   • Example: AI predicting long-term global political shifts struggles because no clear objective function exists.

4.2
Human Competency as a Counterbalance to AI Limitations

Humans can adapt in ways AI cannot due to:

• Intuition & Contextual Reasoning – Humans infer intent even when data is incomplete.
• Creativity & Novelty Handling – Humans invent solutions in ambiguous situations.
• Self-Correction & Learning from One Example – AI requires thousands of examples to learn a concept, while humans can generalize from a single experience.

Thus, while AI competency declines rapidly in high-entropy environments, humans retain functionality across a broader spectrum.

Conclusion: AI’s Role in High-Fidelity Problem Solving

Not all problems are suitable for AI, and the key to successful AI deployment is understanding where it excels and where it fails. The CEA framework (Complexity, Entropy, Ambiguity), along with considerations for data capture and objective functions, provides a methodology for assessing AI applicability.

By defining High-Fidelity AI, we recognize that AI is best utilized in constrained environments with well-defined success metrics. However, as we move toward higher entropy and ambiguity, AI competency drops off sharply compared to humans. Future advancements in AI must focus on:

1. Uncertainty-Aware AI – Systems that can detect when they are unsure and defer to human intervention.
2. Hybrid AI-Human Models – Leveraging human adaptability alongside AI efficiency.
3. Flexible Objective Functions – Creating AI that dynamically refines success criteria based on context.

By understanding and respecting the boundaries of AI competency, we can build systems that perform with high fidelity, reliability, and clarity, rather than forcing AI into domains where it is inherently ill-suited.