Reliable and Adaptive Agentic AI

From Uncertainty-Aware Risk Control to Evidence-Gated Self-Healing

Presenter · Sina Tayebati

Chair & advisor · Prof. Amit Ranjan Trivedi

Committee ·

Prof. A. Enis Cetin
Assoc. Prof. Arman Roohi
Dr. Omesh Tickoo
Dr. Amanda Rios

ECE · University of Illinois Chicago

01 / 45

Autonomy changes the reliability problem

One answer

Predict

Is the answer correct?

Within a task

Act

Does the whole task succeed?

Across tasks

Adapt

Should this change persist?

Reliability must cover answers, trajectories, and the changes that persist.

04 / 45

Incomplete evidence is unavoidable

Visible evidenceStill uncertain

Physical observation

Sensed regions

Unobserved geometry

Model output

Available evidence

Correctness of an answer

Agent interaction

Visible messages and tools

Task state and coordination

Behavioral update

Tested cases

Effects on future behavior

A useful model acts with incomplete evidence. A reliable system must account for what remains unknown.

05 / 45

Evidence should govern commitment

Evidence
What has been observed?
Uncertainty
What could still be wrong?
Commitment
What action is justified?
Consequence
What happens if it fails?

\textcolor{#419777}{a^*}(\textcolor{#4288c6}{e})=\arg\min_{\textcolor{#419777}{a}}\;\mathbb{E}[\textcolor{#e13841}{L}(\textcolor{#419777}{a},Y)\mid\textcolor{#4288c6}{e}]

Given the evidence, choose the action with the lowest expected loss.

Observation: Infer missing structure
Decision: Choose a response
Trajectory: Monitor an interaction
Adaptation: Admit a correction

06 / 45

The object of reliability expands

As autonomy expands, the object of reliability assessment expands with it.

01

Observation

What can we infer from what we did not sense?

R-MAE / ICASSP 2025

02

Decision

How strongly should a model commit?

CAP / ACML 2025

03

Trajectory

When is an interaction going wrong?

TRACER / ICML 2026

04

Adaptation

Which changes deserve to persist?

HARNESS / AAAI · under review

07 / 45

Recover useful structure from partial observations.

08 / 45

Physical observation has a budget

Physical cost

LiDAR acquisition

Every emitted pulse has a physical cost.

Prior work

Masked pretraining

MAE · Point-MAE

Occupancy-MAE · MAELi

Research direction

Generative sensing

SensedInferred

Sensing power

P_{\mathrm{total}}=P_{\mathrm{laser}}+P_{\mathrm{scan}}+P_{\mathrm{signal}}+P_{\mathrm{control}}

Generative sensing: acquire fewer measurements, then reconstruct useful scene structure.

09 / 45

R-MAE learns from radial observations

Angular masking

Select radial scan regions

Range-aware sampling

Reduce distant observations

Occupancy prediction

Reconstruct unsensed structure

Original R-MAE Figure 1: angular masking and range-aware sampling, sparse 3D encoding, occupancy reconstruction and downstream object detection.

Self-supervision uses occupied and empty voxels.

\mathcal L_{\mathrm{occup}}=\operatorname{BCE}(\hat o,o)

10 / 45

Learned structure improves detection

Detector baselineWith R-MAE pretraining

The learned representation improves downstream detection across these settings.

Reported validation results. Waymo uses Level 2 mAP; nuScenes uses its benchmark mAP definition.

11 / 45

Sensing savings exceed reconstruction cost

Physical sensingReconstruction

≈9.1×lower estimated total energy

A small amount of observation
can support useful inference.

The next question is how strongly to trust a generated output.

Paper estimate per 360° scan, including Jetson Nano reconstruction overhead. Table V reports >90% masking.

12 / 45

Let uncertainty guide the decision.

13 / 45

A plausible completion still needs a decision

Recovered information becomes useful only when a system decides how much to commit.

Next reliability question: what action should uncertainty permit?

14 / 45

Calibration turns uncertainty into a prediction set

A held-out calibration set defines which candidate labels remain plausible.

1 · Calibrate a nonconformity threshold

\hat q_\alpha=Q_{\lceil(n+1)(1-\alpha)\rceil/n}\big(\{s_i\}_{i=1}^n\big)

Illustration of the calibration mechanism; dots are schematic.

2 · Include labels within that threshold

\Gamma_\alpha(x)=\{y:s(x,y)\leq\hat q_\alpha\}

ABC

Smaller risk budget → generally larger prediction sets

\Pr\{Y\in\Gamma_\alpha(X)\}\geq 1-\alpha

Marginal coverage · fixed α · exchangeable calibration and test examples

15 / 45

The risk budget should reflect the decision

Confidence signals→Conformal sets→Operational decisions

Prior foundation

One global risk budget

\alpha=\text{constant}

Input 1→Adaptive set

Input 2→Adaptive set

LAC / APS adapt the set, while the nominal risk level stays fixed.

CAP research question

Learn the cost of commitment

x\longmapsto\pi_\theta(\alpha,\beta\mid x)

Input 1→Answer

Input 2→Set or abstain

Optimize instance-dependent decisions for downstream utility.

The new control variable is the risk and abstention policy.

16 / 45

CAP learns when to answer, offer alternatives, or abstain

CAP learns how strongly to commit for each input.

Input

Model evidence

Base model stays fixed

Learned policy

Select risk

Lightweight MLP

(\alpha,\beta)\sim\pi_\theta(\cdot\mid x)

Conformal mechanism

Set thresholds

Held-out calibration scores

\hat q_{\mathrm{predict}}(\alpha)

\hat q_{\mathrm{abstain}}(\beta)

Decision

Choose a response

Three possible actions

The learned controls are the risk parameters; calibration turns them into decision thresholds.

Control structure shown. Exact policy features and action mapping are not fully specified in the paper.

17 / 45

The policy optimizes the cost of commitment

The preferred response depends on the costs of being wrong, being vague, and declining.

+

Penalties

Error

Penalize incorrect decisions

Set size

Penalize uninformative alternatives

Abstention

Account for the cost of deferral

−

Credits

Coverage

Reward retained correct outcomes

Exploration

Entropy encourages policy diversity

C=(1-\mathrm{acc})+\lambda_1\mathrm{avgSet}+\lambda_2\mathrm{abstention}\textcolor{#4288c6}{-\lambda_3\mathrm{coverage}-\lambda_4\mathrm{div}}

\nabla_\theta J\approx(R-b)\nabla_\theta\log\pi_\theta(\alpha,\beta\mid x)

b: moving-average reward baseline

Quantiles and discrete actions can be optimized through sampled utility.

18 / 45

Adaptive decisions need a precise coverage claim

Fixed-risk CP

Exchangeability supports a marginal guarantee for a fixed risk level.

\Pr\{Y\in\Gamma_\alpha(X)\}\geq1-\alpha

Adaptive policy

Choosing α from the input changes the guarantee question.

Fixed-risk marginal validity alone does not establish input-conditional coverage.

Held-out evidence

A fixed policy can be evaluated with a finite-sample coverage bound.

|\widehat{\mathrm{Cov}}_m-\mathrm{Cov}_{\pi}|\leq\sqrt{\frac{\log(2/\delta)}{2m}}\quad\text{with probability }\geq1-\delta

Fresh i.i.d. test data · frozen policy and calibration mechanism

Policy-calibrated coverage bounds estimation error. A 90% target remains an empirical claim to assess on held-out data.

19 / 45

Better uncertainty ranking supports selective answers

CAP improves model-average uncertainty ranking.

BenchCAP

AUROC ↑

Rank correct versus incorrect predictions

AUARC ↑

Retain accurate answers as rejection increases

Table 1 · reported five-task averages per model · Bench = the paper’s VLM-/LLM-Bench baseline · MCQA evaluation

Qwen-7B: 0.45 → 0.60 AUROC and 0.71 → 0.89 AUARC.

20 / 45

Adaptive filtering preserves more factual content

More content retained at a 90% factuality target.

Conformal FactualityCAPYi-34B · claim retention (%)

Table 5 · percentage points (pp) indicate absolute retention gains · target factuality is distinct from retention rate

The next unit of reliability is larger: an agent’s entire trajectory.

21 / 45

Recognize failure across an agent’s trajectory.

22 / 45

Confident steps can still produce a failed task

A sequence of plausible outputs can lose contact with the task.

From output risk to trajectory riskTrack behavior, tool feedback, and user coordination over time.

23 / 45

Uncertainty must account for interaction structure

The object of uncertainty expands from a generated answer to an evolving interaction.

Token confidence

How likely is this generation?Malinin & Gales, 2021 · Jiang et al., 2020

Fluent repetition can remain locally confident.

Semantic uncertainty

Do plausible answers disagree?Kuhn et al., 2023 · Farquhar et al., 2024

Agreement alone does not establish correct tool use.

Agent uncertainty
propagation

How does uncertainty evolve?SAUP: Zhao et al., 2025 · UProp: Duan et al., 2025

Which sparse episodes should dominate the final risk?

TRACERBehavioral evidence + coherence gaps + emphasis on critical episodes.

24 / 45

Observe uncertainty, repetition, and coherence

Measure uncertainty in the generation and in its relationship to the world.

U

Content uncertainty

Surprisal over selected content-bearing tokens

U_t=\frac{1}{|I_t|}\sum_{j\in I_t}-\log p_t(w_{t,j})

Dₐ

Repetition

Semantic similarity × lexical overlap in a local window

D_a(t)=\max_{t'\in W(t)}\operatorname{sim}_{sem}(u_t,u_{t'})\operatorname{sim}_{lex}(u_t,u_{t'})

Dᴬₒ

Action–observation gap

Does the tool response align with the action?

D_o^A(t)=1-\cos\!\left(\phi(x_t),\phi(o_t)\right)

Dᵁₒ

User coordination gap

Does the user response align with the preceding agent turn?

D_o^U(t)=1-\cos\!\left(\phi(x_{t-1}),\phi(x_t)\right)

Content filter: exclude stop words, numeric tokens, and highly predictable tokens. φ embeds the textual event or observation.

25 / 45

Let critical episodes determine trajectory risk

A brief failure signal should remain visible inside a long interaction.

Schematic step risks · synthetic values

Tail mean captures critical episodes; maximum preserves the strongest alarm.

1. Keep the strongest signal

r_t=\max\!\left\{U_t,\alpha D_a,\beta D_o^A,\gamma D_o^U\right\}

Actor masks select the applicable signals.

2. Select the high-risk tail

\mathrm{TM}_k=\frac1K\sum_{i=1}^{K}r_{(i)}

K = max(1, ⌊kN⌋); risks sorted descending.

3. Combine tail and maximum

\mathrm{TRACER}=(1-w)\mathrm{TM}_k+w\max_t r_t

26 / 45

A stable risk score, with conditional failure control

Stable to local perturbations

If every step changes by at most ε, the trajectory score changes by at most ε.

\big|\rho(r)-\rho(s)\big|\leq\lVert r-s\rVert_\infty

Monotone · positively homogeneous · subadditive

Failure-risk control is conditional.

Risk dominates hazardCritical risk is tail-sparse

P(B)\leq cK\,\mathbb{E}[\mathrm{TM}_k(r)]+c\eta

TRACER is a trajectory risk score.

The theorem does not turn an individual score into a calibrated probability of failure.

ρ acts on nonnegative step risks. B: breakdown event; c: hazard-dominance constant; η: risk outside the selected tail.

27 / 45

Failures become more separable across domains

τ²-bench · 3 models × airline, retail, telecom · highest score in all 9 settings

AUROC · failure ranking ↑

AUARC · selective accuracy ↑

Strongest baseline per row and metricTRACER

Baselines: normalized entropy, self-reported confidence, semantic entropy, SAUP. Table 2 values; no confidence intervals reported.

28 / 45

Warnings arrive early in failed trajectories

Gemini 2.5 Pro · failures detected by the first 20% of trajectory progress

Semantic EntropyTRACER

Retail · absolute gain

+31.1 pp

Earlier warnings leave more of the interaction available for corrective action.

This experiment measures detection timing; it does not evaluate successful intervention.

Failed tasks only · paper operating thresholds; a common false-positive rate is not reported. Progress uses final trajectory length.

29 / 45

Both sides of the interaction carry evidence

Gemini 2.5 Flash · Telecom · AUROC ↑

Agent and user signals complement each other

Table 5 actor ablations; combined score from Table 2.

MAX preserves the strongest local evidence

Table 6 · all variants tuned on validation episodes.

Detection creates an opportunity to adapt. What evidence should a correction need before it persists?

30 / 45

Require evidence before behavioral changes persist.

31 / 45

Learning from failure changes the next execution

Reflection

Can the agent repair this answer?

Persistent memory

What knowledge should it reuse?

Empirical selection

Which modifications improve performance?

My question: which self-authored changes should receive persistent authority?

Runtime triggering + protected-case non-regression + oversight of the accumulated rules.

32 / 45

A useful repair can also create a regression

Success on the motivating failure is insufficient evidence for persistent adoption.

55%

211 / 383 rejections

Among replay-decided retirements

A target-only retention rule would have admitted all 211 under the observed replay evidence.

Detected conflicts under sparse protection: ≤2 protected cases per validation round. This is not the incidence among all proposals.

33 / 45

The agent proposes; the runtime grants persistence

An external runtime surrounds an unmodified agent.

Weights stay fixed · rules evolve

c(R)=c_0\oplus\iota(R)

Standing protocol + compact rule index. Rule bodies enter context only when retrieved.

The agent cannot edit the evaluator, admission gate, replay corpus, or promotion record. Assumes a non-adversarial agent.

34 / 45

Detect a stall or regression in the trajectory

Judge the trajectory of progress, then diagnose the latest step.

Schematic trajectories; no experimental measurements are plotted. M₁: task completion and coherence, evaluated on every trace.

Stall

z_j\geq 10\quad\land\quad pk_j<0.5

A gain of ≥0.02 raises the peak and resets the counter.

Regression

m_j(x_i)<pk_j-0.15

Compare the current score with the best observed state.

Trigger → check tool / argument correctness → breach if corroborated; otherwise an advisory trend.

Only corroborated breaches are captured for replay. This is the Harness’s own detector; TRACER is a conceptual precursor.

35 / 45

Require target improvement and protected-case non-regression

Persistent authority requires external evidence.

Replay · matched evidence

Improve a failure. Preserve protected behavior.

≤3 matching failures≤2 protected cases

01

At least one triggering failure improves

\exists i:\;\Delta_{\hat j}(s_i)\geq 0.05

AND

02

Every measured metric on every replayed case

\forall i,j:\;\Delta_j(s_i)>-0.05

Any violation rejects the candidate, even when its target improves.

Δ = replay score − captured score.

Target priority: verifier outcome → rule metric → triggering metric.

Forward trial · weaker fallback

Observe the next 3 sessions

When replay is unavailable or remains inconclusive after ≤3 attempts.

Triggering failure stops or becomes less frequent

\hat p=0\;\lor\;\hat p\leq p_0-0.05

Without history, p₀ = 1.

Unmatched sessions.No protected-case test.

The candidate acts provisionally during evaluation. Promotion grants persistence.

Bounded empirical evidence, not a guarantee against unseen regressions. Protected cases are not ranked for relevance.

36 / 45

Evaluate reliability across 16 matched settings

16 matched comparisons

4 models

4 trials per task

100 turn budget

Benchmark / native split	Environment	Tasks	Models × arms
AppWorld · test_normal	Application and API tasks	168	4 × Baseline / Harness
Terminal-Bench · sample@2.0	Command-line tasks	10	4 × Baseline / Harness
τ²-bench · airline	Tool-using dialogue	50	4 × Baseline / Harness
τ²-bench · retail	Tool-using dialogue	114	4 × Baseline / Harness

Models: gpt-5.6-terra · gpt-5.6-luna · claude-haiku-4.5 · claude-sonnet-5

First-trial success

\mathrm{pass@1}=\frac{1}{|T|}\sum_{t\in T}X_{t,1}

All four trials succeed

\mathrm{pass}^{4}=\frac{1}{|T|}\sum_{t\in T}\prod_{i=1}^{4}X_{t,i}

Benchmark-native outcomes assess reliability; trace evaluations drive adaptation.

Same task order · seed 1 · continuous online adaptation · no held-out transfer test. Results use matched completed task-trials.

37 / 45

Completion improves; most effects remain uncertain

16/16 score deltas positive

2/16 95% CIs exclude zero

12/16 pass⁴ improves; 4 ties

95% percentile bootstrap CIs · 10,000 task-level resamples · score on [0, 1]. Green intervals exclude zero.

Consistent direction across settings; 14 of 16 effects remain statistically unresolved.

38 / 45

Individually valid rules can accumulate into drift

AppWorld / luna: an early advantage becomes a late deficit as admitted rules accumulate.

13 → 70mean active rules, Q1 → Q4

1.50× → 2.52×input tokens / turn vs Baseline

Slope −0.27 × 10⁻³ per task
Permutation p = .022

Individual admission does not establish reliability of the accumulated rule set.

Coverage

Guard replays captured breaches, not the full task stream.

Granularity

Per-case drops ≥.05 can miss smaller aggregate shifts.

Live context

Replay may not reproduce retrieval dilution.

39 / 45

The scope of reliability follows the scope of evidence

External evidence makes behavioral adaptation inspectable and reversible.

Object

What the mechanism checks

What remains outside that check

Candidate

Target gain + non-regression on sampled replay cases

Unseen cases, noisy judgments, provisional execution

Rule corpus

Re-test accumulated rules on the captured evaluation corpus

Uncaptured behavior, small aggregate drift, retrieval effects

Deployment

Audit trail and reversible text rules; fixed model weights

Adversarial circumvention and unmeasured properties

Key causal comparison still open

Gated vs immediate admission, with the same downstream corpus guard.

Generalization and practical cost

Freeze rules and test held-out transfer; reduce 1.7–7× latency.

Reliable autonomy must govern both execution and the authority to change future behavior.

40 / 45

Reliability at four scales

Reliability must be assessed at the scale where a failure changes the outcome.

01

Observation

Recover useful structure from partial physical observations.

R-MAE / ICASSP 2025

02

Decision

Learn how uncertainty changes the response policy.

CAP / ACML 2025

03

Trajectory

Surface sparse critical episodes across an interaction.

TRACER / ICML 2026

04

Adaptation

Require external evidence before retaining a behavioral rule.

HARNESS / AAAI · under review

The final step extends oversight to how experience changes future behavior.

41 / 45

Open questions for reliable adaptation

Adaptive guarantees

How can a learned risk policy retain coverage under changing conditions?

Intervention

When does an early warning translate into a successful recovery?

Persistent adaptation

Which evidence predicts transfer without accumulated regression?

Reliable autonomy requires evidence for its next commitment.

42 / 45

First author papers

Tayebati, Sina, Theja Tulabandhula, and Amit R. Trivedi.
Sense less, generate more: Pre-training LiDAR perception with masked autoencoders for ultra-efficient 3D sensing.
arXiv preprint arXiv:2406.07833 (2024).
Tayebati, Sina, et al.
CAP: Conformalized abstention policies for context-adaptive risk management for LLMs and VLMs.
The 17th Asian Conference on Machine Learning (Conference Track). 2025.
Tayebati, Sina, et al.
TRACER: Trajectory risk aggregation for critical episodes in agentic reasoning.
International Conference on Machine Learning (ICML). PMLR 306, 2026.
Tayebati, Sina, et al.
Self-Healing Harness for Runtime Oversight of Agent Self-Modification.
Under review at AAAI.

43 / 45

Collaborative papers

44 / 45

Thank you.

Sina Tayebati

stayeb3@uic.edu

45 / 45

Reliable and Adaptive Agentic AI

Physical observation

Model output

Agent interaction

Behavioral update

Evidence

Uncertainty

Commitment

Consequence

Observation

Decision

Trajectory

Adaptation

Angular masking

Range-aware sampling

Occupancy prediction

One global risk budget

Learn the cost of commitment

Error

Set size

Abstention

Coverage

Exploration

Fixed-risk CP

Adaptive policy

Held-out evidence

AUROC ↑

AUARC ↑

Token confidence

Semantic uncertainty

Agent uncertaintypropagation

Content uncertainty

Repetition

Action–observation gap

User coordination gap

1. Keep the strongest signal

2. Select the high-risk tail

3. Combine tail and maximum

Stable to local perturbations

AUROC · failure ranking ↑

AUARC · selective accuracy ↑

Agent and user signals complement each other

MAX preserves the strongest local evidence

Reflection

Persistent memory

Empirical selection

Coverage

Granularity

Live context

Key causal comparison still open

Generalization and practical cost

Observation

Decision

Trajectory

Adaptation

2023–2025

2025–2026

2026

Thank you.

Agent uncertainty
propagation