# In-Context PMD in Market Making: A Component Diagnosis

> Where does learned in-context policy mirror descent pay off in market making? An ICAIF 2026 workshop paper that swaps the learned actor and critic for their exact counterparts in an exactly solvable dealer market.

Web page: https://qrsntu.org/projects/policyattention/icaif/ (content last changed 2026-11-07)
Site index for agents: https://qrsntu.org/llms.txt

[← PolicyAttention](https://qrsntu.org/projects/policyattention/)

PolicyAttention · ICAIF 2026 workshop paper · Market making

A **component diagnosis** of the actor and the critic under fee, volatility, liquidity and toxicity stress

When an in-context market maker does badly after a regime shift, its returns cannot tell you which learned part failed. In a market we can solve exactly, we swap the learned actor and the learned critic, one at a time, for their exact counterparts. After 20 rounds the learned actor adds little; **the critic stage carries the excess normalized loss**, and in a follow-up an untrained average of the critic's own six samples loses about as much.

[How it works →](https://qrsntu.org/projects/policyattention/icaif/details/) [See the evidence](https://qrsntu.org/projects/policyattention/icaif/details/#evidence)

Authors: [Xinyue Wu](https://qrsntu.org/people/xinyue-wu/), [Yuhe Sui](https://qrsntu.org/yh/), [Shufang Chen](https://qrsntu.org/people/shufang-chen/), [Yingzhi Tang](https://qrsntu.org/yz/)

Poster 03 · ICAIF 2026 Workshop on AI-Driven Market Microstructure · Milan, 14 November 2026

Paired excess normalized loss over exact TD-PMD after 20 rounds on nominal markets (lower is better): the median of 320 paired values, 5 runs × 64 markets. The learned actor's 95% interval is [−0.0002, +0.0043].

## The paper at a glance

The idea

Policy mirror descent (PMD) improves a policy $π$ against action values $Q$ with a step $η$:

$$π_{k + 1} = softmax( log π_k + ηQ_k )$$

One frozen Transformer runs both halves in context: an **actor** that takes this step and a **critic** that estimates the next $Q$ from 6 sampled transitions per state–action pair. Because our dealer market is exactly solvable, each half has an exact counterpart, and a failure becomes a set of measured component costs.

Experiments

#### Synthetic market tested

Learned actor · +0.0020

Learned critic · +0.0219

Both learned · +0.0235

Excess normalized loss over exact TD-PMD on nominal markets. In a follow-up, an untrained average of the critic's six samples loses 0.70–1.10 times as much as the learned critic, across all 9 families.

#### Against desk rules

The loop beats HBEST, the best nominal rule, under severe fee (+0.1661 in value capture) and toxicity (+0.1247) stress, but loses to it on nominal, volatility and illiquidity markets, and to the order-flow rule HIMB under toxicity.

#### Real markets Not tested

Value capture on a 15-state synthetic market, not trading P&L.

## Open questions from the paper's limitations

1. ### Can a better critic target close the gap?
   In a follow-up, a critic that fitted its six-sample target perfectly would keep at least 70% of the cost; only one sample budget was tested.
2. ### Does the picture hold from a converged start?
   The adaptation test compares 20 rounds from a random policy with an old-regime oracle; no test starts from a converged nominal policy, and every test stops at 20 rounds.
3. ### Why the critic and not the actor?
   The split between the two is measured, not derived: no theory yet explains why the critic limits control.

[The questions in full →](https://qrsntu.org/projects/policyattention/icaif/details/#open)

## Paper

- ### Where Does Learned In-Context Policy Mirror Descent Pay Off in Market Making? A Component Diagnosis
  [Xinyue Wu](https://qrsntu.org/people/xinyue-wu/), [Yuhe Sui](https://qrsntu.org/yh/), [Shufang Chen](https://qrsntu.org/people/shufang-chen/), [Yingzhi Tang](https://qrsntu.org/yz/)
  ICAIF 2026 Workshop on AI-Driven Market Microstructure · 2026 · Accepted, Poster 03 [[Workshop listing]](https://icaif2026.org/workshop-papers/#paper-ai-market-microstructure-1245)

Paper and poster PDFs: not yet posted. This work is part of [PolicyAttention](https://qrsntu.org/projects/policyattention/).