> ## Documentation Index
> Fetch the complete documentation index at: https://docs.axiomancer.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Entity resolution

> How Axiom Overwatch dedupes and links company records with probabilistic Splink models and routes candidate pairs through analyst adjudication.

Axiom Overwatch resolves real-world entities — companies, vessel owners, sanctioned parties, permit applicants — across noisy data sources by running probabilistic record-linkage with [Splink](https://moj-analytical-services.github.io/splink/). High-probability candidate pairs are persisted to `entity_resolution_pairs` and an analyst either confirms, rejects, or marks them as unsure through the adjudication API before any downstream consumer treats them as the same entity.

This is the foundation for investigation case-file consolidation, sanctions screening across naming variants, and UBO chain stitching where a single owner appears under spelling, jurisdiction, or registration-number differences in upstream feeds.

## When to use it

Reach for entity resolution when you need to answer "is this the same company / owner / party as that one?" across heterogeneous records — not "does this string match that string." Typical use cases:

* Collapsing duplicate companies across LEI, OpenSanctions, Equasis, and registry feeds before an [investigation case file](/overwatch/api/investigations) is built
* Stitching ownership chains where one party appears under multiple names, transliterations, or registration numbers
* De-duplicating permit applicants before [sanctions](/overwatch/api/sanctions) or [UBO](/overwatch/api/ubo) screening
* Producing an adjudication worklist for analyst review when automated matching is not safe to act on alone

If you only need a deterministic join on a known key — IMO, MMSI, LEI, `record_id` — use the standard [Codex join keys](/codex/join-keys) instead. Probabilistic resolution is for the cases where deterministic keys are missing, inconsistent, or contested.

## How resolution works

For each input table, Splink streams every candidate pair through a settings-driven scorer: blocking rules narrow the candidate space, comparison vectors score the surviving pairs, and an unsupervised expectation-maximisation training step learns the per-field match weights without labelled examples.

<Steps>
  <Step title="Snapshot input">
    The linker reads from a Parquet snapshot of the source table, never from live Supabase. This keeps training reproducible, lets a model version be re-trained against a fixed input, and prevents long-running linkage jobs from contending with operational reads.
  </Step>

  <Step title="Block candidate pairs">
    Blocking rules narrow the `O(n²)` pair space to a tractable candidate set. The companies linker blocks on a 4-character name prefix, jurisdiction plus first letter of the name, and shared non-null LEI code.
  </Step>

  <Step title="Score with comparison vectors">
    Each surviving pair is scored across the comparison vector — name (Jaro–Winkler at 0.9 and 0.7), jurisdiction (exact match with term-frequency adjustment), registration number (exact match), and legal-form code (Levenshtein at distance 1 and 3 for companies).
  </Step>

  <Step title="Estimate parameters">
    Unsupervised EM training estimates `m` and `u` probabilities per comparison level from the data itself. The two-random-records baseline is anchored on deterministic rules (LEI match, registration-number match, jurisdiction-plus-uppercase-name match) at `recall = 0.8`, and `u` is estimated from one million random pair samples.
  </Step>

  <Step title="Persist candidate pairs">
    Predictions with `match_probability >= 0.95` are upserted to `entity_resolution_pairs` in batches of 500, keyed on `(model_version, id_l, id_r)`. Below `0.95` is discarded — the table is a high-confidence worklist, not a comprehensive scoreboard.
  </Step>

  <Step title="Adjudicate">
    Each persisted pair starts at `adjudication_status = 'pending'`. An analyst confirms (`match`), rejects (`not_match`), or defers (`unsure`) through the adjudication API. Only pairs marked `match` are safe for downstream consumers — investigation case-file merge, dedup on Aleph ingest, UBO chain stitching — to treat as the same entity.
  </Step>
</Steps>

## The companies linker

The first linker covers the `companies` table and runs in Splink's `dedupe_only` mode — same source on both sides of the pair. Address deduplication is intentionally out of scope; address normalization is handled separately by libpostal upstream.

| Setting               | Value                       |
| --------------------- | --------------------------- |
| Link type             | `dedupe_only`               |
| Unique ID column      | `id`                        |
| Persistence threshold | `match_probability >= 0.95` |
| Batch size            | 500 rows per upsert         |
| Default model version | `splink_companies_v0.1`     |

Cross-table `link_only` jobs (`gleif` × `opensanctions` × `equasis`) are a follow-up — the scaffolding supports them, but no production linker for those targets has shipped yet.

### Configuring a run

The linker is exposed from `packages/python-services/splink` and is invoked from a Python worker, not an Edge Function. Pass the path to a Parquet snapshot of the input table:

```python theme={null}
from supabase import create_client
from companies_linker import (
    train_companies_linker,
    predict_and_persist_companies,
)

linker = train_companies_linker("/path/to/companies.parquet")

supabase = create_client(SUPABASE_URL, SUPABASE_SERVICE_ROLE_KEY)
count = predict_and_persist_companies(linker, supabase)
print(f"persisted {count} candidate pairs")
```

To override the model version or threshold for an experiment, pass them through:

```python theme={null}
count = predict_and_persist_companies(
    linker,
    supabase,
    model_version="splink_companies_v0.2",
    threshold=0.97,
)
```

The default `splink_<table>_v<major>.<minor>` versioning convention is enforced by the upsert key `(model_version, id_l, id_r)` — a new version produces a fresh set of candidate pairs alongside the old ones rather than overwriting them, so analysts can compare two versions against the same input.

When to bump versions:

* **Minor** — thresholds, blocking rules, or comparison settings change but the entity target is the same.
* **Major** — the input table contract, link type, or downstream semantics change.

## The `entity_resolution_pairs` table

Every persisted candidate pair lands in `public.entity_resolution_pairs` with the standard APRS envelope (`record_id`, `schema_version`, `normalization_version`, `acl_tier`, `source_system`, `ingested_at`, `modified_at`, `chunk_id`) plus resolution-specific columns:

| Column                             | Meaning                                                                                                                               |
| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| `model_version`                    | `splink_<table>_v<major>.<minor>` — identifies which trained model produced the pair                                                  |
| `source_table_l`, `source_table_r` | Source tables for the left and right side of the pair. Same value for `dedupe_only` runs; different for cross-table `link_only` runs. |
| `id_l`, `id_r`                     | Local IDs from each source table                                                                                                      |
| `match_probability`                | Splink-derived probability in `[0, 1]`; only pairs at or above the persistence threshold are stored                                   |
| `match_weight`                     | Log-odds match weight from Splink                                                                                                     |
| `blocking_rule`                    | The blocking rule that surfaced the pair                                                                                              |
| `adjudication_status`              | `pending` / `match` / `not_match` / `unsure`                                                                                          |
| `adjudicated_by`                   | UUID of the analyst who decided; `NULL` until decided                                                                                 |
| `adjudicated_at`                   | Timestamp of the decision; `NULL` until decided                                                                                       |

The table enforces a unique constraint on `(model_version, id_l, id_r)` so re-running the same model against the same input is idempotent — re-runs update existing rows in place rather than inserting duplicates. Indexes on `adjudication_status`, `match_probability DESC`, `(source_table_l, id_l)`, and `(source_table_r, id_r)` back the common worklist and reverse-lookup queries.

Row-level security is configured for service-role writes and authenticated reads — only the Splink workers (and other service-role callers) can persist or update candidate pairs, but any authenticated analyst can read them.

## Adjudication API

The adjudication API exposes two endpoints under the `entity-resolution` tag for analyst tooling — an inbox endpoint and a decision endpoint.

### `GET` `/pending`

Returns pending pairs ordered by descending match probability, so the strongest candidates are reviewed first.

**Parameters**

| Name    | Type   | Required | Description                                            |
| ------- | ------ | -------- | ------------------------------------------------------ |
| `limit` | number |          | Number of pairs to return. Default 50, min 1, max 500. |

**Response**

```json theme={null}
{
  "pairs": [
    {
      "id": 1842,
      "model_version": "splink_companies_v0.1",
      "source_table_l": "companies",
      "source_table_r": "companies",
      "id_l": "co_018af1c2",
      "id_r": "co_0aa39e44",
      "match_probability": 0.987,
      "match_weight": 6.12,
      "blocking_rule": "l.lei_code = r.lei_code and l.lei_code is not null",
      "adjudication_status": "pending",
      "ingested_at": "2026-05-19T09:00:00Z"
    }
  ]
}
```

### `POST` `/decide`

Records an analyst's decision on a pair.

**Body**

| Field            | Type    | Required | Description                                            |
| ---------------- | ------- | -------- | ------------------------------------------------------ |
| `pair_id`        | integer | ✓        | Primary key of the `entity_resolution_pairs` row       |
| `decision`       | string  | ✓        | One of `match`, `not_match`, `unsure`                  |
| `adjudicated_by` | UUID    |          | Analyst identifier; optional but recommended for audit |

```bash theme={null}
curl -X POST "$OVERWATCH_API_URL/entity-resolution/decide" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "pair_id": 1842,
    "decision": "match",
    "adjudicated_by": "00000000-0000-0000-0000-000000000001"
  }'
```

The endpoint updates `adjudication_status`, stamps `adjudicated_at` and `modified_at` to the current UTC time, and optionally records `adjudicated_by`. A `404` is returned when no row matches `pair_id`.

## Gold-set evaluation

Before promoting a model version — that is, before any downstream consumer is allowed to treat its `match` decisions as authoritative — every model goes through a 200-row gold-set evaluation:

1. Sample 100 pairs from the high-confidence band (`match_probability >= 0.95`), 50 from the review band (`0.80 <= match_probability < 0.95`), and 50 hard negatives below `0.80`.
2. Have an analyst label each pair as `match`, `not_match`, or `unsure` through the adjudication API.
3. Compute precision, recall proxy, false-positive themes, and threshold sensitivity against the 200 labels.
4. Record the gold-set date, model version, input snapshot URI, and metrics before enabling downstream consumers.

A model that hasn't passed gold-set evaluation can still produce candidate pairs and feed the adjudication queue, but its decisions are not yet wired into investigation case-file merge or sanctions screening.

## What the linker doesn't do

* **No live-table linkage.** All inputs are read from Parquet snapshots. Training against operational Supabase tables is explicitly disallowed.
* **No address deduplication.** Addresses are normalized upstream by libpostal; the companies linker doesn't include any address comparison.
* **No automatic merge.** Persisted pairs are candidates, not decisions. Nothing downstream treats two records as the same entity until an analyst sets `adjudication_status = 'match'`.
* **No cross-table link jobs yet.** The scaffolding supports `link_only` mode, but no production cross-table linker (`gleif` × `opensanctions` × `equasis`) has shipped.
* **No supervised training.** The linker relies on unsupervised EM and deterministic anchor rules; no labelled training set is required to produce candidate pairs.

## Related references

* [Investigations API](/overwatch/api/investigations) — case-file consolidation that consumes confirmed matches
* [Sanctions API](/overwatch/api/sanctions) — sanctions screening across name and jurisdiction variants
* [UBO API](/overwatch/api/ubo) — beneficial-ownership chains that benefit from cross-source deduplication
* [Cross-dataset join keys](/codex/join-keys) — deterministic keys to prefer over probabilistic resolution when available