Why two directly-relevant Index2 docs were missing from a title-check report

Session: a8f35ef1-6600-4fbf-9b9d-12336501bff8

Property: Gat 283, Manjari Khurd (मांजरी खुर्द), Haveli, Pune  ·  Report mode: Buyer report, target khata 1153

Investigated: 2026-07-19, read-only against production via cloud-sql-proxy. Code paths verified against current tree.

TL;DR — not a scraping or relevance failure. Index2 scraping and the relevance classifier both worked: 2 documents were correctly tagged directly_relevant and are still stored on the session. They were dropped afterwards by the buyer-report khata-scoping filter (trace_khata_chain), which keeps an Index2 doc only if it's an encumbrance or one of its parties belongs to the buyer's specific ownership chain (khata 1153). Both docs involve other parties on the same Gat, so the report ended up showing 0 — while its own coverage line still lists 2 document-years, making the wording self-contradictory.

Severity of the underlying issues: medium — misleading report wording medium — over-aggressive filter on a broken chain

What the report showed vs. what the data holds

SignalIn the delivered reportIn stored session data
Directly-relevant Index2 docsdocs_examined: 0, docs_direct: 0, empty transaction historyindex2_relevance.directly_relevant = 2
Years with documentsyears_with_docs: [2024, 2026]coverage.years_with_docs: [2024, 2026]
Limitations text"37 documents found, of which 0 pertained directly to this property"2 pertained directly to the property (Gat 283)
judge_relevance steprelevant: 2, discarded: 4, 1 attempt, no re-run

The two dropped documents

Reg. noTypePartiesGat
4807/2024Lease (Leave & Licence)Waghamare Jaidev → Prakash Om283
4334/2026Agreement to sell (साठेखत), ₹2 croreशिवाजी आबाजी थोरात +6 → सुहास अंकुश उंद्रे283

Root cause — the chain of "why"

1. Relevance judge worked

The judge_relevance step classified relevant: 2, discarded: 4 in a single attempt. Both docs survive today in property_data.index2_relevance.directly_relevant.

2. Timeline assembly worked

The stored timeline.index2_timeline contains both docs, and coverage.years_with_docs = [2024, 2026]. Nothing lost yet.

3. Report generation re-scoped the timeline to the buyer's khata

This is a buyer report (target_khata = '1153'). Before generating, the report step runs trace_khata_chain(timeline, '1153') (steps.py:3556-3559). That function keeps an Index2 doc only if it is an encumbrance type or one of its parties fuzzy-matches a name in khata 1153's traced ownership chain (khata_chain.py:264-268).

4. Neither doc qualified → index2_timeline = []

Re-running the trace on the live data, khata 1153's chain parties are:

गणपती म्हपाजी खांदवे · मारुती राघु सावंत · सोपाना राघु सावंत · मे हरिश्चंद्र ॲग्रिकल्चरल ॲन्ड हॅचरिज प्रा लि कंपनी

Result: transaction_history = [], docs_examined = 0. The docs are directly relevant to the property (Gat 283) but not to the buyer's khata (1153), and buyer mode scopes to the khata.

Two issues worth fixing

medium  The report wording is self-contradictory

trace_khata_chain preserves coverage but empties index2_timeline. So the report simultaneously says years_with_docs: [2024, 2026] and docs_examined: 0 / "0 pertained directly to this property." The "0" is post-khata-filter; 2 docs did pertain to the property. Buyer mode should distinguish property-level vs khata-level counts in both the meta and the limitations text.

medium  The khata filter ran on a chain built from a known-bad mutation

This session's own critical check flags that mutation 1165's digital record is wrong — it links a 1959 sale to गणपती म्हपाजी खांदवे instead of the real 1984 sale to the current owner. The chain trace walks through mutation 1165, so chain_parties are unreliable predecessors and there are 2 untraced_ancestors. The party-matching that discarded both docs was therefore working off a broken chain. A recent ₹2-crore साठेखत on the same Gat is exactly the kind of transaction a buyer would want surfaced — even if it belongs to another khata. Consider a fallback "other transactions on this Gat" section when the chain trace is incomplete, rather than silently dropping directly-relevant docs.

Key code references

FileWhat it does
title_checker/pipeline/steps.py:3556-3559Buyer mode calls trace_khata_chain before report generation
title_checker/pipeline/khata_chain.py:264-268Index2 filter: keep only encumbrances or chain-party docs
title_checker/pipeline/khata_chain.py:346-356_doc_involves_chain_party — fuzzy name match (threshold 70)
title_checker/pipeline/report.py:2680, 2895-2900docs_examined / transaction_history derived from the (now empty) index2_timeline