I spent a while getting a three-layer document set into a searchable store,
and six things surfaced that had nothing to do with retrieval. Posting them
in case they save someone an afternoon.
The corpus: a national labor agreement, a joint interpretive manual
maintained by both parties to it, and a signed local agreement covering one
facility. They operate as layers rather than replacements; current
applicability is tracked separately. Most of what follows is not a defect
in the documents themselves. It is what happens when source structure,
extraction, and provenance meet an ingest pipeline.
1. The manual does not reliably quote the contract it explains. Boxed
blocks look like the agreement's own words. In one subsection they are not
— same people, different wording. Do not promote a manual quotation to
source text without opening the source, and keep a running list of the
diffs instead of fixing one passage. The inverse bites too: if the signed
instrument itself has a typo, normalizing it at ingest gives you cleaner
text that is no longer what the parties signed.
2. A margin label is not the edge of a provision. One page boxes two
subsections and labels only the first. Cut on the visible label and the
second subsection's content lands under the first one's number, with nobody
having misread anything.
3. A gloss will cite the clause it widens. The manual states a broader rule
than the subsection it points at, and the citation invites you to attribute
the broader version to the source. An earlier version of my own manifest
took the invitation.
4. Typesetting can hide a heading that is on the page. This one passed on
my contract PDFs, and I am including it because when it fails it fails
silently. It did fail on two unrelated PDFs in the same project:
letter-spaced headings extracted one character per token, and small-caps
names split mid-word, so searches for terms that were plainly there
returned nothing.
5. One scope term, three meanings. The local agreement uses a single phrase
for the whole facility, for a sub-unit, and for one of five named sections,
depending on the item. As a field on the chunk it is wrong somewhere.
6. The table of contents is not a map of the rules. A rule about
displacement from a temporarily held assignment sits under a heading about
something else entirely, and uses an older name for a job category (a
former name for a reserve carrier). Read the file. Keep a synonym table;
instruments outlive the names they were written with.
Four of those are things I hit. One is a check that passed. One is a
property of the instrument I have not measured against my own pipeline.
They are what building surfaced, not what an audit would find, so do not
expect the same six.
If you keep something with a national layer and a local layer on top of it
— statute and agency guidance, guideline and protocol, district policy and
school procedure — I would be glad to compare notes. The shape is not
specific to labor contracts.
Tim Hickey
Letter carrier, USPS. Formerly academic librarian; MLIS and CAS in Digital
Libraries, Syracuse.
Received on Sun Sep 06 2026 - 23:01:59 EDT