Why hallucinations are unacceptable here
A general-purpose chatbot, when it does not know something, makes it up. This is acceptable for casual queries and intolerable for academic ones. A researcher who is told that a paper exists, by an authoritative-sounding source, and then cannot find that paper, has lost something more important than time — they have lost trust in the source.
The academic library is the institution’s commitment to citation. The AI surface attached to it has to honour that commitment or it does not belong in the library at all.
The bargain is simple. If the model has no source, the model has no answer. If the model has an answer, the model has the source.
The Athena design brief, p. 1Retrieval over generation
Every Athena answer is grounded in a retrieved set of source documents drawn from the institution’s own catalogue and licensed content. The generation step is constrained to summarise, paraphrase or quote that retrieved set. The model is not asked to recall facts from training; it is asked to read what was given to it and to say what is there.
The technical name for this is retrieval-augmented generation. The practical implication is that an answer Athena cannot ground is an answer Athena does not give. The system will say so. Patrons get used to this faster than expected.
Citation is a hard requirement
Every sentence Athena emits carries a citation pointing to the retrieved document that supports it. The citation is a working link, not a footnote-shaped decoration. The patron can open the source, read the surrounding paragraph, and decide whether the summary is fair.
This is not a UX flourish. It is the difference between a tool a librarian will recommend and a tool a librarian will quietly steer patrons away from.
What patrons actually ask
Six months into pilot deployments, the question distribution is more practical than we expected:
- “What does the library have on X?” — the most common query. Catalogue search reframed as a conversation. Athena returns a short summary plus the strongest 3–5 matches with citations.
- “How do I cite this?” — closing the loop on citation style is a surprisingly large share of usage. The model is good at format conversion, which it can do without inventing anything.
- “Is there a more recent version?” — checking for newer editions or follow-up papers. Retrieval-augmented systems handle this well; recall-from-training systems do not.
- “What does this database cover?” — meta-questions about scope. The answer is institutional, not encyclopaedic, and that is exactly where retrieval shines.
What we refused to ship
A few features the patron-experience team wanted but the editorial line refused:
- No “creative” mode. Generation without retrieval is not on offer. The model has one job here.
- No training on the patron’s queries. Patron interactions are not used to fine-tune the model. Privacy is not a slider in the settings panel.
- No autonomous follow-up. Athena answers the question that was asked. It does not chain into “and you might also be interested in” without the patron asking. Suggestion is restraint, not feature density.
What this costs the librarian
The honest answer: a few hours a quarter of curation. Athena performs better when the retrieval index reflects what the library actually wants to surface. Librarians flag documents to promote, demote or remove from the answerable set. The system gets better at the questions that matter to the institution because the institution told it what those are.
What it does not cost the librarian: their job. The librarian answers the hard questions Athena will not. Patrons reach the librarian sooner because the easy questions stop reaching them at all.
