Statuses, verdicts and thresholds
Everything below describes what scripts/scholarly.mjs computes. The comparisons are string
heuristics: they are documented so a reader can overrule them with the evidence each result
carries (comparison holds the claimed and the registered value of every field).
How a reference is checked
- Identifier first. A cited DOI is looked up at Crossref; if Crossref does not know it, at doi.org, which covers every registration agency (DataCite, mEDRA, ...). arXiv DOIs and arXiv ids go to the arXiv API.
- Comparison. The cited fields are compared with the record (table below).
- No identifier, or a DOI that does not resolve. Crossref is searched with the whole reference; each candidate is judged; arXiv is searched by title when Crossref has no confident candidate.
- Notices. Crossref update notices attached to the record are read for retractions, expressions of concern and corrections.
Field comparison
| Field | match |
Warning | mismatch |
|---|---|---|---|
| Title | similarity >= 0.9 | close: 0.75 to 0.9 |
below 0.75 |
| First author | same family name as the record's first author | order_differs: an author of the record, not the first |
not an author of the record |
| Authors (when several are cited) | all cited authors are in the record, same count | count_differs |
a cited author is absent from the record |
| Year | equal | off_by_one |
differs by 2 or more |
| Venue | equal, abbreviation, or acronym | differs |
never a mismatch |
| Volume, first page | equal | differs |
never a mismatch |
- Title similarity is the Sørensen-Dice coefficient on title words after lower-casing, removing accents, punctuation, markup and stop words, and a leading "RETRACTED:", "Erratum to:" or similar. A title that equals the other one plus a subtitle counts as 0.9.
- Family names are compared after removing accents; particles stay with the name ("van der Waals"); initials and noisy index entries ("Tim Bollerslev B") are tolerated.
- Venue tolerates abbreviations ("J. Financ. Econ.") and acronyms ("JFE"). Venue, volume and pages never produce a mismatch alone, because styles and registries disagree on them routinely.
- Year off by one is a warning because online-first and issue years differ legitimately.
Verdict: mismatch when any field mismatches, minor_differences when there are only
warnings, match otherwise.
Statuses of verify-doi
| Status | Condition |
|---|---|
invalid_doi |
The string is not a DOI (10.<4-9 digits>/<suffix>) |
not_found |
Neither Crossref nor doi.org knows the DOI |
doi_resolves_unchecked |
The DOI resolves, but neither a title nor both author and year were supplied |
metadata_mismatch |
The DOI resolves and the verdict is mismatch |
retracted |
The record carries a retraction, withdrawal or removal notice (wins over the others) |
verified |
The DOI resolves and the verdict is match or minor_differences |
Statuses of verify-bib
One per entry: verified, doi_missing_found, metadata_mismatch, doi_not_found,
doi_invalid, ambiguous, not_found, retracted, unverifiable, error. The meaning of
each and what to do is in the table of SKILL.md. Details that matter:
doi_not_foundwithsuggested_doi: the cited DOI does not exist, and a record with the same title plus author or year exists under another DOI. Typical of a mistyped or invented DOI on a real paper. Withoutsuggested_doi, neither the DOI nor the title was found.doi_missing_found: the entry had no DOI and exactly this rule held: title similarity >= 0.9 and the first author or the year agrees, with neither contradicting.alternativeslists other records that also satisfy it (a reprint, a preprint and its published version).ambiguous: at least one record has a similar title (>= 0.75) but fails the rule above.error: a registry timed out, rate limited or failed. The entry was not checked.
The report also gives duplicates (one DOI under several citation keys), parse_errors
(BibTeX syntax errors with line numbers) and needs_attention (keys of every entry that is
neither verified nor doi_missing_found).
Confident match for a search candidate
A candidate is accepted only when all of these hold:
- the title agrees: similarity >= 0.9, or, for a raw reference whose title could not be isolated, at least 90 % of the candidate's title words appear in the reference;
- the first author agrees or the year agrees (within one year);
- neither the first author nor the year contradicts;
- for titles of fewer than three significant words ("Introduction", "Editorial"), both the author and the year agree.
This is deliberately strict. It prefers ambiguous to a wrong match, because a wrong match
turns into a wrong DOI in someone's bibliography.
Known weaknesses
- Two different papers with the same title, first author and year (a conference paper and its
journal extension) cannot be told apart: both appear, one as
match, the other inalternatives. - Titles in non-Latin scripts fold to few or no comparable words; results for them are unreliable.
- A reference whose free text does not follow a common style may be split badly into fields. The whole string is still sent to the registry search, and the title-word rule above applies.
- Crossref metadata has errors of its own. A
metadata_mismatchmeans the citation and the registry disagree, not that the citation is the one in error.