SPB Git

Repository Public Research preview

statuses.md 5.4 KB

# Statuses, verdicts and thresholds

Everything below describes what scripts/scholarly.mjs computes. The comparisons are string heuristics: they are documented so a reader can overrule them with the evidence each result carries (comparison holds the claimed and the registered value of every field).

# How a reference is checked

  1. Identifier first. A cited DOI is looked up at Crossref; if Crossref does not know it, at doi.org, which covers every registration agency (DataCite, mEDRA, ...). arXiv DOIs and arXiv ids go to the arXiv API.
  2. Comparison. The cited fields are compared with the record (table below).
  3. No identifier, or a DOI that does not resolve. Crossref is searched with the whole reference; each candidate is judged; arXiv is searched by title when Crossref has no confident candidate.
  4. Notices. Crossref update notices attached to the record are read for retractions, expressions of concern and corrections.

# Field comparison

Field match Warning mismatch
Title similarity >= 0.9 close: 0.75 to 0.9 below 0.75
First author same family name as the record's first author order_differs: an author of the record, not the first not an author of the record
Authors (when several are cited) all cited authors are in the record, same count count_differs a cited author is absent from the record
Year equal off_by_one differs by 2 or more
Venue equal, abbreviation, or acronym differs never a mismatch
Volume, first page equal differs never a mismatch
  • Title similarity is the Sørensen-Dice coefficient on title words after lower-casing, removing accents, punctuation, markup and stop words, and a leading "RETRACTED:", "Erratum to:" or similar. A title that equals the other one plus a subtitle counts as 0.9.
  • Family names are compared after removing accents; particles stay with the name ("van der Waals"); initials and noisy index entries ("Tim Bollerslev B") are tolerated.
  • Venue tolerates abbreviations ("J. Financ. Econ.") and acronyms ("JFE"). Venue, volume and pages never produce a mismatch alone, because styles and registries disagree on them routinely.
  • Year off by one is a warning because online-first and issue years differ legitimately.

Verdict: mismatch when any field mismatches, minor_differences when there are only warnings, match otherwise.

# Statuses of verify-doi

Status Condition
invalid_doi The string is not a DOI (10.<4-9 digits>/<suffix>)
not_found Neither Crossref nor doi.org knows the DOI
doi_resolves_unchecked The DOI resolves, but neither a title nor both author and year were supplied
metadata_mismatch The DOI resolves and the verdict is mismatch
retracted The record carries a retraction, withdrawal or removal notice (wins over the others)
verified The DOI resolves and the verdict is match or minor_differences

# Statuses of verify-bib

One per entry: verified, doi_missing_found, metadata_mismatch, doi_not_found, doi_invalid, ambiguous, not_found, retracted, unverifiable, error. The meaning of each and what to do is in the table of SKILL.md. Details that matter:

  • doi_not_found with suggested_doi: the cited DOI does not exist, and a record with the same title plus author or year exists under another DOI. Typical of a mistyped or invented DOI on a real paper. Without suggested_doi, neither the DOI nor the title was found.
  • doi_missing_found: the entry had no DOI and exactly this rule held: title similarity >= 0.9 and the first author or the year agrees, with neither contradicting. alternatives lists other records that also satisfy it (a reprint, a preprint and its published version).
  • ambiguous: at least one record has a similar title (>= 0.75) but fails the rule above.
  • error: a registry timed out, rate limited or failed. The entry was not checked.

The report also gives duplicates (one DOI under several citation keys), parse_errors (BibTeX syntax errors with line numbers) and needs_attention (keys of every entry that is neither verified nor doi_missing_found).

# Confident match for a search candidate

A candidate is accepted only when all of these hold:

  • the title agrees: similarity >= 0.9, or, for a raw reference whose title could not be isolated, at least 90 % of the candidate's title words appear in the reference;
  • the first author agrees or the year agrees (within one year);
  • neither the first author nor the year contradicts;
  • for titles of fewer than three significant words ("Introduction", "Editorial"), both the author and the year agree.

This is deliberately strict. It prefers ambiguous to a wrong match, because a wrong match turns into a wrong DOI in someone's bibliography.

# Known weaknesses

  • Two different papers with the same title, first author and year (a conference paper and its journal extension) cannot be told apart: both appear, one as match, the other in alternatives.
  • Titles in non-Latin scripts fold to few or no comparable words; results for them are unreliable.
  • A reference whose free text does not follow a common style may be split badly into fields. The whole string is still sent to the registry search, and the title-word rule above applies.
  • Crossref metadata has errors of its own. A metadata_mismatch means the citation and the registry disagree, not that the citation is the one in error.
Files main
Menu

Theme