# Grading standard: citation accuracy benchmark This is the standard the citations were graded against. It was written before grading started. One verdict per tool per link, for the whole citation. Everything below applies to every tool in the same way. Two rules apply everywhere: - **The value is graded, not the format.** Name order and short forms (`Doe, Jane`, `Jane Doe`, `Doe, J.`), date format, capital letters in any case style, a first word `The`, and URL forms that reach the same document are never errors. Three of the four tools apply a style after they capture the values. To mark a tool for its style is to measure the style. - **A source can state more than one correct answer.** Where the source gives two names, two dates or two titles for the same fact, each one is Correct. The source must give it as that fact about this document. The publisher, date and syndication sections are examples of this rule. --- ## Author - **Correct**: the entity the source credits. - **Missing**: the source credits someone and the field is blank. An author list that only includes some of the given authors is also missing. - **Bad value**: something that is not the author. Examples: a site name on a page with a person listed, a section name, `2 min read`, a social handle, a photographer credit. ### A source that names no person Many pages credit an organization and no person. In that case: > The value must be a name that the responsible entity uses. For instance, `Tea > Weekly` is Correct. `Tea Weekly Staff` is a Bad value, because no organization > has that name. `Staff`, `Newsroom`, `Desk`, `Team` and `Report` are role words that a content management system adds. They say who typed the text. They do not say who is responsible for it. - `Staff Report`, `admin` and `site-admin` name no entity. Bad value. **A blank is not counted** when the tool's publisher field holds the responsible organization. The tool found the fact. Which field it puts the fact in is house style, and MLA 9 leaves the author out on these sources. If the publisher field does not hold that organization however, the blank is **Missing**. **The condition is necessary.** If the source credits a person, an organization in the author field is a Bad value. ### Syndication News sites republish each other. A republished article can show two credits. The host site puts its own byline above the headline. The first line of the text carries a `By ...` credit that came with the article. The writer named in the text is the author. The host's byline names the person who posted the article to the host site, which is not the same thing. On desotocountynews.com the byline reads `Peyton Taylor`, and the text starts `By Leonardo Bevilacqua | Originally published by Mississippi Today`. Leonardo Bevilacqua is the author, and Peyton Taylor is a Bad value. These are not two correct answers, because only one of the two names wrote the article. A blank is Missing here, because the source names a person. **A credit that is only in the page data still counts, if nothing on screen competes with it.** Where the page shows no credit at all, the name in the data is the only credit the source gives, and it is Correct. A blank is Missing there, because the source names a person. **On a syndicated page, the host site in the publisher field is enough.** Where the credit is a newsroom and the source names no person, the blank author rule applies as usual. The publisher field that holds the site which republished the work satisfies it. The originating newsroom is not necessary in that field. A field that gives the originating newsroom as the author is also Correct, because the source credits it. --- ## Date - **Correct**: the date that the source shows to a reader as the date of this document. A shown update date ("Updated 5 January 2026") counts. A month and a year, or a year alone, is Correct if that is all the source shows. - **Missing**: the source states a date and the field is blank. - **Bad value**: the wrong day. A month and a year, or a year alone, where the source displays the full date. A date that is only in the page data while the source displays a different one. A crawl date or an access date. A date from the URL that the document contradicts. **The day the source displays is the one graded.** If no date appears in the text and only the page data carries one, a difference of one day is not an error. A UTC timestamp and a local date fall on the two sides of midnight. **A publication date and a "last updated" date are two facts, and each one is Correct.** skyrefund.com shows `Last updated: June 12, 2026` and gives `datePublished` `2018-10-17` in its structured data. Both answer "the date of this document". **A contradiction is a different case.** A publication date against an update date is two facts, and both stand. A modification stamp that is older than the update the page shows is neither fact, and it is a Bad value. If the document states no date: A blank is Correct. **An issue of a periodical is a publication date.** Where an article carries an issue and a web timestamp, both are Correct. They are two facts. For instance, `Fall 2025` and `article:published_time` `2025-10-28`. **This applies only where the timestamp fits the issue.** A back issue that was loaded onto a site later carries a stamp for the upload, not for the work. If a site displays `Volume 92, Number 6` and `December 2017`, while `article:published_time` reads `2018-08-06`, the issue is the only publication date, and the timestamp is a Bad value. The test is whether the stamp could be the date when this work was published. **This does not permit all short dates.** A short value is Correct only where the source is short. If the source displays a full date and the field holds a month and a year, accuracy was lost, and it should be marked as wrong. Displays, not carries. Nearly every page has a full timestamp in its page data. A season names the issue that the article appeared in, which is its own fact. A month and a year is a full date with the day removed, which is the same fact with less of it. **A journal article's year is not a short date.** Where an article carries a volume and an issue, the year is the value that every style puts in the date slot. The year and the online publication date are both Correct. **The precision that the source carries is the precision graded.** Only the parts of the date that the source states are compared. A year alone is the whole fact where the source gives a year alone. The day is part of the fact where the source displays a day, and a value that falls back to the year has lost it. **More precision than the source carries is Correct only if it is true.** A month that comes from a DOI record is a real fact about a journal article, and a repository record can simply not print it: `2010-06` against a record that states 2010 is Correct. A day that no source supports is a Bad value. --- ## Title - **Correct**: the title of the document, with the subtitle where the source presents one. - **Missing**: blank. - **Bad value**: a title that is cut off at a colon or a dash, a title that stops in the middle of a phrase, a title that holds the site suffix (`Title | Tea Weekly`), the site name or a section name in place of a title, or a file name or a document number. **The subtitle is the part after the colon.** In `Failure to Act: Water and Wastewater Infrastructure`, "Failure to Act" is the title and the rest is the subtitle. A field that stops at the colon lost half of the title. On a web page there is often no formal subtitle, and `og:title` and `` differ for good reasons, so a missing subtitle is not an error there. There is one exception to the colon and dash rule. Where the part that follows is the site name, to drop it is correct. `Eurocities - Home` is a site suffix. It is not a subtitle. **A page can state two different titles, and each one is Correct.** A content management system often gives an article one headline in the page, and a different title in `<title>`, `og:title` and the `WebPage` name. The source gives each one as the title of this document, so the rule on more than one correct answer applies. **This does not forgive a cut title.** The short value is compared to the part of the long value before the colon. If the words are the same, the field lost the subtitle, and it is a Bad value. If the wording is different, the source states two titles, and both are Correct. **Corrupt text is a Bad value. Typography is not.** A `’` where an apostrophe belongs is an error, and so is a missing space (`PatentLaw`). Curly quotes against straight quotes, and an en dash against a hyphen, are not. --- ## Publisher - **Correct**: the name of the publication or the site, or the body that issued the document. - **Missing**: blank when the source identifies itself. - **Bad value**: a hostname, a content management system, a tagline, or a parent company that the source does not name as the publisher. **A hostname is still a hostname after it is tidied up.** `teaweekly.com` is obvious. So are `Kingcounty`, `Ontarioca` and `Ctetrailblazers`, which are the domain with the dots removed. The site states a real name in each case, so these should be marked as wrong. The exception is a site whose own masthead includes the domain suffix. There the suffix is part of the name, so for instance `Study.com` is Correct. **More than one name can be Correct**, if the source supports each one. - A local TV site: the `NewsMediaOrganization` block gives `KVLY`, and the dateline gives `Valley News Live`. To select between them would grade tool preference. - An institution and a part of it: a report can name both `National Renewable Energy Laboratory` and `U.S. Department of Energy`. Either is Correct where the document names it as the issuer. - An edition or a service of the same publisher, where the site names itself that way. aljazeera.com declares `Al Jazeera` in `og:site_name` and in the organisation block, and its own menu labels this site `Al Jazeera English` beside `Al Jazeera Arabic`. Both are Correct. The finer name must be a name that the site uses for itself on the page. It must identify the same publisher more precisely, and never a different one. `Al Jazeera Media Network` is the parent, so it is a Bad value. - An initialism that the source declares as its own name, and the expansion of it. americanactionforum.org sets `og:site_name` to `AAF`, names itself `American Action Forum` in the organisation block and the footer, and writes `American Action Forum's (AAF)` in the body. Both are Correct. The initialism must be one that the source uses. It must not be one built from the words of a name: a tool that turns `Tea Weekly Institute` into `TWI` gives a Bad value, because no reader would find that organisation. **"Supported" means that the source gives the name as the publisher** of this work: the site name, the masthead, the JSON-LD `publisher` block, the dateline credit, or the imprint of a document. It is not enough that the string is somewhere on the page. An owner in a copyright notice fails that test. fox5sandiego.com declares `FOX 5 San Diego & KUSI News` in `og:site_name` and in its `publisher` block, and names `Nexstar` only in a copyright line and the footer. `Nexstar` is a Bad value. Without this test the parent company rule has no effect, because every station page of a large group carries an owner copyright line. **A joined value is graded on what is attached, not on the separator.** - Every part names the same publisher: **Correct**. `Columbia | Zuckerman Institute`, `Information Technology and Innovation Foundation | ITIF`, `Office of the Governor - Michelle Lujan Grisham`. - The attached part is a tagline, a section, a date or a menu label: **Bad value**. `HCPLive - Clinical news for connected physicians`, `Eurocities - Home`, `Reports & Briefings | Jul 31, 2012 | ITIF`. --- ## URL - **Correct**: anything that reaches this document. The file itself, the publisher's landing page or abstract page for that same file, a `doi.org` link, and variants that resolve in the same way (parameters, `http`, a trailing slash, AMP). - **Missing**: blank. - **Bad value**: the site home page, a search page or a category page, a sign-in page or a subscribe page that is not the document's own page, or a different document. --- ## The other fields **Container** is only the book or the anthology that holds a chapter. In this benchmark a journal name and a site name belong in Publisher, for every tool, so a blank Container on a journal article or a web page is **Correct**. **Volume, issue and pages** are graded only where the source states them, which means a journal or a periodical. - **Missing**: the source states one and the field is blank. - **Bad value**: a value that the source does not state, or a page count in place of a page range. **Edition, editor, translator and place** are graded in the same way: only where the source states them. No source in this corpus states them, so a blank is Correct, and a filled field needs the source to support it. --- ## The verdict - **Correct**: every field is Correct, with any blank fields being fields that the page did not give information for. - **Wrong**: one field or more failed.