Wikidata – OpenCitations blog

Wikipedia Citations in Wikidata

Xosema, CC BY-SA 4.0, via Wikimedia Commons

The interconnection between Wikipedia and Wikidata is now larger than ever.

The Wikipedia Citations dataset currently includes around 30M citations from Wikipedia pages to a variety of sources – of which 4M are to scientific publication. The increase of the connection with external data services and the provision of structured data to one of the key elements of Wikipedia articles has two significant benefits: first of all, a better discoverability of relevant encyclopedic articles related to scholarly studies; furthermore, the enacting of Wikipedia as a social authority and policy hub which would enable policymakers to assess the importance of an article, person, research group and institution by looking at how many Wikipedia articles cite them.

These are the motivations behind the “Wikipedia Citations in Wikidata” project, supported by a grant from the WikiCite Initiative. From January 2021 until the end of April, the team of Silvio Peroni (director of OpenCitations), Giovanni Colavizza, Marilena Daquino, Gabriele Pisciotta and Simone Persiani from the University of Bologna (Department of Classical Philology and Italian Studies) has been working in developing a codebase to enrich Wikidata with citations to scholarly publications that are currently referenced in English Wikipedia. This codebase consists of four software modules in Python and integrates new components (a classifier to distinguish citations by cited source and a look-up module to equip citations with identifiers from Crossref or other APIs). In so doing, Wikipedia Citations extends upon prior work which only focused on citations already equipped with identifiers.

In the first two steps of the workflow (extractor and converter) the mapping between the various ways Wikipedia citations are represented in Wikipedia articles and the OpenCitations Data Model (OCDM) has been implemented and then enriched with a component responsible to find new identifiers to the entities in a dataset compliant with OCDM (enricher), while in the pusher step the mapping between the OCDM and Wikidata has been enabled, and the code has been finally released in GitHub.

The extensive documentation that has accompanied the release of the codebase is crucial for one of the principal aims of the project, I.e., the adoption and reuse of the codebase by the community in other relevant Wikimedia projects, while the engagement of various communities (Wikidata, libraries, scholars…) is favored on one side by offering an increased number of citations data included in Wikidata, on the other side by blogging and sharing the updates on Twitter and public mailing lists

This project, whose ambitious purpose is to make Wikipedia contents better discoverable and enrich Wikidata with a ready-to-use corpus for further analysis or for developing new services, is opened to future perspectives. The intention is to use the software to create a dataset of Wikipedia English citations to understand, in particular, how many new entities (i.e., citing Wikipedia pages, cited articles and venues, authors) should be added to Wikidata in order to upload all the set of extracted citations, with the result of adding a massive amount of new bibliographic-related entities to the dataset.

The first steps have been taken, now we aim to extend the engagement of the community involved, especially those scholars that leverage Wikidata in existing services, and to interact with the scholars, libraries and institutions interested in a new approach to research, focused on people (from individuals to research groups) and their intellectual relevance.

This post was first published on Diff, a Wikimedia community blog.

Citations as First-Class Data Entities: The Open Citation Identifier Resolution Service

Requirements for citations to be treated as first-class data entities

In my introductory blog post, I listed five requirements for the treatment of citations as first-class data entities. The fifth and final of these requirements is that there must be a Web-based identifier resolution service that takes the citation identifier as input and returns a description of the citation.

At the recent PIDapalooza Conference on persistent identifiers, held in Gerona, Spain, I described the Open Citation Identifier Resolution Service, the new resolution service for Open Citation Identifiers created and operated by OpenCitations [1].

In this post, I describe this Open Citation Identifier Resolution Service, which supports the resolution of Open Citation Identifiers not only of the citations documented in the OpenCitations Corpus (OCC), but also of open citations recorded in other bibliographic databases.

What is the Open Citation Identifier Resolution Service

The Open Citation Identifier Resolution Service runs on the OpenCitations server, presenting itself to the user as a web page with the URI http://opencitations.net/oci.

When a user enters a valid OCI and clicks the “Look up citation” button, this activates the resolution service, which, after a brief delay, returns information about the citation itself and about the citing and cited bibliographic resources, as shown in the following screen image (which for clarity omits the provenance data associated with this citation).

This information can optionally be returned to the user in a variety of other formats: RDF/XML, Turtle or JSON-LD.

Clicking on the links provided will return additional metadata held by the OpenCitations Corpus for the citing and the cited documents. In the near future, this service will be integrated with LUCINDA, the forthcoming OCC browse interface, to present this information in a more user-friendly fashion.

Using the Resolution Service with citations in an external resource via a SPARQL endpoint

The Open Citation Identifier Resolution Service currently works for citations between bibliographic resources both within the OpenCitations Corpus and within external bibliographic databases, provided that the external service uses bibliographic resource identifiers having a unique numerical part, and provides a SPARQL endpoint to makes available information about bibliographic resources and the references they contain.

It can therefore resolve OCIs identifying citations within Wikidata, such as oci:01027931310-01022252312, where, as explained in the previous blog post, “010” is the assigned OCC supplier prefix for Wikidata.

Entering this OCI in the Open Citation Identifier Resolution Service pulls live data from the Wikidata SPARQL endpoint and returns the following information about that citation, as shown in the following screen image (which, again, omits for clarity the provenance data associated with that citation):

Clicking on the links provided here returns information about the relevant Wikidata entities.

Citing paper:

Cited paper:

How the Resolution Service works

The bibliographic database supplying the metadata for a particular citation identified by an OCI is specified by the assigned OCC supplier prefix that forms part of the OCI, as described in the previous blog post. Each OCI is thus specific for and unique within a particular bibliographic database.

The resolution service takes the OCI entered into the search box, recognises the supplier prefix specifying the bibliographic database holding the citation information, parses the OCI into the database identifiers for the citing and cited entities, and then sends an appropriate SPARQL query to interrogate the SPARQL endpoint of the relevant database. When that database has returned information about the citation itself and about the citing and cited bibliographic resources, this is displayed to the user as shown in screen images above – or in other RDF formats (Turtle, JSON-LD, RDF/XML) according to the request.

It is important to realize that no other databases are contacted during this resolution process, and that the quality and accuracy of the metadata retrieved by the Open Citation Identifier Resolution Service is the responsibility of the database hosting that citation. The OCI Resolution Service does no more than retrieve this information, and does nothing to address possible errors or omissions in the metadata coming from the hosting database.

Using the Resolution Service with external citations via a REST API

While the resolution service presently works only to retrieve information from bibliographic databases having a SPARQL endpoint, we plan soon to extend this resolution service to work with information supplied by a bibliographic database via a REST API.

Coupled with the ability to create OCIs by numerical conversions of Digital Object Identifiers (DOIs), as explained in the previous blog post, the Open Citation Resolution Service could then be used to pull metadata live from the Crossref REST API for any of the ~350 million Crossref open references in which the cited paper as well as the citing paper has a DOI, and for which an OCI can thus be created.

Watch this space!

References

[1] David Shotton (2018). Citations as first-class data entities. Open Citation Identifiers. Conference presentation. PIDapalooza 2018, Girona, 23-23 January 2018. https://doi.org/10.6084/m9.figshare.5844972

Citations as First-Class Data Entities: Open Citation Identifiers

Requirements for citations to be treated as First-Class Data Entities

In my introductory blog post, I listed five requirements for the treatment of citations as first-class data entities. The fourth of these requirements is that they must be identifiable using a global persistent identifier scheme.

At the recent PIDapalooza Conference on persistent identifiers, held in Girona, Spain, I launched the Open Citation Identifier (abbreviated OCI, in line with DOI), the new persistent identifier for citations [1].

In this post, I describe the Open Citation Identifier scheme, created and operated by OpenCitations, which supports the assignment of Open Citation Identifiers not only to the citations present in the OpenCitations Corpus (OCC) but also to open citations present in other bibliographic databases.

Structure and syntax of the Open Citation Identifier

Each OCI has a simple structure: oci:number-number, where “oci:” is the identifier prefix.

OCIs for citations stored within the OpenCitations Corpus are constructed by combining the OpenCitations Corpus local identifiers for the citing and cited bibliographic resources, separating them with a dash. (For definition of OCC local identifiers, see the OpenCitations Data Model).

For example, oci:2544384-7295288 is a valid OCI for the citation between two papers stored within the OpenCitations Corpus, the first number being the OCC local identifier for the citing bibliographic resource [2], and the second being the OCC local identifier for the cited bibliographic resource [3], these bibliographic resource local identifiers being unique within the OCC. [Note: Supplier prefixes are omitted from OCC local identifiers of bibliographic resources ingested into the OpenCitations Corpus prior to February 2018, but will be included within all OCC local identifiers of bibliographic resources ingested into Corpus after that date.]

OCIs for external resources identifies by numerical identifiers

OCIs can also be created for bibliographic resources described in an external bibliographic database, if they are similarly identified there by identifiers having a unique numerical part. For example, the OCI for the citation that exists between Wikidata resources Q27931310 (the citing resource, [4]) and Q22252312 (the cited resource, [5]) is oci:01027931310–01022252312, where “010” is the assigned OCC supplier prefix for Wikidata. (The colours here and below are added simply for clarity.)

The OCC supplier prefix consist of a positive number (following the pattern “nnn”, where “nnn” is a string of numerals of variable length which includes no zeros), enclosed between two zeros (e.g. “0420”). The list of all assigned OCC supplier prefixes is given at https://github.com/opencitations/oci/blob/master/suppliers.csv.

OCIs for citations between resources identified by DOIs

OCIs can also be created for bibliographic resources described in external bibliographic database such as Crossref or DataCite where they are identified by alphanumeric Digital Object Identifiers (DOIs), rather than purely numerical strings.

To achieve this, each case-insensitive DOI is first normalized to lower case letters. Then, after omitting the initial “doi:10.” prefix, the alphanumeric string of the DOI is converted reversibly to a pure numerical string using the simple two-numeral lookup table for numerals, lower case letters and other characters presented at https://github.com/opencitations/oci/blob/master/lookup.csv. For example, using this lockup table, “1” becomes “01”, “2” becomes “02”, “a” becomes “10”, “b” becomes “11”, and “/” becomes “36”. To the resulting number, the appropriate OCC supplier prefix is then added, to clearly identify its provenance.

A citation documented in Crossref exists between the two publications [3] and [6], which are there identified by the DOIs doi:10.1108/jd-12-2013-0166 and doi:10.1371/journal.pcbi.1000361. We can thus create an OCI for this Crossref citation by using numerical representations of the two DOIs. These numerical representations are:

0200101000836191363010263020001036300010606

and

02001030701361924302723102137251211183701000000030601

where the initial “020” in each case is the assigned OCC supplier prefix for Crossref.

From these two numerical representations of DOIs, the OCI for the Crossref citation between these two paper is easily constructed, and is:

oci:0200101000836191363010263020001036300010606-02001030701361924302723102137251211183701000000030601

While this is long for an identifier, it should be remembered that it will be processed computationally, and is not intended for human readability.

In this way, Crossref OCIs can be assigned to all ~350 million open references within Crossref in which the cited paper as well as the citing paper has a DOI [7].

OCIs for the same citation recorded within different databases

If a citation is recorded in more than one bibliographic database, a separate OCI can be created for each instance, each OCI having a distinct supplier prefix and being specific to that database.

Thus, in addition to the Crossref OCI created from DOIs and described above for the citation from [3] to [6], a Wikidata OCI exists for the same citation recorded within Wikidata, having the form oci:01024260641-01021092566.

Upon resolution of an OCI, the Open Citation Identifier Resolution Service will pull metadata only from the database specified by the supplier prefix of the OCI. Details of the Open Citation Identifier Resolution Service are given in the next blog post.

It is important to note that an OCI can only be used to specify a citation between a citing and a cited publication which is actually recorded within a bibliographic database. For this reason, the OCI “oci:7295288-3962641” shown below the second diagram in the introductory blog post to this series is presently invalid. While the OpenCitations Corpus has metadata describing both bibliographic resources [3] and [6], it has not yet ingested the reference list for the first bibliographic resource [3] (which has the OCC local identifier 7295288), having information about it only from a reference within a third paper, with no information about the references [3] itself contains. As a result, at present OCC has no record that a citation actually exists between [3] and the second bibliographic resource [6] (which has the OCC local identifier 3962641).

Representing OCIs in RDF

To permit the description of OCIs in RDF, “oci” has been added as a new member of the class datacite:ResourceIdentifierScheme within the DataCite Ontology.

The resolvable URL for any citation identified by a OCI has the form “https://w3id.org/oc/virtual/ci/nnn-mmm”, where nnn-mmm represents the OCI with its “oci:” prefix removed. Currently, we are able to return the RDF description of all the citations contained in the OpenCitations Corpus and Wikidata. We are working to extend the coverage so as to include other datasets, e.g. Crossref.

References

[2] Armen Yuri Gasparyan, Marlen Yessirkepov et al. (2015). Preserving the integrity of citations and references by all stakeholders of science communication. J. Korean Med. Sci. 30:1545-1552. (English.) https://doi.org/10.3346/jkms.2015.30.11.1545

[3] Silvio Peroni, Alexander Dutton, Tanya Gray and David Shotton (2015). Setting our bibliographic references free: towards open citation data. Journal of Documentation, 71 (2): 253-277. https://doi.org/10.1108/jd-12-2013-0166

[4] Daniel K. Bricker, Eric B. Taylor et al. (2012). A Mitochondrial Pyruvate Carrier Required for Pyruvate Uptake in Yeast, Drosophila, and Humans. Science 337: 96-100.
https://doi.org/10.1126/science.1218099

[5] Douglas Hanahan and Robert A. Weinberg (2011). Hallmarks of cancer: the next generation. Cell 144: 646–674. https://doi.org/10.1016/j.cell.2011.02.013

[6] David Shotton, Katie Portwin, Graham Klyne and Alistair Miles (2009). Adventures in semantic publishing: exemplar semantic enhancement of a research article. PLoS Computational Biology 5: e1000361. http://dx.doi.org/10.1371/journal.pcbi.1000361

[7] Daniel Ecer (2017). Crossref Data Notebook (updated). Available at https://elifesci.org/crossref-data-notebook

Citations as First-Class Data Entities: The OpenCitations Data Model

Requirements for citations to be treated as First-Class Data Entities

In my introductory blog post, I listed five requirements for the treatment of citations as first-class data entities. The second of these requirements is that they must have metadata structured using a generic yet appropriately detailed data model.

To fulfil that requirement, OpenCitations is pleased to announce the publication on 13 February 2018 of the OpenCitations Data Model, v1.6 [1]. This replaces the previous version, v1.5.3, published on 13 July 2016.

The data model has been expanded and enhanced to improve the recording of publication dates, to include the treatment of citations as first-class data entities, and to permit the model’s adoption by third parties who may wish to use it to model their own citation data, or to prepare their citation data for publication in the OpenCitations Corpus (OCC). To facilitate this, the document describing this data model is published under a Creative Commons Attribution 4.0 International license.

In addition to a change in the title from “Metadata for the OpenCitations Corpus” to “The OpenCitations Data Model”, and the use of the name “OpenCitations” (one token with two words in camel case) in place of “Open Citations” (with the space separating the two words), the substantive changes in the model from the previous version are as follows:

New class

A new class, Archival document, has been added as a subclass of bibliographic resource, to permit the model to be used for work on ancient manuscripts.

Publication dates

The mechanism for recording the publication dates of bibliographic resources has been improved, and now accepts the full date of publication (yyyy-mm-dd, if available), or the year plus the month of publication (yyyy-mm, if the full date is not available), or failing that just the year of publication (yyyy, as in the previous version of the data model). In order to support this modification in the OWL mapping, prism:publicationDate is now used instead of fabio:hasPublicationYear.

Citations as first-class data entities

A new class of bibliographic entity, Citation, has been added to permit the description of citations as first-class data entities. This class has been assigned sub-classes (e.g. Author self-citation) and properties (e.g. citation time span) to permit the description of citations in a manner helpful for bibliometric analysis. These, and associated changes to CiTO, the Citation Typing Ontology, are described more fully in the previous blog post.

Virtual entities

The OpenCitations Data Model now permits the definition of virtual entities, i.e. bibliographic entities that are defined on-the-fly, only when they are requested (for example, by accessing their URLs). These are defined either by using data relating to non-virtual bibliographic entities that are already available within the OCC, or by using data that are themselves obtained on-the-fly from an external supplier (e.g. Wikidata).

This approach of using virtual RDF resources is optional, and is simply employed for storage efficiency, to avoid duplication of information within the OCC triplestore. As of January 2018, only one type of bibliographic entity is defined as a virtual entity, namely a citation (a members of the class Citation).

Such a virtual entity does not have the full provenance information normally associated with other bibliographic entities within the OCC, but it does have associated with itself the date of its creation and direct links both to the agent responsible for such creation and to the source data used in its construction.

Because we do not separately store these virtual entities within the Corpus triplestore, they cannot be directly queried by means of the OCC SPARQL end-point, neither are they stored within its data dumps. However, the data associated with an OCC virtual entity can be obtained by accessing its URL, which has form “https://w3id.org/oc/virtual/xyz”, clearly distinguishable from those URIs used for other (non-virtual) OCC bibliographic entities which have the form “https://w3id.org/oc/corpus/xyz”. More details and examples are given in the Data Model document itself.

Additionally, for citations defined using Open Citation Identifiers (OCIs, described in a subsequent blog post), details of the cited and citing publications may be readily obtained by using the Open Citation Identifier Resolution Service at http://opencitations.net/oci.

Supplier prefixes

To enable citation data created by third parties to be incorporated within the OpenCitations Corpus, from February 2018 the OCC local identifiers for bibliographic resources now include a supplier prefix which clearly identifies the provenance of the data. The prefix consists of a positive number (following the pattern “nnn”, where “nnn” is a string of numerals of variable length which includes no zeros), enclosed between two zeros (e.g. “0420”).

To ensure uniqueness of prefixes used by different suppliers, all organizations wishing to adopt the OpenCitations Data Model and to use it to create publicly available citation data, whether these are published in the OpenCitations Corpus or independently, must apply to OpenCitations for a unique supplier prefix, by sending an email to support@opencitations.net. A list of already assigned supplier prefixes is available at https://github.com/opencitations/oci/blob/master/suppliers.csv.

The appropriate supplier prefix is combined with a unique numerical string that forms the ‘body’ of the identifier to create the local identifier used in OCC to identify an individual bibliographic resource. OCC local identifiers for citations (as opposed to bibliographic resources) are constructed by combining the local identifiers for the citing and cited bibliographic resources, separating them with a dash. Thus, for a citation between two bibliographic resources described in an external bibliographic database where they are each identified by an identifier having a unique numerical part, the OCC local identifiers for the citing and cited bibliographic resources are combined, separating them with a dash.

For example, the citation between citing Wikidata resource Q27931310 and cited Wikidata resource Q22252312 is given the OCC local citation identifier “01027931310-01022252312”, where “010” is the OCC supplier prefix (defined above) for Wikidata. How these OCC local identifiers for citations are used to create Open Citation Identifiers is described in a separate blog post.

We commend the OpenCitations Data Model to anyone considering the storage of citation information, particularly if it is to be encoded in RDF, and we welcome contributions of citation data encoded using this model for publication within the OpenCitations Corpus.

Reference

[1] Silvio Peroni, David Shotton (2018). The OpenCitations Data Model. Version 1.6. figshare. https://doi.org/10.6084/m9.figshare.3443876