A new revolutionary workflow for a unified collection of citations: say hello to the OpenCitations Index

Blog post by Ivan Heibi (University of Bologna), Arianna Moretti (University of Bologna) and Chiara Di Giambattista (University of Bologna).

In the past five years, the OpenCitations data has been enriched with numerous new indexes of open citation data from different sources. However, the quantity and diversification of the ingested information have raised several issues, which recently made it essential to conduct a complete revision of the ingestion workflow. The result was a revolution in the way OpenCitations data is delivered. In this blog post, we will explain the context and challenges raised by the old procedure. Then, we will present the new ingestion workflow, designed to produce just two comprehensive collections: OpenCitations Index, collecting open citation data, and OpenCitations Meta, for the open bibliographical metadata. 

Once upon a time, there were five OpenCitations indexes…

In 2018, OpenCitations released the kickoff version of its first citation index, COCI (citations from Crossref), which contained around 300 million citation links derived from the subset of the reference lists in the Crossref database, where citing and cited entities were identified using Digital Object Identifiers (DOIs). COCI gathered citations with associated metadata in compliance with the recommendations from the Initiative for Open Citations (I4OC) that citation data should be structured, separable, and open, thus marking a turning point by providing a disruptive and free and open alternative to earlier sources such as Google Scholar, which provided freely accessible data although not downloadable, and Web of Science or Scopus, which demanded paid access. 

In a short time, COCI became a competitive and trusted index of citation data, used by numerous institutional repositories, including B!son and Optimeta. In 2021, COCI was taken into account in a comparative study with the most relevant sources in the landscape, including the proprietary ones, which showed its coverage approaching parity with those of the other sources involved in the analysis (Microsoft Academic, Scopus, Dimensions, and Web of Science). At the time of its most recent update in January 2023, COCI counted more than 1.4 billion citations. The reason behind this outstanding number lies in several factors, including Elsevier’s endorsement of the Declaration on Research Assessment (DORA) in December 2020, leading to the open release via Crossref of the reference lists of the articles published in all its journals, and confirming the value of initiatives such as the Initiative for Open Citations (I4OC)

However, before this change of heart, in 2019 OpenCitations had tried to narrow the open citations coverage gap by launching its second index, the Crowdsourced Open Citations Index (CROCI). This index allowed publishers and scholars to contribute directly by uploading crowdsourced open citations into the OpenCitations infrastructure.

In December 2022, a new concrete step towards a factual plurality of OpenCitations indexes was taken by the ingestion of new data sources into the infrastructure, with the publication of the inaugural dumps of DOCI (citations from DataCite) and POCI (citations from PubMed). In June 2023, the first version of the OROCI (citations from OpenAIRE) dump was released too, and JOCI (citations from JALC) is expected to be available by the end of November 2023, for a total of five collections from different sources. 

Why a new workflow? The issues with multiple sources management and new challenges

While having such a variety and richness of indexes helped present the extent of OpenCitations sources, the recent increment in the number of sources and the diversification of data integrated led to two primary issues:

    1. the necessity to handle the ingestion of new identifier types in a DOI-based software infrastructure, and
    2. the consequent possibility of encountering the same citation expressed by several sources with different identifiers.

Moreover, it soon became evident the need to optimize the reuse of the already developed software components to facilitate the metadata crosswalk processes between the new sources’ data models and the OpenCitations Data Model, with the aim to define a functional and easily extendable workflow to be easily reused when it comes to incorporating new data sources, which should be: 

    1. sufficiently generic to establish a globally unique procedure; 
    2. customizable enough to capture the necessary information within each of the specific data models and formats. 

As a solution, we decided to use OpenCitations Meta, the new OpenCitations database and tool for managing bibliographic data related to the publications involved in the citations. OpenCitations Meta makes it possible to assign each entity involved in a citation an internal identifier, nominally the OpenCitations Meta Identifier (OMID), to which all the associated persistent identifiers of the same publication are redirected.

As a result, the allocation of an OMID for each bibliographic resource also enabled the unambiguous identification of each citation, regardless of the persistent identifier schema originally used by the data source to identify the resources. This approach allowed us to perform data deduplication and finally make all the sources’ contributions converge into a unified index containing all the unique citations managed by OpenCitations, expressed as OMID to OMID citation links.

The revised workflow

The new workflow is based on three main components with the benefit of optimizing the process both in terms of computational cost and in terms of flexibility. As shown in Fig. 1, in a preliminary step, source-specific software converts the input dataset – structured according to the source data model – to extract two OpenCitations Data Model compliant data collections in tabular format for bibliographic metadata and citation data, respectively.

The following steps are common to the process of each dataset.  

STEP 1: The bibliographic metadata collection is used as input for the META software. At this stage, it is checked whether or not the bibliographic entities have been previously integrated into our infrastructure (coming from other data sources). If so, the existing OMID is linked also to the new alternative identifiers of the new bibliographic resources. New metadata values, if any, are also integrated. A new OMID identifier is produced for entities never previously encountered, uniquely representing the bibliographic resource in OpenCitations. The outputs of the process are: (I) an updated version of the OpenCitations Meta collection that also includes the metadata of the bibliographic entities provided by the new source, and (II) a collection of provenance data. An internal database is constantly refreshed to preserve correspondence between IDs and the associated internal OMIDs.

STEP 2: Starting from the collection of citations expressed as directional links between identifiers of potentially any type (e.g., DOI-DOI, PMID-PMID, PMC-PMID, etc.), the INDEX software queries the internal database mapping IDs to OMIDs to produce an updated version of the OpenCitations Index: unique citations expressed as OMID-OMID links in different formats, accompanied by their corresponding provenance data.

Fig. 1: An overview of the data ingestion workflow, starting from the data source-specific conversion and production of citations and bibliographic metadata tables, progressing through the META process and the assignation of an OMID identifier to each bibliographic record involved in a citation, and culminating with the exposition of the OpenCitations Index collection of OMID-OMID unique citations.

What we have now: The OpenCitations Index 

From now on, OpenCitations will no longer display an index of citation data for each source. Instead, we will publish a single collection of citations into which the contributions from each of the sources will flow, which we will simply call ‘The OpenCitations Index‘. The first version of this unified index of OMID-OMID citations is posted on Figshare. It was produced in RDF, CSV, and SCHOLIX formats, together with a collection of its provenance information, provided in RDF and CSV formats. For each citation, it is possible to trace the source of the information by consulting the Provenance data collection, thanks to the http://www.w3.org/ns/prov#atLocation property, which defines the location of each citation.

This new solution has the benefit of simplifying the consultation of the data maintained by our infrastructure without reducing the information content. In addition, by including efficient handling of the deduplication problem, the new Index not only provides accurate data on the exact number of unique citations exposed by the framework but also verifies the individual contribution of each source, as well as their overlapping data (Fig. 2).

Fig. 2: An overview of the number of citations stored in the OpenCitations Index as of October 31, 2023. The diagonal cells in the table (highlighted in yellow) show the unique contribution of each collection to the OpenCitations Index, while the other cells represent the citations that are shared between the collections. More in detail, the green cells show the overall input of each source, while the pink cells represent the number of overlapping citations between two data sources.

Currently, the Index contains almost 2 billion unique citations. By the end of November, a new version of the collection will be published, including the contribution of the new Japan Link Centre (JaLC) source. 

How to access the OpenCitations Index data

To maximize the reuse of the exposed information and to ensure the greatest possible interoperability, the collection will always be published on Figshare in all formats listed above. In addition, the data will be accessible via an API, a SPARQL endpoint, and a web interface.

The redesign of the ingestion workflow marks a fundamental step for OpenCitations towards a more intuitive and simple access to our services while always preserving and improving the quality of our data. If you need further information on how the new workflow works, please visit our website, contact us at contact@opencitations.net  or leave feedback and/or suggestions in the dedicated card on our public roadmap to help us improve our services and communications. Thank you!

Discover POCI, the index of open citations from PubMed 

We’re happy to announce POCI, the OpenCitations Index of PubMed open PMID-to-PMID citations, an RDF dataset containing details of all the citations from publications bearing PubMed Identifiers (PMIDs) to other PMID-identified publications, harvested from the National Institutes of Health Open Citations Collection (NIH-OCC). The citations available in POCI are treated as first-class data entities, with accompanying properties including the citations timespan, modelled according to the OpenCitations Data Model. 

Currently, POCI’s December 2022 release contains 717,654,703 citations from 26,024,862 bibliographic resources, and is based on the dump of NIH Open Citation Collection dated November 2022. 

Citation URLs

Each citation (i.e. an individual of the class cito:Citation) is identified by an URL structured as follows:

https://w3id.org/oc/index/poci/ci/[[OCI]].

Open Citation Identifiers

Each Open Citation Identifier [[OCI]] has a simple structure: the lower-case letters “oci” followed by a colon, followed by two numbers separated by a dash (e.g. https://w3id.org/oc/index/poci/ci/01600102060800080706-016002060909030401), in which the first number identifies the citing work and the second number identifies the cited work.

For citations in which the citing and cited works are identified by PMIDs, which includes all the POCI citations, the OCI is created in the following manner, as explained more fully here. Each converted numeral part of OCI is prefixed by a 0160, which indicates that NIH is the supplier of the original metadata of the citation (as indicated at http://opencitations.net/oci).

OCIs can be resolved using the OpenCitations OCI Resolution Service.

Access to POCI data

All the data in POCI:

What is an Open Citation Index?

A citation index is a bibliographic index recording citations between publications, allowing the user to establish which later documents cite earlier documents. The current indexes available in OpenCitations are:  

All the OpenCitations Indexes have six characteristics in common, summarized here: https://opencitations.net/index   

 

Discover DOCI, the index of open citations from DataCite

We’re excited to introduce DOCI, the OpenCitations Index of Datacite open DOI-to-DOI citations, a new tool containing citations derived from publications bearing DataCite DOIs to other DOI-identified publications, harvested from DataCite. The citations available in DOCI are treated as first-class data entities, with accompanying properties including the citations timespan, modelled according to the OpenCitations Data Model

Currently, DOCI’s December 2022 release contains 169,822,752 citations from 1,753,860  citing resources, and is based on the last dump of DataCite dated 22 October 2021 provided by the Internet Archive

Citation URLs

Each citation (i.e. an individual of the class cito:Citation) is identified by an URL structured as follows:

 https://w3id.org/oc/index/doci/ci/[[OCI]].

Open Citation Identifiers

Each Open Citation Identifier [[OCI]] has a simple structure: the lower-case letters “oci” followed by a colon, followed by two numbers separated by a dash (e.g. https://opencitations.net/index/doci/ci/080010504060836132137200707121027-080010504060836161221130313.html), in which the first number identifies the citing work and the second number identifies the cited work.

For citations in which the citing and cited works are identified by DOIs, which includes all the DOCI citations, the OCI is created in the following manner, as explained more fully here. Each case-insensitive DOI is first normalized to lower case letters. Then, after omitting the initial doi:10. prefix, the alphanumeric string of the DOI is converted reversibly to a pure numerical string using the simple two-numeral lookup table for numerals, lower case letters and other characters presented at https://github.com/opencitations/oci/blob/master/lookup.csv. Finally, each converted numeral is prefixes by a 080, which indicates that DataCite is the supplier of the original metadata of the citation (as indicated at http://opencitations.net/oci).

OCIs can be resolved using the OpenCitations OCI Resolution Service.

Access to DOCI data

All the data in DOCI:

More information is available at https://opencitations.net/index/doci. 

What is an Open Citation Index? 

A citation index is a bibliographic index recording citations between publications, allowing the user to establish which later documents cite earlier documents. The current indexes available in OpenCitations are: 

All the OpenCitations Indexes have six characteristics in common, summarized here: https://opencitations.net/index  

Tutorial: how to process COCI’s zipped CSV dump without decompressing it

Blog post by Ivan Heibi (Universiy of Bologna) and Arcangelo Massari (University of Bologna).

OpenCitations publishes the COCI dataset after each new release in three main formats: CSV, N-Triples, and Scholix (see https://opencitations.net/download#coci). The CSV format is the most popular and downloaded one due to its comprehensive data organization (i.e. tabular format) and smaller size (compared to the other formats provided). Therefore, this is also the format we suggest using for a local process of the entire COCI dataset. 

The CSV dumps of COCI are uploaded on Figshare. You can check and download the last dump released from https://doi.org/10.6084/m9.figshare.6741422. The dump consists of one main ZIP file, including other smaller ZIP archives (one for each release) containing the actual CSV files (Figure 1).

Figure 1. The contents of the COCI CSV dataset (after the August 2022 release)

It is possible to process this data without unzipping the internal archives, thus saving a lot of disk space. In this tutorial, we will see how to achieve this in Python. Same process could be done in other programming languages.

Processing the COCI dump using Python

Step 1) Downloading the COCI dump

First, you need to download the last CSV dump release of COCI from https://doi.org/10.6084/m9.figshare.6741422 and decompress only the external archive. After this operation, you should have a folder containing the internal ZIP files such as in Figure 1.

Note: It is beneficial to decompress the external archive because doing so does not increase the space occupied on the disk (compressing archives results in a compression rate of 0%) and because working on nested archives would significantly increase RAM requirements. 

Step 2) Working with the ZIP files

Python provides the built-in zipfile module, whose ZipFile class allows you to create, read, write, edit and list the contents of a ZIP file. Given as input the path of the root directory containing all the ZIP files (FOLDER_PATH), the process elaborates each of these files on a different iteration. Each cycle initializes a ZipFile object by specifying the path to the ZIP file (archive_path).

from zipfile import ZipFile
import os

for archive_name in os.listdir(FOLDER_PATH):        
archive_path = os.path.join(FOLDER_PATH, archive_name)
with ZipFile(archive_path) as archive:     # ...

Step 3) Accessing the ZIP files

Use the namelist() method to return the list of CSV files contained in each archive. Then to open the inner CSV files, simply cycle through the list of names and feed them to the open() method of the ZipFile instance, i.e. archive in the example below.

from zipfile import ZipFile
import os

for archive_name in os.listdir(FOLDER_PATH):
    archive_path = os.path.join(FOLDER_PATH, archive_name)
    with ZipFile(archive_path) as archive:
        for csv_name in archive.namelist():
with archive.open(csv_name) as csv_file:       # ...

Step 4) Reading the CSVs

The .open() method returns a buffer. To read the CSV file as a list of dictionaries (i.e. represent each row of the CSV in dictionary format, e.g., {“column1″:”val1”, “column2″:”val2”}) we need to transform the buffer using the TextIOWrapper class and read it using the DictReader class of csv. Then we convert the result of DictReader into a list. 

from io import TextIOWrapper
from zipfile import ZipFile
import os

for archive_name in os.listdir(FOLDER_PATH):
    archive_path = os.path.join(FOLDER_PATH, archive_name)
    with ZipFile(archive_path) as archive:
        for csv_name in archive.namelist():
with archive.open(csv_name) as csv_file:
reader = csv.DictReader(io.TextIOWrapper(csv_file))
rows = list(reader)
# ...

Step 5) Processing the CSVs content

Now you can go through each row of the list and process the citation data as you want. The following example prints the citing and cited entity of each citation in the dump. 

from io import TextIOWrapper
from zipfile import ZipFile
import os

for archive_name in os.listdir(FOLDER_PATH):
    archive_path = os.path.join(FOLDER_PATH, archive_name)
    with ZipFile(archive_path) as archive:
        for csv_name in archive.namelist():
with archive.open(csv_name) as csv_file:
reader = csv.DictReader(io.TextIOWrapper(csv_file))
rows = list(reader)
# Process the CSV here
for r in rows:
print("Citing entity:",r["citing"])
print("Cited entity:",r["cited"])

 

Additional 48 million citations in COCI, including references from IEEE 

We announce the August 2022 release of COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations, which is based on open references to works with DOIs within the Crossref dump dated August 2022. This new release extends COCI with more than 48 million additional citations, giving a total number of more than 1.36 billion DOI-to-DOI citation links. 

This release includes citations from the articles published over the last four years by IEEE, whose bibliographic references were opened in June 2022. 

A fundamental role in pushing the commercial publishers to open their citation data was played by Crossref’s recent announcement to change its reference distribution policy, by making all its metadata open.  

Besides IEEE, COCI already includes the citation data derived from Elsevier (open via Crossref since December 2020) and from the last articles published by the American Chemical Society (whose references were opened in February 2021) 

You can find more information about COCI in our open-access article  

Ivan Heibi, Silvio Peroni & David Shotton (2019). Software review: COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations. Scientometrics, 121 (2): 1213-1228. DOI: https://doi.org/10.1007/s11192-019-03217-6    

Finally, just a reminder that the bibliographic and citation data in COCI:  

    • can be queried using the OpenCitations Indexes SPARQL endpoint;  
    • can be retrieved by using the COCI REST API;  
    • can be searched by using the OpenCitations Indexes Search Interface;  
    • are also available as dumps on Figshare in CSV, N-Triples, and Scholix; and  
    • can be freely re-used for any purpose.

New documents that present OpenCitations’ mission, unique benefits, present status and future plans

Posted on August 10th 2022 by Chiara Di Giambattista

More than a year ago, Ginny Hendricks, Director of Member & Community Outreach for Crossref, and a valued member of the OpenCitations International Advisory Board, published on the Crossref blog the post “The road ahead: our strategy through 2025”. In order to describe all Crossref’s principles and activities, Ginny presented the Crossref strategic planning framework as a diagram summarizing Crossref’s statements, key messages and truths. The clarity and immediacy of the diagram were such that we adapted it to present  OpenCitations’ own statements and goals. The resulting poster “OpenCitations – what does the future hold?” was presented by our Director David Shotton at the OASPA2021 conference, and can be found in this blog post.

Although the poster offered a wide overview of OpenCitations values, unique traits, benefits and plans, it differed slightly from Ginny’s original diagram, in particular because it lacked a “Mission Statement”, scattering the relevant information within the “Values” and “Principles” boxes. Indeed, at that time (September 2021), we didn’t have a clearly defined Mission Statement.

Nevertheless, the creation of that poster was crucial in helping us start to articulate more clearly the purpose and meaning of OpenCitations. As David underlined in his post “From little acorns…a retrospective on OpenCitations”, since 2018 OpenCitations activities have progressively increased and, with them, the number of related journal articles, conference papers and technical definitions. OpenCitations’ involvement in international networks and collaborations (such as SCOSS and the OpenAIRE-Nexus project), together with our need of identifying and reaching out to new stakeholders to assure OpenCitations’ development and sustainability, has made it necessary to publicly define OpenCitations’ mission, unique strengths and next developmental steps.

After numerous revisions, aided by wise advice from members of the OpenCitations Advisory Board members, we’re now happy to publish the following three OpenCitations documents:

OpenCitations Mission Statement,

The Uniqueness of OpenCitations   and

OpenCitations – Present Status and Future Plans,

which together provide a summary of why we exist and where we are heading.

We are particularly proud of the definition of OpenCitations’ primary mission, namely

to harvest and openly publish accurate and comprehensive metadata describing the world’s academic publications and the scholarly citations that link them, and to preserve ongoing access to this information by secure archiving.

The Mission Statement also presents brief descriptions of the OpenCitations context, our vision, our value proposition and our relationship with the community and stakeholders.

The Uniqueness of OpenCitations provides the answer to the question ‘Why choose to use OpenCitations?’, and is a detailed presentation of OpenCitations’ benefits.

OpenCitations – Present Status and Future Plans summarizes OpenCitations’ ongoing activities, that can be quickly visualized on our public roadmap. It also introduces the OpenCitations Working Groups, served by the members of the OpenCitations International Advisory Board, which are currently working on the themes of governance evolution and community building, with the common purpose of driving OpenCitations along the path from being a ‘sustainable infrastructure’ (in POSI terms) to being an enduring community led and financially sustained infrastructure.

In fulfilling our mission and reaching our goals, the support and vital interest of our community members is fundamental. We request that you, as a member of our community, provide us with feedback on these documents and the ideas they contain, or indeed to ask for clarifications, to help us improving our mission and our communications to explain it. You can reach us here: contact@opencitations.net.

Thank you!

92 million new citations added to COCI

It’s been a month since the announcement of 1.09 Billion Citations available in the July 2021 release of COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations.  

We’re now proud to announce the September 2021 release of COCI, which is based on open references to works with DOIs within the Crossref dump dated August 2021. This new release extends COCI with more than 92 Million additional citations, giving a total number of more than 1.18 Billion DOI-to-DOI citation links.

This latest release includes citations from the most recent articles published by the American Chemical Society, whose bibliographic references were opened in February 2021. The ACS back number citations will be available in the next COCI release, when a new processing of all the Crossref data will be completed.

You can find more information about COCI in our open-access article 

Ivan Heibi, Silvio Peroni & David Shotton (2019). Software review: COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations. Scientometrics, 121 (2): 1213-1228. DOI: https://doi.org/10.1007/s11192-019-03217-6  

Finally, just a reminder that the bibliographic and citation data in COCI: 

  • can be queried using the OpenCitations Indexes SPARQL endpoint; 
  • can be retrieved by using the COCI REST API
  • can be searched by using the OpenCitations Indexes Search Interface; 
  • are also available as dumps on Figshare in CSV, N-Triples, and Scholix; and 
  • can be freely re-used for any purpose. 

From little acorns . . . A retrospective on OpenCitations

The initial vision

Now that OpenCitations is hosting over one billion freely available scholarly bibliographic citations, this is perhaps an opportune moment to look back to the start of this initiative. A little over eleven years ago, on 24 April 2010, I spoke at the Open Knowledge Foundation Conference, OKCon2010, in London, on the topic

OpenCitations: Publishing Bibliographic Citations as Linked Open Data

I reported that, earlier that same week, I had applied to Jisc for a one-year grant to fund the OpenCitations Project (opencitations.net). Jisc (at that time ‘The JISC’, the Joint Information Systems Committee) was tasked by the UK government, among other things, to support research and development in information technology for the benefit of the academic community.

The purpose of that original OpenCitations R&D project was to develop a prototype in which we:

  • harvested citations from the open access biomedical literature in PubMed Central;
  • described and linked them using CiTO, the Citation Typing Ontology [1];
  • encoded and organized them in an RDF triplestore; and
  • published them as Linked Open Data in the OpenCitations Corpus (OCC).

I told those at the conference that in this demonstration project, with limited JISC funding, we could not hope to “boil the whole ocean”, but that nevertheless there would be substantial benefits from even partial coverage of citation data from the scholarly literature:

  • We could show the way and establish best practice.
  • Despite partial coverage, all key papers would most likely be cited several times.
  • The overall topological structure of the citation network would be revealed.
  • We would create a ‘benchmark’ corpus of high-quality RDF citation data that could be used to develop analytical and visualization tools.
  • We could show the value of open citation data in helping scholars to discover full text articles of all types, and thus encourage subscription-access publishers to release their reference metadata.

The important thing, I said, was to make a start!

The Jisc OpenCitations Project

That JISC grant application was funded, and the project, to last for a year with modest funding of £100K, started in my lab in the Department of Zoology at Oxford University on 1st June 2010, and was subsequently extended for a further six months.

Using data from the Open Access subset of PubMed Central, we created the first prototype release of the OpenCitations Corpus of linked bibliographic citation data, containing 6,529,815 independent bibliographic records of both citing and cited entities, comprising references to ~20% of all post-1980 articles recorded in PubMed, including those to all the most important highly cited papers in every field of biomedical endeavour.

This achievement was almost entirely the result of the excellent work by our chief data wrangler Alex Dutton, whose skill and natural feel for linked data did wonders for this project. Ben O’Steen, Graham Klyne and Alistair Miles made important contributions.

The project also resulted in many other development, described here, most which were developed or at least initiated during a short but wonderfully productive collaboration with Silvio Peroni, who spent six months with me in 2010 as a doctoral student intern from the University of Bologna, to which he subsequently returned to complete his thesis and develop his academic career.

These included:

  • the deconstruction and re-development of the original version of CiTO into a suite of orthogonal and complementary ontologies covering the whole domain of scholarly publishing – the SPAR (Semantic Publishing and Referencing) Ontologies [2, 3];
  • the mapping of various existing metadata schemas into RDF using SPAR, including the DataCite Metadata Schema, and subsequently JATS, now the default NISO standard for XML markup of scholarly documents) [4]; and
  • the initiation of the Semantic Publishing Blog and this OpenCitations Blog.

Life after Jisc – the flowering of OpenCitations

After the Jisc funding ended and I, after a long career in biological teaching and research, formally retired from the Department of Zoology at the Oxford University, members of the initial OpenCitations team moved on to other things. Like so many grant-funded academic project whose initial financial support had dried up, OpenCitations could have foundered at that stage, as an interesting prototype but with too little content to be useful. However, the concept of providing an open alternative to proprietary citation indexes was too important to abandon. But how could it be transitioned into something enduring and useful, particularly when as a matter of principle one had decided that the citation data should be made freely available, thus precluding income generation by charging for ‘premium’ services or the formation of a commercial spin-off?

Finally, I realized that something radical needed to be done to move OpenCitations forward. I had maintained a lively collaboration with Silvio Peroni at the University of Bologna, resulting between 2011 and 2014 in the publication of 18 articles and conference papers concerning the SPAR ontologies, ontology development, documentation and visualization, and related topics, and in 2015 I invited him to start working with me directly on OpenCitations. It was the best decision I could have made. We decided to take the initial concept and re-implement it from the bottom up. OpenCitations gave Silvio a major computer science project to which he could apply his considerable talent, and soon resulted in the development of a revised RDF data model for describing citation data, the OpenCitations Data Model (OCDM) [5] and a suite of new software tools to harvest, organise and publish citations at linked open data [6]. The credit for almost all the subsequent conceptual and technical developments within OpenCitations, which have incrementally led to our present situation, is due to Silvio Peroni, and the scholarly community is indebted to him for the intelligence, skill and diligent application he has given to OpenCitations over the past six years. I am truly honoured to have Silvio as co-Director of OpenCitations, and wish to take this opportunity to acknowledge his contributions and to thank him publicly.

Our work on OpenCitations at that stage, summarized in [7], would not have been possible without the enthusiastic support of Silvio’s senior colleague Fabio Vitali and of the Department of Computer Science and Engineering at the University of Bologna, which not only provided a stimulating environment for Silvio’s post-doctoral work, but also supplied computing services and infrastructure at no charge to OpenCitations. It was also greatly helped by Professor David De Roure of Oxford University, who gave me an academic home and a formal affiliation within the Oxford e-Research Centre after my retirement from the Department of Zoology, which enabled me to continue to hold research grants.

As has been documented in earlier posts in this blog, we greatly benefitted in 2017 from a grant from the Alfred P. Sloan Foundation which enabled us to purchase a new and more powerful computing infrastructure for the sole use of OpenCitations and to extend and improve our software, and subsequently in 2019 by a project grant from the Wellcome Trust to develop the Open Biomedical Citations in Context Corpus, that permitted the extension of OCDM and SPAR for the characterization of in-text references and their textual contexts.

A significant breakthrough came in January 2018 with our decision to treat citations as first-class data entities, each with its own persistent identifier (PID), the Open Citations Identifier (OCI) [8]. This gave Silvio the freedom to envision a new kind of database, a citation index in which each citation had its own metadata, including citation timespan, citation categorization (e.g. self-citation), and of course the DOIs of the citing and cited publications. The creation of this new index was possible only with the incredible effort by Ivan Heibi, who served as a Research Fellow in the project funded by the Alfred P. Sloan Foundation at that time, and who was entirely responsible for developing the first version of the code necessary for creating such a database. Having harvested all the open references from Crossref metadata dumps, Silvio and Ivan created COCI, the OpenCitations Index of Crossref DOI-to-DOI Citations, which immediately became our principal source of open citations, the original OpenCitations Corpus being retained as a ‘sandbox’ in which to experiment with new data representations, for example those required for the Open Biomedical Citations in Context Corpus. Access to COCI was facilitated by Silvio’s development of a REST API, using his software tool RAMOSE (Restful API Manager Over SPARQL Endpoints), which enables the easily configurable deployment of a REST API over any SPARQL endpoint to an RDF triplestore [9]. We were able to organize our all data, both ‘traditional’ and new, and to encode it in RDF, thanks to the comprehensive OpenCitations Data Model [5], itself based on our SPAR Ontologies [3], which we evolved as necessary to accommodate new data representation requirements.

During this period we published a number of definitions, conference papers and journal articles documenting these advances, details of which can be found here. Of these, the most recent canonical publication describing OpenCitations as an infrastructure for open scholarship, and its datasets, tools, services and activities, is Peroni and Shotton (2020) [10]. We also established the Research Centre for Open Scholarly Metadata at the University of Bologna, primarily to handle administrative, financial and academic aspects of OpenCitations activities.

OpenCitations’ future

The problem remained: how to sustain the OpenCitations infrastructure financially. We were greatly helped by Bilder, Lin and Neylon’s formulation of the Principles of Open Scholarly Infrastructures (POSI) [11], in which they clearly pointing out that reliance solely on grant funding for specific projects was not the answer. OpenCitations compliance with POSI is described here. We were thus immensely grateful that SPARC Europe and other institutions had the wisdom to establish SCOSS (The Global Sustainability Coalition for Open Science Services) to facilitate the crowd-sourced financial support of useful open infrastructures by the scholarly community, including academic libraries, government agencies and other stakeholders. OpenCitations applied for SCOSS support in 2019, which led to the selection of OpenCitations for support in the SCOSS second round.

The donations we are now starting to receive from such stakeholders, and the new staff that this funding has recently allowed us to hire, signal the start of our transition from a financially vulnerable academic project to a sustainable open scholarly infrastructure of real value to the community.

The work of opening more of the global citation graph now requires two things:

  • that each publisher takes responsibility for ensuring that the references from all of its journal articles and books are submitted, together with all other bibliographic metadata, to open scholarly bibliographic metadata aggregators such as Crossref and DataCite, from which they can be indexed into open citation indexes of sufficient quality, depth of detail and breadth of coverage that these offer genuine alternatives to the expensive proprietary citation indexing services upon which the academic community presently relies; and
  • that the entire scholarly stakeholder community re-directs a fraction of the enormous sums currently spent on its subscriptions to proprietary bibliographic services in order to support Open Science infrastructures such as OpenCitations that making citations and other forms of scholarly metadata and objects freely available.

References

[1] David Shotton (2010). CiTO, the Citation Typing Ontology. J. Biomedical Semantics 1 (Suppl. 1): S6. http://dx.doi.org/10.1186/2041-1480-1-S1-S6

[2] Silvio Peroni, David Shotton (2012). FaBiO and CiTO: ontologies for describing bibliographic resources and citations. Web Semantics, 17: 33-34. https://doi.org/10.1016/j.websem.2012.08.001, OA at http://speroni.web.cs.unibo.it/publications/peroni-2012-fabio-cito-ontologies.pdf

[3] Silvio Peroni, David Shotton (2018). The SPAR Ontologies. In Proceedings of the 17th International Semantic Web Conference (ISWC 2018): 119-136. https://doi.org/10.1007/978-3-030-00668-6_8

[4] Peroni S, Lapeyre DA and Shotton D (2012) From Markup to Linked Data: Mapping NISO JATS v1.0 to RDF using the SPAR (Semantic Publishing and Referencing) Ontologies. Proc. 2012 JATS Conference, National Library of Medicine, Bethesda, Maryland, USA (October 2012): 16-17. http://www.ncbi.nlm.nih.gov/books/NBK100491/

[5] Marilena Daquino, Silvio Peroni , David Shotton (2020). The OpenCitations Data Model. Figshare. https://doi.org/10.6084/m9.figshare.3443876.v7

[6] Silvio Peroni, David Shotton, Fabio Vitali (2017). One Year of the OpenCitations Corpus: Releasing RDF-based scholarly citation data into the Public Domain. In The Semantic Web – ISWC 2017 (Lecture Notes in Computer Science Vol. 10588, pp. 184–192). Springer, Cham. https://doi.org/10.1007/978-3-319-68204-4_19

[7] Silvio Peroni, Alexander Dutton, Tanya Gray, David Shotton (2015). Setting our bibliographic references free: towards open citation data. Journal of Documentation, 71 (2): 253-277. http://dx.doi.org/10.1108/JD-12-2013-0166, OA at http://speroni.web.cs.unibo.it/publications/peroni-2015-setting-bibliographic-references.pdf

[8] Silvio Peroni, David Shotton (2019). Open Citation Identifier: Definition. Figshare. https://doi.org/10.6084/m9.figshare.7127816

[9] Daquino, M., Heibi, I., Peroni, S., & Shotton, D. (2021). Creating Restful APIs over SPARQL endpoints with RAMOSE. Semantic Web. http://arxiv.org/abs/2007.16079

[10] Silvio Peroni, David Shotton (2020). OpenCitations, an infrastructure organization for open scholarship. Quantitative Science Studies, 1(1): 428-444. https://doi.org/10.1162/qss_a_00023

[11] Geoffrey Bilder, Jenny Lin, Cameron Neylon (2015). Principles for Open Scholarly Infrastructure. http://dx.doi.org/10.6084/m9.figshare.1314859

Crossing a significant threshold: more than one billion citations now available in COCI!

“The competitive benefits of closing access to citation data diminish with each new citation released to the public domain, but the benefits of open data remain. Going forward, citation data is almost completely public domain”.

With these words, from the article “A tipping point for open citations data” (July 15, 2021), Ian Hutchins celebrated the threshold crossing of one billion citations on public-domain databases in February 2021.

Now, a new significant milestone has been reached. We are enthusiastic to announce that COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations has just been extended with 334 million additional citations. Its most recent release, the COCI July 2021 release, now contains a total of 1.09 billion DOI-to-DOI citation links derived from open references within Crossref,which includes the references of articles deposited or opened in Crossref between November 2020 and January 2021.

These numbers make us proud, and confirm the essential value of the Initiative for Open Citations (I4OC). Since 2018, the mission of I4OC has been to persuade publishers to provide open citation data by means of the Crossref platform. The I4OC untiring commitment has led the major academic publishers to a progressive change of heart regarding open citations, and the scholarly community to a deeper interest in this openness.

These factors contributed to the creation of COCI in 2018, the first open citation index created by OpenCitations, in which we applied the concept of citations as first-class data entities (Heibi I., Peroni S., Shotton D., 2019). Over the last three years, COCI has been extended in a series of releases, by harvesting citations mostly from Crossref data dumps, starting from an initial coverage of 300 million citations (First release).

A crucial event that preceded (and delayed!) this latest COCI release was Elsevier’s endorsement in the DORA Declaration on Research Assessment in December 2020, thereby making “reference lists for all articles published in Elsevier journals openly available via Crossref so they can be available for reuse. This means other important initiatives like I4OC can draw on this metadata”. As described in our previous post, Elsevier’s welcome commitment led to the opening of many previously closed references from its numerous academic journals submitted to Crossref. Now, after an extended period of data ingestion and processing, all these newly opened Elsevier references are available at OpenCitations within COCI.

Elsevier’s involvement has both an effective and a symbolical value. Even if publishing more than one billion citations is a thrilling achievement, and – as Hutchins wrote – we are now at a tipping point with regard to open citations data, this milestone is not the last stop. Together with the other organizations and projects that participate in the Initiative for Open Citations, we will keep claiming the urgency for the remaining academic publishers to join our cause, and sharing our values with the whole academic community to make all existing citations data freely open and accessible. Recalling what Dario Taraborelli wrote in the conclusion of his article “The citation graph is one of humankind’s most important intellectual achievements“, “the world is waiting for the citation graph to become a public good”.

Search OpenEdition Search

You will be redirected to OpenEdition Search