Digital resources in the Social Sciences and Humanities OpenEdition Our platforms OpenEdition Books OpenEdition Journals Hypotheses Calenda Libraries OpenEdition Freemium Follow us

Academia’s missing references

No-one is quite sure of the total number of scholarly publications within the global corpus. Indeed that number will be strongly influenced by the degree to which, in addition to books and journal articles, one includes within the definition of scholarly publications ‘grey literature’ such as reports published by official bodies, patents, etc. Consequentially, the total number of scholarly references within those publications is also unknown, and this number too will vary according to the inclusion criteria chosen. Furthermore, Crossref Event Data and similar datasets recording social media mentions of journal articles in blog posts and tweets extends the concept of a reference beyond that used in ‘conventional’ citation indexes such as COCI.

We celebrate the fact that well over one billion bibliographic citations are now openly available under CC0 waivers in NIH OCC (the National Institutes of Health Open Citation Collection) [1,2] and COCI (the OpenCitations Index of Crossref DOI-to-DOI Citations) [3]. Despite present gaps in their coverage, they include references to all the most important publications within the global corpus, because these will all have been cited multiple times.

Open references available from Crossref and other aggregators and indexes

Crossref, with over 1.6 billion open references, is the largest single source of such bibliographic metadata. Significant numbers of references are also available in a variety of other databases, repositories and indexes.

NIH OCC (the National Institutes of Health Open Citation Collection) is a merger of several citation databases, drawing on PubMed for crucial article metadata, and augmenting this with information from full-text articles that have been made freely available on the internet [1]. The CiteSeerX database, the arXiv preprint repository, and the Dryad data repository are examples of different types of infrastructure that also publish open bibliographic references, while there is further availability of article references from open aggregators such as DataCite and Wikidata. These may either use their own DOIs, DOIs from the Crossref DOI registration agency, or no DOIs at all. Either way, those references will not appear in Crossref.

What is lacking is semantic coherence and interoperability between these sources, permitting federated queries across them. This makes difficult the task of obtaining a comprehensive overview of the availability of open bibliographic references.

However, there are even more citations that are not yet freely and easily available anywhere in bulk, relating to the reference lists within publications of a number of distinct types. This blog post explores academia’s missing references – those that have not yet been documented within open freely accessible citation indexes – and what might be done to bring these into the public domain.

1 References that are closed at Crossref

Eight years ago, I wrote

“In this open-access age, it is a scandal that reference lists from journal articles — core elements of scholarly communication that permit the attribution of credit and integrate our independent research endeavours — are not readily and freely available for use by all scholars.” [4]

I stand by that statement, and, through OpenCitations [5], I have been working with colleagues to rectify the situation, as I described in my previous post. The Initiative for Open Citations (I4OC) can rightly be applauded for its part in encouraging almost all the major academic publishers who deposit references at Crossref to make them open.

The only major scholarly publisher not to be listed as an I4OC Participating Publisher is the Institute of Electrical and Electronics Engineers (IEEE), that, having deposited at Crossref reference lists for 58% of its preprints and publications, persists in keeping these deposited references closed and unavailable for indexing and re-use. It is to be hoped that IEEE will now realize that what it looses by not having references openly available outweighs any benefits it might have received from keeping them closed, and will join its fellow publishers in ensuring that its Crossref-deposited references are made open, both for its current issues and for its back-number journal articles. I thus call again upon IEEE to change its present position, as both Elsevier and the American Chemical Society had the courage to do recently, and to instruct Crossref to open all IEEE references. A single email to Crossref Support, with the instruction “Open all references”, is all it would take!

References in Crossref are either open, ‘limited’ or closed. Limited references are available to those subscribing to Crossref Metadata Plus, which includes OpenCitations, but not to the general public. Closed references are not freely available to anyone. The following table shows the number of Crossref works in each category, and the number of references within those categories.


WorksReferencesAverage references per work
Total126,627,618

Without references70,477,843
(55.7% of total)


With references56,149,775
(44.3% of total)
1,734,831,31130.9
Open49,274,155
(38.9% of total)
1,605,120,22932.6
Limited2,933,323
(2.3% of total)
66,459,70422.7
Closed3,942,297
(3.1% of total)
63,251,37816.0
Table 1. Numbers of works and their references recorded in Crossref on 31 July 2021.

2 References not deposited at Crossref for publications with Crossref DOIs

The number of works with Crossref DOIs that lack submitted references is surprisingly large. As of 31 July 2021, 70,477,843 publications (55.7% of all works recorded at Crossref) lacked deposited references (Table 1).

Crossref classifies all types of journal content, including editorials, book reviews and letters, as “journal articles”, thus some of these works without deposited references genuinely lack them. However, the majority are conventional journal articles and books with reference lists that the publishers have simply not deposited at Crossref along with the other metadata for these works.

The average number of references per Crossref work with deposited references is 30.9 (Table 1). If, to make allowance for those works that genuinely lack references, we assume a conservative average of 25 references per work for the 70,477,843 works lacking deposited references, this means that there are over 1.75 billion references within these works that have not been deposited at Crossref, and thus are not conveniently available for indexing and reuse.

These missing references relate both large publishers that have submitted references for only some of their publications, and small publishers that perhaps lack, or think they lack, the resources to deposit any reference lists in addition to the other metadata they are already sending to Crossref for each of their DOIs. However, there are several easy methods for depositing reference lists, as detailed by Crossref here. So I encourage all publishers who are supportive of Open Science to update their procedures and commence or complete the deposition of their publication reference lists at Crossref, starting with their current issues. Crossref Support will provide assistance if required. Note that a publisher does not have to subscribe to the Crossref Cited-by service to deposit its references!

3 Citations missing in COCI: open references in Crossref to publications lacking DOIs

COCI is the OpenCitations Index of Crossref DOI-to-DOI Citations, and, as the name suggests, it indexes Crossref open references from works with Crossref DOIs to other works that have DOIs [3]. It therefore does not index open references in Crossref to works that, for whatever reason, lack DOIs.

The most recent (September 2021) release of COCI, based on the August Crossref dump, contains 1,186,958,898 citations between 69,074,291 unique work, comprising 51,103,720 citing bibliographic resources bearing Crossref DOIs and 56,105,783 cited bibliographic resources. Of the cited bibliographic resources, 38,135,212 bear a DOI issued by Crossref and have open or limited references (thus also being COCI citing resources), while 17,970,571 either have a Crossref DOI but lack open or limited references or have a DOI issued by another DOI registration agency such as DataCite (thus not being COCI citing resources).

Note that in Crossref, the ratio of works with open or limited references to works without open or limited references is 0.7:1 (Table 1). However, in COCI, the ratio of cited works with Crossref DOIs containing open or limited references to all other cited works is 2.1:1. Thus works with Crossref DOIs containing open or limited references are three times more likely to be cited than works that either have a Crossref DOI but lacking open or limited references or have a DOI issued by another DOI registration agency. This is most likely because the most important journals from almost all the larger publishers now have open references. However, it is still a remarkable ratio.

Because references to works lacking DOIs are not included in COCI, the average number of bibliographic references per citing article in COCI is only 23.2, in contrast to the numbers of references to works of all types given in Table 1.

From these data, it can be seen that there are over 480 million open Crossref references to a wide variety of works lacking DOIs that OpenCitations does not index in COCI. This is because of an intentional and fundamental limitation in the structure of the Open Citation Identifier (OCI) [6], requiring both citing and cited publications to have identifiers of the same type, that lies at the heart of the functionality of our OpenCitations Indexes.

OpenCitations is currently developing a solution that, without compromising that intentional design limitation in OCIs, will nevertheless permit us to index and publish these ‘missing’ references as Linked Open Data. We will report on this development in due course.

Crossref Event Data is a Crossref service / database that records mentions of publications bearing Crossref DOIs in social media including tweets and blog posts, and in other non-traditional citation sources such as news items and Wikipedia articles. From today (Thursday 23rd September 2021), Crossref Event Data will start to include in its holdings open references from publications bearing Crossref DOIs to other publications bearing DOIs. Limited and closed references will not be included. Initially, open references from current publications will be included in Crossref Event Data, with open references from older works with DOIs being added later. In that respect it will come to resemble COCI, except that COCI also included ‘limited’ references, treats citations as first-class data entities with their own identifiers, and makes its citation data available in RDF as Linked Open Data, as well as via a REST API. Subsequently, Crossref Event Data will also record references to publications with other forms of identifier, as OpenCitations also plans to do.

4 References available on publishers’ web sites

Many publishers, particularly those of Open Access works, already make their publication reference lists openly available on their own web sites. While this is commendable, it is not sufficient, since, if scholarly references are not made available in a centralized aggregator such as Crossref from which they can be conveniently harvested in bulk for analysis and re-use, they are much more difficult to access.

Scraping references from the HTML of individual web sites is difficult, time-consuming and liable to be incomplete. While Microsoft Academic achieved considerable success in scraping references from publishers’ web sites, possible because of the special relationships these publishers have with the Microsoft search engine Bing, this service will unfortunately soon no longer be available, illustrating a problem inherent with scholarly infrastructures provided by commercial companies that do not adopt the Principles of Open Scholarly Infrastructures.

A consequence is that such publications will become increasingly ‘invisible’, as bibliographic and analytical services come to rely more and more on centrally available data.

6 References within PDFs of scholarly works lacking DOIs

There is a large but unknown quantity of reference-containing books, academic reports, patents and journal articles from publishers that, for their own good reasons, choose not to use DOIs. The text of many of these publications is already available in a marked-up machine-readable format such as JATS, used in preparation for publication, from which the reference lists could easily be extracted. Other publications are only available as PDFs, both as published Versions of Record, or as preprints deposited in a variety of preprint repositories such as arXiv or CORE. Mining reference lists out of PDFs required expertise in text mining and AI technologies, and is labour-intensive, since it usually required the tuning of extraction algorithms to handle the particular styles and formatting of individual journals, one at a time. Two stages are involved: first, the recognition and extraction of the text of the individual references from the PDF, and second the parsing of each text string into the component parts of the reference (author names, title, publication year, etc.) A considerable number of the citations within NIH-OCC have been obtained in this manner [1], commercial companies such as Lexical Intelligence specialize in this area, and publicly available software such as GROBID is available for the purpose. However, the overall task of extracting ‘missing’ academic references from the global PDF corpus is daunting in magnitude and would require a well-funded organization.

The correct way to proceed would be for each publisher to take responsibility for liberating the references of their own publications, whether or not the publications themselves are open access, and whether or not these references are already available in a marked-up machine-readable format or only within PDF documents. Then, if the publisher still chose not to use DOIs and to submit these metadata to Crossref, these references could be submitted directly to OpenCitations for aggregation and publication as Linked Open Data.

Conclusion

From the foregoing discussion it is clear that the academic community has a long way to go before the majority of scholarly citations, the products of their own labours, are openly available for analysis and re-use. We at OpenCitations are working to address these issues and to publish more of these missing citations. However, completion of the task will require a coordinated collaborative international effort.

Are you willing to be involved?

References

[1] B. Ian Hutchins et al. (2019). The NIH Open Citation Collection: A public access, broad coverage resource. PLoS Biol. 17 (10): e3000385. https://doi.org/10.1371/journal.pbio.3000385

[2] B. Ian Hutchins (2021). A tipping point for open citation data. Quantitative Science Studies 2 (2): 433–437. https://doi.org/10.1162/qss_c_00138

[3] Ivan Heibi, Silvio Peroni, David Shotton (2019). Software review: COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations. Scientometrics 121 (2): 1213-1228. https://doi.org/10.1007/s11192-019-03217-6

[4] David Shotton (2013). Open citations. Nature, 502 (7471): 295-297. http://dx.doi.org/10.1038/502295a

[5] Silvio Peroni, David Shotton (2020). OpenCitations, an infrastructure organization for open scholarship. Quantitative Science Studies, 1(1): 428-444. https://doi.org/10.1162/qss_a_00023

[6] Silvio Peroni, David Shotton (2019). Open Citation Identifier: Definition. Figshare. https://doi.org/10.6084/m9.figshare.7127816

92 million new citations added to COCI

It’s been a month since the announcement of 1.09 Billion Citations available in the July 2021 release of COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations.  

We’re now proud to announce the September 2021 release of COCI, which is based on open references to works with DOIs within the Crossref dump dated August 2021. This new release extends COCI with more than 92 Million additional citations, giving a total number of more than 1.18 Billion DOI-to-DOI citation links.

This latest release includes citations from the most recent articles published by the American Chemical Society, whose bibliographic references were opened in February 2021. The ACS back number citations will be available in the next COCI release, when a new processing of all the Crossref data will be completed.

You can find more information about COCI in our open-access article 

Ivan Heibi, Silvio Peroni & David Shotton (2019). Software review: COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations. Scientometrics, 121 (2): 1213-1228. DOI: https://doi.org/10.1007/s11192-019-03217-6  

Finally, just a reminder that the bibliographic and citation data in COCI: 

  • can be queried using the OpenCitations Indexes SPARQL endpoint; 
  • can be retrieved by using the COCI REST API
  • can be searched by using the OpenCitations Indexes Search Interface; 
  • are also available as dumps on Figshare in CSV, N-Triples, and Scholix; and 
  • can be freely re-used for any purpose. 

From little acorns . . . A retrospective on OpenCitations

The initial vision

Now that OpenCitations is hosting over one billion freely available scholarly bibliographic citations, this is perhaps an opportune moment to look back to the start of this initiative. A little over eleven years ago, on 24 April 2010, I spoke at the Open Knowledge Foundation Conference, OKCon2010, in London, on the topic

OpenCitations: Publishing Bibliographic Citations as Linked Open Data

I reported that, earlier that same week, I had applied to Jisc for a one-year grant to fund the OpenCitations Project (opencitations.net). Jisc (at that time ‘The JISC’, the Joint Information Systems Committee) was tasked by the UK government, among other things, to support research and development in information technology for the benefit of the academic community.

The purpose of that original OpenCitations R&D project was to develop a prototype in which we:

  • harvested citations from the open access biomedical literature in PubMed Central;
  • described and linked them using CiTO, the Citation Typing Ontology [1];
  • encoded and organized them in an RDF triplestore; and
  • published them as Linked Open Data in the OpenCitations Corpus (OCC).

I told those at the conference that in this demonstration project, with limited JISC funding, we could not hope to “boil the whole ocean”, but that nevertheless there would be substantial benefits from even partial coverage of citation data from the scholarly literature:

  • We could show the way and establish best practice.
  • Despite partial coverage, all key papers would most likely be cited several times.
  • The overall topological structure of the citation network would be revealed.
  • We would create a ‘benchmark’ corpus of high-quality RDF citation data that could be used to develop analytical and visualization tools.
  • We could show the value of open citation data in helping scholars to discover full text articles of all types, and thus encourage subscription-access publishers to release their reference metadata.

The important thing, I said, was to make a start!

The Jisc OpenCitations Project

That JISC grant application was funded, and the project, to last for a year with modest funding of £100K, started in my lab in the Department of Zoology at Oxford University on 1st June 2010, and was subsequently extended for a further six months.

Using data from the Open Access subset of PubMed Central, we created the first prototype release of the OpenCitations Corpus of linked bibliographic citation data, containing 6,529,815 independent bibliographic records of both citing and cited entities, comprising references to ~20% of all post-1980 articles recorded in PubMed, including those to all the most important highly cited papers in every field of biomedical endeavour.

This achievement was almost entirely the result of the excellent work by our chief data wrangler Alex Dutton, whose skill and natural feel for linked data did wonders for this project. Ben O’Steen, Graham Klyne and Alistair Miles made important contributions.

The project also resulted in many other development, described here, most which were developed or at least initiated during a short but wonderfully productive collaboration with Silvio Peroni, who spent six months with me in 2010 as a doctoral student intern from the University of Bologna, to which he subsequently returned to complete his thesis and develop his academic career.

These included:

  • the deconstruction and re-development of the original version of CiTO into a suite of orthogonal and complementary ontologies covering the whole domain of scholarly publishing – the SPAR (Semantic Publishing and Referencing) Ontologies [2, 3];
  • the mapping of various existing metadata schemas into RDF using SPAR, including the DataCite Metadata Schema, and subsequently JATS, now the default NISO standard for XML markup of scholarly documents) [4]; and
  • the initiation of the Semantic Publishing Blog and this OpenCitations Blog.

Life after Jisc – the flowering of OpenCitations

After the Jisc funding ended and I, after a long career in biological teaching and research, formally retired from the Department of Zoology at the Oxford University, members of the initial OpenCitations team moved on to other things. Like so many grant-funded academic project whose initial financial support had dried up, OpenCitations could have foundered at that stage, as an interesting prototype but with too little content to be useful. However, the concept of providing an open alternative to proprietary citation indexes was too important to abandon. But how could it be transitioned into something enduring and useful, particularly when as a matter of principle one had decided that the citation data should be made freely available, thus precluding income generation by charging for ‘premium’ services or the formation of a commercial spin-off?

Finally, I realized that something radical needed to be done to move OpenCitations forward. I had maintained a lively collaboration with Silvio Peroni at the University of Bologna, resulting between 2011 and 2014 in the publication of 18 articles and conference papers concerning the SPAR ontologies, ontology development, documentation and visualization, and related topics, and in 2015 I invited him to start working with me directly on OpenCitations. It was the best decision I could have made. We decided to take the initial concept and re-implement it from the bottom up. OpenCitations gave Silvio a major computer science project to which he could apply his considerable talent, and soon resulted in the development of a revised RDF data model for describing citation data, the OpenCitations Data Model (OCDM) [5] and a suite of new software tools to harvest, organise and publish citations at linked open data [6]. The credit for almost all the subsequent conceptual and technical developments within OpenCitations, which have incrementally led to our present situation, is due to Silvio Peroni, and the scholarly community is indebted to him for the intelligence, skill and diligent application he has given to OpenCitations over the past six years. I am truly honoured to have Silvio as co-Director of OpenCitations, and wish to take this opportunity to acknowledge his contributions and to thank him publicly.

Our work on OpenCitations at that stage, summarized in [7], would not have been possible without the enthusiastic support of Silvio’s senior colleague Fabio Vitali and of the Department of Computer Science and Engineering at the University of Bologna, which not only provided a stimulating environment for Silvio’s post-doctoral work, but also supplied computing services and infrastructure at no charge to OpenCitations. It was also greatly helped by Professor David De Roure of Oxford University, who gave me an academic home and a formal affiliation within the Oxford e-Research Centre after my retirement from the Department of Zoology, which enabled me to continue to hold research grants.

As has been documented in earlier posts in this blog, we greatly benefitted in 2017 from a grant from the Alfred P. Sloan Foundation which enabled us to purchase a new and more powerful computing infrastructure for the sole use of OpenCitations and to extend and improve our software, and subsequently in 2019 by a project grant from the Wellcome Trust to develop the Open Biomedical Citations in Context Corpus, that permitted the extension of OCDM and SPAR for the characterization of in-text references and their textual contexts.

A significant breakthrough came in January 2018 with our decision to treat citations as first-class data entities, each with its own persistent identifier (PID), the Open Citations Identifier (OCI) [8]. This gave Silvio the freedom to envision a new kind of database, a citation index in which each citation had its own metadata, including citation timespan, citation categorization (e.g. self-citation), and of course the DOIs of the citing and cited publications. The creation of this new index was possible only with the incredible effort by Ivan Heibi, who served as a Research Fellow in the project funded by the Alfred P. Sloan Foundation at that time, and who was entirely responsible for developing the first version of the code necessary for creating such a database. Having harvested all the open references from Crossref metadata dumps, Silvio and Ivan created COCI, the OpenCitations Index of Crossref DOI-to-DOI Citations, which immediately became our principal source of open citations, the original OpenCitations Corpus being retained as a ‘sandbox’ in which to experiment with new data representations, for example those required for the Open Biomedical Citations in Context Corpus. Access to COCI was facilitated by Silvio’s development of a REST API, using his software tool RAMOSE (Restful API Manager Over SPARQL Endpoints), which enables the easily configurable deployment of a REST API over any SPARQL endpoint to an RDF triplestore [9]. We were able to organize our all data, both ‘traditional’ and new, and to encode it in RDF, thanks to the comprehensive OpenCitations Data Model [5], itself based on our SPAR Ontologies [3], which we evolved as necessary to accommodate new data representation requirements.

During this period we published a number of definitions, conference papers and journal articles documenting these advances, details of which can be found here. Of these, the most recent canonical publication describing OpenCitations as an infrastructure for open scholarship, and its datasets, tools, services and activities, is Peroni and Shotton (2020) [10]. We also established the Research Centre for Open Scholarly Metadata at the University of Bologna, primarily to handle administrative, financial and academic aspects of OpenCitations activities.

OpenCitations’ future

The problem remained: how to sustain the OpenCitations infrastructure financially. We were greatly helped by Bilder, Lin and Neylon’s formulation of the Principles of Open Scholarly Infrastructures (POSI) [11], in which they clearly pointing out that reliance solely on grant funding for specific projects was not the answer. OpenCitations compliance with POSI is described here. We were thus immensely grateful that SPARC Europe and other institutions had the wisdom to establish SCOSS (The Global Sustainability Coalition for Open Science Services) to facilitate the crowd-sourced financial support of useful open infrastructures by the scholarly community, including academic libraries, government agencies and other stakeholders. OpenCitations applied for SCOSS support in 2019, which led to the selection of OpenCitations for support in the SCOSS second round.

The donations we are now starting to receive from such stakeholders, and the new staff that this funding has recently allowed us to hire, signal the start of our transition from a financially vulnerable academic project to a sustainable open scholarly infrastructure of real value to the community.

The work of opening more of the global citation graph now requires two things:

  • that each publisher takes responsibility for ensuring that the references from all of its journal articles and books are submitted, together with all other bibliographic metadata, to open scholarly bibliographic metadata aggregators such as Crossref and DataCite, from which they can be indexed into open citation indexes of sufficient quality, depth of detail and breadth of coverage that these offer genuine alternatives to the expensive proprietary citation indexing services upon which the academic community presently relies; and
  • that the entire scholarly stakeholder community re-directs a fraction of the enormous sums currently spent on its subscriptions to proprietary bibliographic services in order to support Open Science infrastructures such as OpenCitations that making citations and other forms of scholarly metadata and objects freely available.

References

[1] David Shotton (2010). CiTO, the Citation Typing Ontology. J. Biomedical Semantics 1 (Suppl. 1): S6. http://dx.doi.org/10.1186/2041-1480-1-S1-S6

[2] Silvio Peroni, David Shotton (2012). FaBiO and CiTO: ontologies for describing bibliographic resources and citations. Web Semantics, 17: 33-34. https://doi.org/10.1016/j.websem.2012.08.001, OA at http://speroni.web.cs.unibo.it/publications/peroni-2012-fabio-cito-ontologies.pdf

[3] Silvio Peroni, David Shotton (2018). The SPAR Ontologies. In Proceedings of the 17th International Semantic Web Conference (ISWC 2018): 119-136. https://doi.org/10.1007/978-3-030-00668-6_8

[4] Peroni S, Lapeyre DA and Shotton D (2012) From Markup to Linked Data: Mapping NISO JATS v1.0 to RDF using the SPAR (Semantic Publishing and Referencing) Ontologies. Proc. 2012 JATS Conference, National Library of Medicine, Bethesda, Maryland, USA (October 2012): 16-17. http://www.ncbi.nlm.nih.gov/books/NBK100491/

[5] Marilena Daquino, Silvio Peroni , David Shotton (2020). The OpenCitations Data Model. Figshare. https://doi.org/10.6084/m9.figshare.3443876.v7

[6] Silvio Peroni, David Shotton, Fabio Vitali (2017). One Year of the OpenCitations Corpus: Releasing RDF-based scholarly citation data into the Public Domain. In The Semantic Web – ISWC 2017 (Lecture Notes in Computer Science Vol. 10588, pp. 184–192). Springer, Cham. https://doi.org/10.1007/978-3-319-68204-4_19

[7] Silvio Peroni, Alexander Dutton, Tanya Gray, David Shotton (2015). Setting our bibliographic references free: towards open citation data. Journal of Documentation, 71 (2): 253-277. http://dx.doi.org/10.1108/JD-12-2013-0166, OA at http://speroni.web.cs.unibo.it/publications/peroni-2015-setting-bibliographic-references.pdf

[8] Silvio Peroni, David Shotton (2019). Open Citation Identifier: Definition. Figshare. https://doi.org/10.6084/m9.figshare.7127816

[9] Daquino, M., Heibi, I., Peroni, S., & Shotton, D. (2021). Creating Restful APIs over SPARQL endpoints with RAMOSE. Semantic Web. http://arxiv.org/abs/2007.16079

[10] Silvio Peroni, David Shotton (2020). OpenCitations, an infrastructure organization for open scholarship. Quantitative Science Studies, 1(1): 428-444. https://doi.org/10.1162/qss_a_00023

[11] Geoffrey Bilder, Jenny Lin, Cameron Neylon (2015). Principles for Open Scholarly Infrastructure. http://dx.doi.org/10.6084/m9.figshare.1314859

Crossing a significant threshold: more than one billion citations now available in COCI!

“The competitive benefits of closing access to citation data diminish with each new citation released to the public domain, but the benefits of open data remain. Going forward, citation data is almost completely public domain”.

With these words, from the article “A tipping point for open citations data” (July 15, 2021), Ian Hutchins celebrated the threshold crossing of one billion citations on public-domain databases in February 2021.

Now, a new significant milestone has been reached. We are enthusiastic to announce that COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations has just been extended with 334 million additional citations. Its most recent release, the COCI July 2021 release, now contains a total of 1.09 billion DOI-to-DOI citation links derived from open references within Crossref,which includes the references of articles deposited or opened in Crossref between November 2020 and January 2021.

These numbers make us proud, and confirm the essential value of the Initiative for Open Citations (I4OC). Since 2018, the mission of I4OC has been to persuade publishers to provide open citation data by means of the Crossref platform. The I4OC untiring commitment has led the major academic publishers to a progressive change of heart regarding open citations, and the scholarly community to a deeper interest in this openness.

These factors contributed to the creation of COCI in 2018, the first open citation index created by OpenCitations, in which we applied the concept of citations as first-class data entities (Heibi I., Peroni S., Shotton D., 2019). Over the last three years, COCI has been extended in a series of releases, by harvesting citations mostly from Crossref data dumps, starting from an initial coverage of 300 million citations (First release).

A crucial event that preceded (and delayed!) this latest COCI release was Elsevier’s endorsement in the DORA Declaration on Research Assessment in December 2020, thereby making “reference lists for all articles published in Elsevier journals openly available via Crossref so they can be available for reuse. This means other important initiatives like I4OC can draw on this metadata”. As described in our previous post, Elsevier’s welcome commitment led to the opening of many previously closed references from its numerous academic journals submitted to Crossref. Now, after an extended period of data ingestion and processing, all these newly opened Elsevier references are available at OpenCitations within COCI.

Elsevier’s involvement has both an effective and a symbolical value. Even if publishing more than one billion citations is a thrilling achievement, and – as Hutchins wrote – we are now at a tipping point with regard to open citations data, this milestone is not the last stop. Together with the other organizations and projects that participate in the Initiative for Open Citations, we will keep claiming the urgency for the remaining academic publishers to join our cause, and sharing our values with the whole academic community to make all existing citations data freely open and accessible. Recalling what Dario Taraborelli wrote in the conclusion of his article “The citation graph is one of humankind’s most important intellectual achievements“, “the world is waiting for the citation graph to become a public good”.

Elsevier endorses DORA and opens its journal article reference lists

We congratulate and thank Elsevier, the world’s largest academic publisher, for endorsing the DORA Declaration on Research Assessment (https://sfdora.org/), thereby joining the hundreds of other publishers and scientific organizations which have endorsed DORA over the previous eight years, and also for making a commitment to open the references from all its journal articles submitted to Crossref. The text of Elsevier’s endorsement, dated 16th December 2020, is to be found at https://www.elsevier.com/connect/advancing-responsible-research-assessment, and includes the statement:

“We will make reference lists for all articles published in Elsevier journals openly available via Crossref so they can be available for reuse. This means other important initiatives like I4OC can draw on this metadata.”

Particular thanks are due to Kumsal Bayazit, Elsevier’s CEO, and to Andrew Plume, head of Elsevier’s International Center for the Study of Research (ICSR), for spearheading this change in stance on the part of Elsevier, which until this week has been alone among the major scholarly publishers in keeping its reference lists at Crossref closed, for which it has attracted much criticism from the academic community.

This change of heart on the part of Elsevier now means that by next spring, after Crossref has had a chance to implement this change in status over the large corpus of Elsevier journal metadata, the reference lists of articles in the vast majority of the world’s academic journals will be open, enabling such metadata to be used to enhance publication discovery and enable transparent research assessment.  The I4OC web site and COCI, OpenCitations Index of Crossref open DOI-to-DOI citations, will reflect this change once it has happened.

The Institute of Electrical and Electronics Engineers (IEEE), the American Chemical Society (ACS), and the University of Chicago Press now stand alone as the only significant scholarly publishers who choose not to make their publication reference lists open.

The Initiative for Open Abstracts is launched

OpenCitations is proud to be part of the launch of the Initiative for Open Abstracts, a new cross-publisher initiative calling for the unrestricted availability of abstracts to boost the discovery of research.

The Initiative for Open Abstracts (I4OA), launched on September 24th, calls on all scholarly publishers to open their abstracts, and specifically to deposit them with Crossref, in order to facilitate large-scale access and promote discovery of critical research.

Making abstracts openly available helps scholarly publishers to maximize the visibility and reach of their journals and books. Open abstracts make it easier for scholars to discover, read and then cite these publications; promotes their inclusion in systematic reviews; expands and simplifies the use of text mining, natural language processing and artificial intelligence techniques in bibliometric analyses; and facilitates scholarship across all disciplines by those without subscription access to commercial bibliographic services.

Many abstracts are already available in various bibliographic databases, but these sources have limitations, for example because they require a subscription, are not machine-accessible, or are restricted to a specific discipline. I4OA thus calls on all scholarly publishers using Crossref DOIs to make their abstracts openly available by depositing them with Crossref. This can be done as part of established workflows that publishers already have in place for submitting publication metadata to Crossref.

As detailed on the I4OA web site at https://i4oa.org, 40 publishers have already agreed to support I4OA and to make their abstracts openly available. I4OA is also supported by 56 other stakeholders including research funders, libraries and library associations, infrastructure providers, and open science organizations, demonstrating the importance and relevance of this Initiative to the scholarly community. The launch press release is available at https://i4oa.org/press.html#pressrelease.

I4OA was inspired by the success of the Initiative for Open Citations (I4OC, https://i4oc.org/), which encourages the submission of references to Crossref. Since the launch of I4OC in 2017, over two thousand scholarly publishers have chosen to make the reference lists of their journal articles and book chapters openly available through Crossref. I4OA aims to replicate the success of I4OC by achieving a rapid jump in the open availability of scholarly abstracts via Crossref.

Further information may be obtained from the I4OA web site at https://i4oa.org, from the I4OA poster at https://doi.org/10.5281/zenodo.4047454, by attending the free I4OA launch webinar on October 5th 2020 at 4 pm CEST (register at https://tinyurl.com/i4oa-webinar), by emailing Professor Ludo Waltman (CWTS, Leiden University; coordinator of I4OA) at openabstracts@gmail.com, or by following @open_abstracts on Twitter.

More than 733M citations now available in COCI

Today, we have published the bi-monthly release of COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations. In this latest release (dated 6 September 2020), we extended COCI with more than 11 million additional citations. Now, COCI contains more than 733 million DOI-to-DOI citation links between more than 59.4 million bibliographic entities.

These new citations were harvested from the most recent Crossref data dump, downloaded on 19 August 2020, which includes the references of articles deposited in Crossref between 4 June 2020 and 3 August 2020.

We remind you that COCI has been fully described in our open-access article

Ivan Heibi, Silvio Peroni & David Shotton (2019). Software review: COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations. Scientometrics, 121 (2): 1213-1228. DOI: https://doi.org/10.1007/s11192-019-03217-6

and that all the bibliographic and citation data in COCI:

A final remark: COCI and other OpenCitations services will be the topic of a presentation that we will have in the context of the Workshop on Open Citations and Open Scholarly Metadata 2020. The workshop is a 3-hour event for researchers, scholarly publishers, funders, policymakers, and opening citations advocates, interested in the creation, reuse, and improvement, of open citation data and open scholarly metadata, with invited speakers. No registration is needed to follow it – we hope to see you there on 9 September at 15:00 CET!

Where you can follow the Workshop on Open Citations and Open Scholarly Metadata 2020.

OpenCitations described

OpenCitations is an infrastructure organization for open scholarship dedicated to the publication of open bibliographic and citation data. We at OpenCitations are proud to announce the publication, in the first issue of Quantitative Science Studies, of a canonical paper in which we introduce and describe OpenCitations and outline its achievements and goals [1].

Here, I outline the contents of our paper, and provide definitive links on the topics described. Many of these topics have been the subjects of earlier blog posts.

This paper appears in the first Special Issue of QSS, dedicated to the description of the bibliometric data sources that lie at the heart of scientometric research, which aims to characterize the most important data sources currently available and to show how they differ in various dimensions, for instance in the data they provide, their level of openness, and their support for making research reproducible. The first three papers in this special issue cover the most important commercial bibliographic data sources: Web of Science (Clarivate Analytics), Scopus (Elsevier), and Dimensions (Digital Science), while the remaining three articles describe open data sources: Microsoft Academic, Crossref and OpenCitations.

In the introduction to our own paper, we describe the origins of OpenCitations, discuss the growth and benefits of open science, and introduce the Semantic Web techniques used at OpenCitations for recording and publishing our data. We then go on to describe OpenCitations’ services and data, namely Open Citation Identifiers, the OpenCitations Data Model, the SPAR (Semantic Publishing and Referencing) Ontologies, the OpenCitations Corpus, and the OpenCitations Indexes of citation data, of which the first and largest is COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations, that currently holds information on over 624 million citations. We conclude our survey of OpenCitations’ services and data by outlining the generic open source software developed at OpenCitations, including OSCAR, the OpenCitations RDF Search Application for searching over RDF datasets, LUCINDA, OSCAR’s associated OpenCitations RDF Resource Browser, and RAMOSE, OpenCitations’ application for creating REST APIs over SPARQL endpoints, thus opening Semantic Web datasets to those not familiar with SPARQL, the RDF query language.

In the second half of the paper, we describe OpenCitations as an organization in terms of its compliance with the principles for the sustainability of open infrastructures proposed by Bilder, Lin and Neylon (2015) [2], and report the selection of OpenCitations by the Global Sustainability Coalition for Open Science Services (SCOSS) as an open infrastructure organization worthy of crowd-funding support by the stakeholder community. We then provide usage statistics for our datasets and web site, and describe the adoption of OpenCitations data and services by the community, before concluding with a forward look at our proposed developments of OpenCitations activities.

References

[1] Silvio Peroni and David Shotton (2020). OpenCitations, an infrastructure organization for open scholarship. Quantitative Science Studies 1 (1): 428-444. https://doi.org/10.1162/qss_a_00023

[2] Geoffrey Bilder, Jennifer Lin and Cameron Neylon (2015). Principles for open scholarly infrastructures. Figshare. https://doi.org/10.6084/m9.figshare.1314859

More than 624 million citations now available on COCI

COCI is the OpenCitations Index of Crossref open DOI-to-DOI citations, all released as CC0 material, and is described in the article

Heibi I, Peroni S, Shotton D (2019). Software review: COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations. Scientometrics 121(2): 1213-1228. https://doi.org/10.1007/s11192-019-03217-6

COCI is our first OpenCitations Index of open citations, in which we have applied the concept of citations as first-class data entities, each identified using a unique persistent Open Citation Identifier (OCI), to index the contents of one of the major databases of open scholarly citation information, namely Crossref, and to render and make available this information in machine-readable RDF.

We are now proud to announce the third release of COCI, which contains more than 624 million DOI-to-DOI citation links coming from both ‘the ‘Open’ and the ‘Limited’ sets of Crossref reference data. This represents an increase of 40% in the number of indexed citations, compared with the second release of COCI on 12th November 2018, which indexed more than 445 million citations. The data model used for this third release of COCI is the updated revision of the OpenCitation Data Model, published on 8 November 2019 and available at https://doi.org/10.6084/m9.figshare.3443876.

This new release of COCI has been created using new software developed specifically for this purpose, which is available on our GitHub repository under an open ISC license. This software automates the process of creating an OpenCitations Index compliant with the OpenCitations Data Model and creates the citation data and related provenance information in three different formats: CSV, N-Triples (RDF), and Scholix. The support for Scholix – a high-level interoperability framework supported by Crossref, DataCite, Europe PubMed Central, OpenAIRE and others  – has recently been added to provide an additional format for the exchange of information about the links between scholarly literature and datasets.

A great advantage of the new software is that it will now enable us to extend COCI (and any other OpenCitations Index) by means of incremental additions, rather than having to re-create the entire index at each update. This should enable us to release index updates more frequently than hitherto, thus keeping the index more closely in synchrony with the latest reference data released by Crossref. Note that we are currently run the software on previous dumps of Crossref data so as to retrieve all the citations that involve references in citing articles that were in the ‘Limited’ set when we downloaded it, but that currently appear in the Crossref ‘Closed’ data set due to more recent restrictive policy decisions taken by their publishers.

Finally, we wish to remind you that all the bibliographic and citation data in COCI: