Performing live time-traversal queries on RDF datasets

Guest post by Arcangelo Massari, University of Bologna

In this post, Arcangelo Massari, who recently graduated in Digital Humanities and Digital Knowledge under Professor Silvio Peroni at the University of Bologna, shares the results of his master thesis.

A particular problem in information retrieval is that of obtaining data from an evolving dataset, independent of the time at which that item of data was added, changed or removed. To permit such time-independent queries to be performed over evolving RDF datasets, I have developed two new pieces of open source software, time-agnostic-library [1] and time-agnostic-browser [2], that are now available from the OpenCitations GitHub repository.

The time-agnostic-library is a Python library to perform live time-traversal queries on RDF datasets. Time-traversal means being agnostic about time: a SPARQL query that is not run on the current state of the collection but over its entire history or over a specified timespan of that history [3]. This tool allows materializations – obtaining all versions of an entity over time, or its status at a given time. Furthermore, SPARQL queries can be performed to get the delta between two or more versions of one or more resources. Thereby, the time-agnostic-library realizes all the retrieval functionalities described in the taxonomy by Fernández et al. [3].

To complement this query software, the time-agnostic-browser is a web application built on top of the time-agnostic-library to achieve the same results via a graphical user interface.

The primary purpose of these developments is to offer a system for browsing the provenance [4] of RDF statements across time: who produced them, when, where the information was taken from, and what changes were made compared to the previous state of the resource. Knowledge of such information is essential because data changes over time, either because of the natural evolution of concepts or due to the correction of mistakes. Indeed, the latest version of knowledge may not be the most accurate. Such phenomena are particularly tangible in the Web of Data, as highlighted in a study by the Dynamic Linked Data Observatory, which noted the modification of about 38% of the nearly 90,000 RDF documents monitored for 29 weeks, and the permanent disappearance of 5% of them [5] (Figure 1).

Figure 1. Donut chart showing the results of the study conducted by the Dynamic Linked Data Observatory on the evolution of RDF documents [5].

Additionally, the truthfulness of data cannot be assessed without provenance records and a system to query them. In fact, the truth value of an assertion on the Web is never absolute, as demonstrated by Wikipedia, which in its official policy on the subject states: “The threshold for inclusion in Wikipedia is verifiability, not truth.” [6]. The Semantic Web does not alter that condition, and trustworthiness has to be evaluated by each application by probing the context of the statements [7]. It is a challenging task and thus, in the Semantic Web Stack, trust is the highest and most complex level to satisfy, subsuming all the previous ones (Figure 2).

Figure 2.The Semantic Web layers [7]. Trust is the uppermost level of the stack, subsuming all the others.

Notwithstanding these premises, at present the most extensive RDF datasets – DBPedia [8], Wikidata [9], Yago [10], and the Dynamic Linked Data Observatory [11] – do not use RDF to track changes and record the provenance of such changes. Instead, they all adopt backup-based archiving policies. Some of them, such as Yago 4, record provenance but not changes. As far as citation databases are concerned, OpenCitations is the only infrastructure to implement change-tracking mechanisms and to record full RDF provenance records for each data entity. Among the leading players in this field, neither Web of Science nor Scopus have adopted similar solutions.

In accordance with the OpenCitations Data Model (OCDM) [12], a provenance snapshot is generated by OpenCitations every time a bibliographical entity is created or modified. Each snapshot (prov:Entity) records the responsible agent (prov:wasAttributedTo), the generation time (prov:generatedAtTime), the invalidation time (prov:invalidatedAtTime), the primary source (prov:hadPrimarySource), and a link to the previous snapshot (prov:wasDerivedFrom), using terms from the Provenance Ontology. In addition, OCDM introduced a system to simplify restoring an entity’s status at a given time, by saving the delta between two versions as a SPARQL update query (prov:hasUpdateQuery) [13] (Figure 3). This approach enables one to restore an entity to a specific timepoint (snapshot) in a straightforward way by applying the inverse operations, i.e., deletions instead of additions, etc.

Figure 3. Provenance in the OpenCitations Data Model.

This solution is concretely used in all the datasets related to the OpenCitations infrastructure, such as COCI, an open index containing almost 1.2 billion DOI-to-DOI citation links derived from the open reference data available in Crossref [14]. It is important to note that this OpenCitations provenance model is generic and reusable in any other context. Since the time-agnostic-library leverages OCDM, it too is generic and can be used for any RDF dataset that tracks changes and provenance as OpenCitations does.

The time-agnostic-library is released under the ISC license and is downloadable through pip [1]. Test-driven development was adopted as a software development process during its creation [15]. It makes three main classes available to the user: AgnosticEntity, VersionQuery, and DeltaQuery, for materializations, version queries, and delta queries, respectively (Listing 1).

Listing 1. Code template to achieve materializations, time-traversal queries, and delta queries.

All three operations can be performed over the entire available history of the dataset, or by specifying a time interval via a tuple in the form (START, END).

The time-agnostic-browser [2] is also released under the ISC license and can be run as a Flask application. It is organized into two macro-sections: “Explore” and “Query”. In the former, a text input accepts a URI. By submitting it, the entire history of the corresponding resource is displayed. In the latter, a text area receives a SPARQL query, which is resolved on all dataset states. Its main added value is hiding the triples and the complexity of the underlying RDF model: predicate URIs, as well as subjects and objects, appear in a human-readable format. Moreover, all the entities are displayed as links, providing shortcuts to reconstruct the history of the related resources (Figure 4).

Figure 4. Graphical user interface of an entity history reconstruction through the time-agnostic-browser.

The efficiency of time-agnostic-library was measured with two types of benchmarks [16], one on execution times and the other on the amount of computer memory (RAM) required by ten different use cases, each repeated ten times to produce significant results and avoid outliers. In light of these benchmarks, time-agnostic-library has proven effective for any materialization. Regarding structured queries, they are swift if all subjects are known or deductible. On the other hand, the presence of unknown subjects in the user’s SPARQL query involves the identification of all present and past entities that satisfy that pattern, and so requires a more significant amount of time and resources. Specifically, all materializations and the cross-version structured query with known subjects required about half a second and about 50 MB of RAM; conversely, with unknown subjects, 581 seconds and 519 MB of RAM are required. It can be concluded that the proposed software can be used effectively in all cases where the subject is known, that is, for any materialization or formulated SPARQL queries without isolated triple patterns containing unknown subjects.

Other software solutions for such problems have been proposed. Table 1 shows the list of available software to perform materializations and time-traversal queries on RDF datasets. As can be observed, time-agnostic-library is the only one to support all retrieval functionalities without requiring pre-indexing processes. This feature makes it particularly suitable for use in scenarios with large amounts of data that often change over time. Moreover, compared to the approach of Im, Lee and Kim [17] and OSTRICH [18], the OpenCitations Data Model only requires storing the current state of the dataset, rather than the original one, allowing one to query the latest version, without additional computational effort to first re-create the original version.

SoftwareVersion materializationDelta materializationSingle-version structured queryCross-version structured querySingle-delta structured queryCross-delta structured queryLive
PromptDiff [19]+++
SemVersion [20]+++
Im, Lee, & Kim, 2012 [17]+++++
R&Wbase [21]++++
x-RDF-3X [22]+++
v-RDFCSA [23]++++++
OSTRICH [18]+++
Tanon & Suchanek, 2019 [24]++++++
time-agnostic-library[1]+++++++
Table 1. Comparative between time-agnostic-library and preexisting software to achieve materializations and time traversal queries on RDF datasets. (Scroll right to see Columns 6-8).

The OpenCitations Data Model and the time-agnostic-library software are the pre-requisites that will allow OpenCitations to involve third parties, for example members of staff in academic libraries, in the submission, curation and updating of OpenCitations bibliographic and citation data. At this stage, all entities in COCI have a single snapshot — the one made at the time of creation. However, since these entities may become modified, corrected or enriched over time, it is imperative to have appropriate software tools available for use by curators. With the time-agnostic-library software and its associated time-agnostic-browser, it will be possible for a curator to explore the entire history of the changes within an RDF dataset, to know when they were made, based on which source, and by which responsible agent, thus ensuring the reliability and verifiability of data, and facilitating any necessary further changes.

References

[1] A. Massari, time-agnostic-library. 2021. Available: https://archive.softwareheritage.org/swh:1:snp:d7fd1754377f45d16afb61efc770815b5a3c8f83

[2] A. Massari, time-agnostic-browser. 2021. Available: https://archive.softwareheritage.org/swh:1:dir:337f641375cca034eda39c2380b4a7878382fc4c

[3] J. D. Fernández, A. Polleres, and J. Umbrich, ‘Towards Efficient Archiving of Dynamic Linked’, in DIACRON@ESWC, Portorož, Slovenia: Computer Science, 2015, pp. 34–49.

[4] December, ‘Provenance XG Final Report’. 2010. Available: http://www.w3.org/2005/Incubator/prov/XGR-prov-20101214/

[5] T. Käfer, A. Abdelrahman, J. Umbrich, P. O’Byrne, and A. Hogan, ‘Observing Linked Data Dynamics’, in The Semantic Web: Semantics and Big Data, vol. 7882, P. Cimiano, O. Corcho, V. Presutti, L. Hollink, and S. Rudolph, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2013, pp. 213–227. doi: 10.1007/978-3-642-38288-8_15

[6] S. L. Garfinkel, ‘Wikipedia and the Meaning of Truth’, MIT Technology Review, 2008, [Online]. Available: https://stephencodrington.com/Blogs/Hong_Kong_Blog/Entries/2009/4/11_What_is_Truth_files/Wikipedia%20and%20the%20Meaning%20of%20Truth.pdf

[7] M.-R. Koivunen and E. Miller, ‘Semantic Web Activity’, W3C, Nov. 02, 2001. https://www.w3.org/2001/12/semweb-fin/w3csw

[8] F. Orlandi and A. Passant, ‘Modelling provenance of DBpedia resources using Wikipedia contributions’, Journal of Web Semantics, vol. 9, no. 2, pp. 149–164, Jul. 2011, doi: 10.1016/j.websem.2011.03.002.

[9] P. Dooley and B. Božić, ‘Towards Linked Data for Wikidata Revisions and Twitter Trending Hashtags’, in Proceedings of the 21st International Conference on Information Integration and Web-based Applications & Services, Munich Germany, Dec. 2019, pp. 166–175. doi: 10.1145/3366030.3366048.

[10] Yago Project, ‘Download data, code, and logo of Yago projects’, Yago, 2021. https://yago-knowledge.org/downloads (accessed Sep. 24, 2021).

[11] J. Umbrich, M. Hausenblas, A. Hogan, A. Polleres, and S. Decker, ‘Towards Dataset Dynamics: Change Frequency of Linked Open Data Sources’, in Proceedings of the WWW2010 Workshop on Linked Data on the Web, Raleigh, USA, 2010. Available: http://ceur-ws.org/Vol-628/ldow2010_paper12.pdf

[12] M. Daquino, S. Peroni, and D. Shotton, ‘The OpenCitations Data Model’, p. 836876 Bytes, 2020, doi: 10.6084/M9.FIGSHARE.3443876.V7.

[13] S. Peroni, D. Shotton, and F. Vitali, ‘A Document-inspired Way for Tracking Changes of RDF Data’, in Detection, Representation and Management of Concept Drift in Linked Open Data, Bologna, 2016, pp. 26–33. Available: http://ceur-ws.org/Vol-1799/Drift-a-LOD2016_paper_4.pdf

[14] I. Heibi, S. Peroni, and D. Shotton, ‘Software review: COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations’, Scientometrics, vol. 121, no. 2, pp. 1213–1228, Nov. 2019, doi: 10.1007/s11192-019-03217-6.

[15] K. Beck, Test-driven development: by example. Boston: Addison-Wesley, 2003.

[16] A. Massari, ‘time-agnostic-library: benchmark results on execution times and RAM’. Zenodo, Oct. 05, 2021. doi: 10.5281/ZENODO.5549648.

[17] D.-H. Im, S.-W. Lee, and H.-J. Kim, ‘A Version Management Framework for RDF Triple Stores’, Int. J. Softw. Eng. Knowl. Eng., vol. 22, pp. 85–106, 2012.

[18] R. Taelman, M. V. Sande, and R. Verborgh, ‘OSTRICH: Versioned Random-Access Triple Store’, in Companion Proceedings of the Web Conference 2018, 2018, pp. 127–130. Available: https://core.ac.uk/download/pdf/157574975.pdf

[19] N. F. Noy and M. A. Musen, ‘Promptdiff: A Fixed-Point Algorithm for Comparing Ontology Versions’, in Proc. of IAAI, 2002, pp. 744–750.

[20] M. Völkel, W. Winkler, Y. Sure, S. Kruk, and M. Synak, ‘SemVersion: A Versioning System for RDF and Ontologies’, 2005.

[21] M. V. Sande, P. Colpaert, R. Verborgh, S. Coppens, E. Mannens, and R. V. Walle, ‘R&Wbase: Git for triples’, 2013.

[22] T. Neumann and G. Weikum, ‘x-RDF-3X: Fast Querying, High Update Rates, and Consistency for RDF Databases’, Proceedings of the VLDB Endowment, vol. 3, pp. 256–263, 2010.

[23] A. Cerdeira-Pena, A. Farina, J. D. Fernandez, and M. A. Martinez-Prieto, ‘Self-Indexing RDF Archives’, in 2016 Data Compression Conference (DCC), Snowbird, UT, USA, Mar. 2016, pp. 526–535. doi: 10.1109/DCC.2016.40.

[24] T. Pellissier Tanon and F. Suchanek, ‘Querying the Edit History of Wikidata’, in The Semantic Web: ESWC 2019 Satellite Events, vol. 11762, P. Hitzler, S. Kirrane, O. Hartig, V. de Boer, M.-E. Vidal, M. Maleshkova, S. Schlobach, K. Hammar, N. Lasierra, S. Stadtmüller, K. Hose, and R. Verborgh, Eds. Cham: Springer International Publishing, 2019, pp. 161–166. doi: https://doi.org/10.1007/978-3-030-32327-1_32.

Academia’s missing references

No-one is quite sure of the total number of scholarly publications within the global corpus. Indeed that number will be strongly influenced by the degree to which, in addition to books and journal articles, one includes within the definition of scholarly publications ‘grey literature’ such as reports published by official bodies, patents, etc. Consequentially, the total number of scholarly references within those publications is also unknown, and this number too will vary according to the inclusion criteria chosen. Furthermore, Crossref Event Data and similar datasets recording social media mentions of journal articles in blog posts and tweets extends the concept of a reference beyond that used in ‘conventional’ citation indexes such as COCI.

We celebrate the fact that well over one billion bibliographic citations are now openly available under CC0 waivers in NIH OCC (the National Institutes of Health Open Citation Collection) [1,2] and COCI (the OpenCitations Index of Crossref DOI-to-DOI Citations) [3]. Despite present gaps in their coverage, they include references to all the most important publications within the global corpus, because these will all have been cited multiple times.

Open references available from Crossref and other aggregators and indexes

Crossref, with over 1.6 billion open references, is the largest single source of such bibliographic metadata. Significant numbers of references are also available in a variety of other databases, repositories and indexes.

NIH OCC (the National Institutes of Health Open Citation Collection) is a merger of several citation databases, drawing on PubMed for crucial article metadata, and augmenting this with information from full-text articles that have been made freely available on the internet [1]. The CiteSeerX database, the arXiv preprint repository, and the Dryad data repository are examples of different types of infrastructure that also publish open bibliographic references, while there is further availability of article references from open aggregators such as DataCite and Wikidata. These may either use their own DOIs, DOIs from the Crossref DOI registration agency, or no DOIs at all. Either way, those references will not appear in Crossref.

What is lacking is semantic coherence and interoperability between these sources, permitting federated queries across them. This makes difficult the task of obtaining a comprehensive overview of the availability of open bibliographic references.

However, there are even more citations that are not yet freely and easily available anywhere in bulk, relating to the reference lists within publications of a number of distinct types. This blog post explores academia’s missing references – those that have not yet been documented within open freely accessible citation indexes – and what might be done to bring these into the public domain.

1 References that are closed at Crossref

Eight years ago, I wrote

“In this open-access age, it is a scandal that reference lists from journal articles — core elements of scholarly communication that permit the attribution of credit and integrate our independent research endeavours — are not readily and freely available for use by all scholars.” [4]

I stand by that statement, and, through OpenCitations [5], I have been working with colleagues to rectify the situation, as I described in my previous post. The Initiative for Open Citations (I4OC) can rightly be applauded for its part in encouraging almost all the major academic publishers who deposit references at Crossref to make them open.

The only major scholarly publisher not to be listed as an I4OC Participating Publisher is the Institute of Electrical and Electronics Engineers (IEEE), that, having deposited at Crossref reference lists for 58% of its preprints and publications, persists in keeping these deposited references closed and unavailable for indexing and re-use. It is to be hoped that IEEE will now realize that what it looses by not having references openly available outweighs any benefits it might have received from keeping them closed, and will join its fellow publishers in ensuring that its Crossref-deposited references are made open, both for its current issues and for its back-number journal articles. I thus call again upon IEEE to change its present position, as both Elsevier and the American Chemical Society had the courage to do recently, and to instruct Crossref to open all IEEE references. A single email to Crossref Support, with the instruction “Open all references”, is all it would take!

References in Crossref are either open, ‘limited’ or closed. Limited references are available to those subscribing to Crossref Metadata Plus, which includes OpenCitations, but not to the general public. Closed references are not freely available to anyone. The following table shows the number of Crossref works in each category, and the number of references within those categories.


WorksReferencesAverage references per work
Total126,627,618

Without references70,477,843
(55.7% of total)


With references56,149,775
(44.3% of total)
1,734,831,31130.9
Open49,274,155
(38.9% of total)
1,605,120,22932.6
Limited2,933,323
(2.3% of total)
66,459,70422.7
Closed3,942,297
(3.1% of total)
63,251,37816.0
Table 1. Numbers of works and their references recorded in Crossref on 31 July 2021.

2 References not deposited at Crossref for publications with Crossref DOIs

The number of works with Crossref DOIs that lack submitted references is surprisingly large. As of 31 July 2021, 70,477,843 publications (55.7% of all works recorded at Crossref) lacked deposited references (Table 1).

Crossref classifies all types of journal content, including editorials, book reviews and letters, as “journal articles”, thus some of these works without deposited references genuinely lack them. However, the majority are conventional journal articles and books with reference lists that the publishers have simply not deposited at Crossref along with the other metadata for these works.

The average number of references per Crossref work with deposited references is 30.9 (Table 1). If, to make allowance for those works that genuinely lack references, we assume a conservative average of 25 references per work for the 70,477,843 works lacking deposited references, this means that there are over 1.75 billion references within these works that have not been deposited at Crossref, and thus are not conveniently available for indexing and reuse.

These missing references relate both large publishers that have submitted references for only some of their publications, and small publishers that perhaps lack, or think they lack, the resources to deposit any reference lists in addition to the other metadata they are already sending to Crossref for each of their DOIs. However, there are several easy methods for depositing reference lists, as detailed by Crossref here. So I encourage all publishers who are supportive of Open Science to update their procedures and commence or complete the deposition of their publication reference lists at Crossref, starting with their current issues. Crossref Support will provide assistance if required. Note that a publisher does not have to subscribe to the Crossref Cited-by service to deposit its references!

3 Citations missing in COCI: open references in Crossref to publications lacking DOIs

COCI is the OpenCitations Index of Crossref DOI-to-DOI Citations, and, as the name suggests, it indexes Crossref open references from works with Crossref DOIs to other works that have DOIs [3]. It therefore does not index open references in Crossref to works that, for whatever reason, lack DOIs.

The most recent (September 2021) release of COCI, based on the August Crossref dump, contains 1,186,958,898 citations between 69,074,291 unique work, comprising 51,103,720 citing bibliographic resources bearing Crossref DOIs and 56,105,783 cited bibliographic resources. Of the cited bibliographic resources, 38,135,212 bear a DOI issued by Crossref and have open or limited references (thus also being COCI citing resources), while 17,970,571 either have a Crossref DOI but lack open or limited references or have a DOI issued by another DOI registration agency such as DataCite (thus not being COCI citing resources).

Note that in Crossref, the ratio of works with open or limited references to works without open or limited references is 0.7:1 (Table 1). However, in COCI, the ratio of cited works with Crossref DOIs containing open or limited references to all other cited works is 2.1:1. Thus works with Crossref DOIs containing open or limited references are three times more likely to be cited than works that either have a Crossref DOI but lacking open or limited references or have a DOI issued by another DOI registration agency. This is most likely because the most important journals from almost all the larger publishers now have open references. However, it is still a remarkable ratio.

Because references to works lacking DOIs are not included in COCI, the average number of bibliographic references per citing article in COCI is only 23.2, in contrast to the numbers of references to works of all types given in Table 1.

From these data, it can be seen that there are over 480 million open Crossref references to a wide variety of works lacking DOIs that OpenCitations does not index in COCI. This is because of an intentional and fundamental limitation in the structure of the Open Citation Identifier (OCI) [6], requiring both citing and cited publications to have identifiers of the same type, that lies at the heart of the functionality of our OpenCitations Indexes.

OpenCitations is currently developing a solution that, without compromising that intentional design limitation in OCIs, will nevertheless permit us to index and publish these ‘missing’ references as Linked Open Data. We will report on this development in due course.

Crossref Event Data is a Crossref service / database that records mentions of publications bearing Crossref DOIs in social media including tweets and blog posts, and in other non-traditional citation sources such as news items and Wikipedia articles. From today (Thursday 23rd September 2021), Crossref Event Data will start to include in its holdings open references from publications bearing Crossref DOIs to other publications bearing DOIs. Limited and closed references will not be included. Initially, open references from current publications will be included in Crossref Event Data, with open references from older works with DOIs being added later. In that respect it will come to resemble COCI, except that COCI also included ‘limited’ references, treats citations as first-class data entities with their own identifiers, and makes its citation data available in RDF as Linked Open Data, as well as via a REST API. Subsequently, Crossref Event Data will also record references to publications with other forms of identifier, as OpenCitations also plans to do.

4 References available on publishers’ web sites

Many publishers, particularly those of Open Access works, already make their publication reference lists openly available on their own web sites. While this is commendable, it is not sufficient, since, if scholarly references are not made available in a centralized aggregator such as Crossref from which they can be conveniently harvested in bulk for analysis and re-use, they are much more difficult to access.

Scraping references from the HTML of individual web sites is difficult, time-consuming and liable to be incomplete. While Microsoft Academic achieved considerable success in scraping references from publishers’ web sites, possible because of the special relationships these publishers have with the Microsoft search engine Bing, this service will unfortunately soon no longer be available, illustrating a problem inherent with scholarly infrastructures provided by commercial companies that do not adopt the Principles of Open Scholarly Infrastructures.

A consequence is that such publications will become increasingly ‘invisible’, as bibliographic and analytical services come to rely more and more on centrally available data.

6 References within PDFs of scholarly works lacking DOIs

There is a large but unknown quantity of reference-containing books, academic reports, patents and journal articles from publishers that, for their own good reasons, choose not to use DOIs. The text of many of these publications is already available in a marked-up machine-readable format such as JATS, used in preparation for publication, from which the reference lists could easily be extracted. Other publications are only available as PDFs, both as published Versions of Record, or as preprints deposited in a variety of preprint repositories such as arXiv or CORE. Mining reference lists out of PDFs required expertise in text mining and AI technologies, and is labour-intensive, since it usually required the tuning of extraction algorithms to handle the particular styles and formatting of individual journals, one at a time. Two stages are involved: first, the recognition and extraction of the text of the individual references from the PDF, and second the parsing of each text string into the component parts of the reference (author names, title, publication year, etc.) A considerable number of the citations within NIH-OCC have been obtained in this manner [1], commercial companies such as Lexical Intelligence specialize in this area, and publicly available software such as GROBID is available for the purpose. However, the overall task of extracting ‘missing’ academic references from the global PDF corpus is daunting in magnitude and would require a well-funded organization.

The correct way to proceed would be for each publisher to take responsibility for liberating the references of their own publications, whether or not the publications themselves are open access, and whether or not these references are already available in a marked-up machine-readable format or only within PDF documents. Then, if the publisher still chose not to use DOIs and to submit these metadata to Crossref, these references could be submitted directly to OpenCitations for aggregation and publication as Linked Open Data.

Conclusion

From the foregoing discussion it is clear that the academic community has a long way to go before the majority of scholarly citations, the products of their own labours, are openly available for analysis and re-use. We at OpenCitations are working to address these issues and to publish more of these missing citations. However, completion of the task will require a coordinated collaborative international effort.

Are you willing to be involved?

References

[1] B. Ian Hutchins et al. (2019). The NIH Open Citation Collection: A public access, broad coverage resource. PLoS Biol. 17 (10): e3000385. https://doi.org/10.1371/journal.pbio.3000385

[2] B. Ian Hutchins (2021). A tipping point for open citation data. Quantitative Science Studies 2 (2): 433–437. https://doi.org/10.1162/qss_c_00138

[3] Ivan Heibi, Silvio Peroni, David Shotton (2019). Software review: COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations. Scientometrics 121 (2): 1213-1228. https://doi.org/10.1007/s11192-019-03217-6

[4] David Shotton (2013). Open citations. Nature, 502 (7471): 295-297. http://dx.doi.org/10.1038/502295a

[5] Silvio Peroni, David Shotton (2020). OpenCitations, an infrastructure organization for open scholarship. Quantitative Science Studies, 1(1): 428-444. https://doi.org/10.1162/qss_a_00023

[6] Silvio Peroni, David Shotton (2019). Open Citation Identifier: Definition. Figshare. https://doi.org/10.6084/m9.figshare.7127816

92 million new citations added to COCI

It’s been a month since the announcement of 1.09 Billion Citations available in the July 2021 release of COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations.  

We’re now proud to announce the September 2021 release of COCI, which is based on open references to works with DOIs within the Crossref dump dated August 2021. This new release extends COCI with more than 92 Million additional citations, giving a total number of more than 1.18 Billion DOI-to-DOI citation links.

This latest release includes citations from the most recent articles published by the American Chemical Society, whose bibliographic references were opened in February 2021. The ACS back number citations will be available in the next COCI release, when a new processing of all the Crossref data will be completed.

You can find more information about COCI in our open-access article 

Ivan Heibi, Silvio Peroni & David Shotton (2019). Software review: COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations. Scientometrics, 121 (2): 1213-1228. DOI: https://doi.org/10.1007/s11192-019-03217-6  

Finally, just a reminder that the bibliographic and citation data in COCI: 

  • can be queried using the OpenCitations Indexes SPARQL endpoint; 
  • can be retrieved by using the COCI REST API
  • can be searched by using the OpenCitations Indexes Search Interface; 
  • are also available as dumps on Figshare in CSV, N-Triples, and Scholix; and 
  • can be freely re-used for any purpose. 

OpenCitations’ compliance with the Principles of Open Scholarly Infrastructure

What should an open scholarly infrastructure look like? 

An answer to this tough question can be found in the original February 2015 blog post by Geoffrey Bilder, Jennifer Lin and Cameron Neylon

Bilder G., Lin J., Neylon C. (2015) Principles for Open Scholarly Infrastructure , http://dx.doi.org/10.6084/m9.figshare.1314859

and in the summary of the principles to be found as:  

Bilder G, Lin J, Neylon C (2020), The Principles of Open Scholarly Infrastructurehttps://doi.org/10.24343/C34W2H : 

Infrastructure at its best is invisible. We tend to only notice it when it fails.  If successful, it is stable and sustainable. Above all, it is trusted and relied on by the broad community it serves. Trust must run strongly across each of the following areas: running the infrastructure (governance), funding it (sustainability), and preserving community ownership of it (insurance)”. 

These areas are fully define the Principles of Open Scholarly Infrastructure (POSI), which provide a set of guidelines by which open scholarly infrastructure organizations and initiatives that support the research community can be run and sustained.  

As far as we are aware, Crossref was the first infrastructure to publish its compliance with POSI, detailed in Geoffrey Bilder’s December 2020 blog post

Crossref’s Board votes to adopt the Principles of Open Scholarly Infrastructure.

OpenCitations too espouses POSI and, in January 2021, we monitored the extent of our own compliance with POSI, the results of which are shown in the following diagram. 

Governance 

 Coverage across the research enterprise We gather citations from global scholarship 
 Stakeholder governed Advisory board 
currently lacks
executive power and is not elected 
 Non-discriminatory membership Membership open to all those espousing 
open science 
● Transparent operations Everything is open 
 Cannot lobby OpenCitations lobbies to achieve open 
scholarly citations 
and bibliographic 
metadata; 
it does not engage in political or financial 
lobbying 
 Living will Since all our data open, others can 
recreate our service 
 Formal incentives to fulfill mission & wind-down No formal plan for wind-down 
has yet been drawn up 

Sustainability 

 Time-limited funds used only for time-limited activities Grant income should 
be used solely for grantprojects 
 Goal to generate surplus Goal not yet realized – 
income so far too limited 
 Goal to create contingency fund to support operations for 12 months Goal not yet realized – 
income so far too limited 
 Mission-consistent revenue generation Membership fees and 
solicited donations 
 Revenue based on services, not data All data and services freely given to community, and thus do not 
generate income 

Insurance 

 Open source All software under open source licenses 
 Open data All data available 
under CC0 waiver 
 Available data All data available via REST APIs, SPARQL endpoints, query interfaces and data dumps 
 Patent non-assertion We will not 
patent anything: 
OpenCitations’ 
infrastructure 
is free to replicate 

 
We at OpenCitations are proud of the results reached in the Insurance area, but realise that we still have some was to go in the other areas. Although the general situation is already satisfying, we are working to strengthen our weak points. 

Crossing a significant threshold: more than one billion citations now available in COCI!

“The competitive benefits of closing access to citation data diminish with each new citation released to the public domain, but the benefits of open data remain. Going forward, citation data is almost completely public domain”.

With these words, from the article “A tipping point for open citations data” (July 15, 2021), Ian Hutchins celebrated the threshold crossing of one billion citations on public-domain databases in February 2021.

Now, a new significant milestone has been reached. We are enthusiastic to announce that COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations has just been extended with 334 million additional citations. Its most recent release, the COCI July 2021 release, now contains a total of 1.09 billion DOI-to-DOI citation links derived from open references within Crossref,which includes the references of articles deposited or opened in Crossref between November 2020 and January 2021.

These numbers make us proud, and confirm the essential value of the Initiative for Open Citations (I4OC). Since 2018, the mission of I4OC has been to persuade publishers to provide open citation data by means of the Crossref platform. The I4OC untiring commitment has led the major academic publishers to a progressive change of heart regarding open citations, and the scholarly community to a deeper interest in this openness.

These factors contributed to the creation of COCI in 2018, the first open citation index created by OpenCitations, in which we applied the concept of citations as first-class data entities (Heibi I., Peroni S., Shotton D., 2019). Over the last three years, COCI has been extended in a series of releases, by harvesting citations mostly from Crossref data dumps, starting from an initial coverage of 300 million citations (First release).

A crucial event that preceded (and delayed!) this latest COCI release was Elsevier’s endorsement in the DORA Declaration on Research Assessment in December 2020, thereby making “reference lists for all articles published in Elsevier journals openly available via Crossref so they can be available for reuse. This means other important initiatives like I4OC can draw on this metadata”. As described in our previous post, Elsevier’s welcome commitment led to the opening of many previously closed references from its numerous academic journals submitted to Crossref. Now, after an extended period of data ingestion and processing, all these newly opened Elsevier references are available at OpenCitations within COCI.

Elsevier’s involvement has both an effective and a symbolical value. Even if publishing more than one billion citations is a thrilling achievement, and – as Hutchins wrote – we are now at a tipping point with regard to open citations data, this milestone is not the last stop. Together with the other organizations and projects that participate in the Initiative for Open Citations, we will keep claiming the urgency for the remaining academic publishers to join our cause, and sharing our values with the whole academic community to make all existing citations data freely open and accessible. Recalling what Dario Taraborelli wrote in the conclusion of his article “The citation graph is one of humankind’s most important intellectual achievements“, “the world is waiting for the citation graph to become a public good”.

OpenCitations at LIBER Annual Conference 2021: ‘How Can Open Infrastructures Support the Role of Research Libraries?’

For the second year, OpenCitations has taken part in the LIBER annual conference.  LIBER (Ligue des Bibliothèques Européennes de Recherche – Association of European Research Libraries) is a network that gathers 440 research libraries, based in more than 40 countries all over the world, with the mission of supporting Europe’s research libraries by highlighting their value to policymakers, providing resources and training, and forming valuable partnerships. 

Since 1951, the LIBER Annual Conference is a key event for the entire network, a keenly anticipated meeting for research library professionals whose mission is “to identify the most pressing needs for research libraries, and to share information and ideas for addressing those needs”. Due to the ongoing pandemic restrictions, the 50th LIBER meeting (23-25 June 2021) was held online, as was the 2020 meeting, with digital co-hosting by the University of Belgrade Library in Serbia. The online-showcase format, however, didn’t constrain the creation of a vital virtual square, fostered by the voices of 70 speakers. The main theme of the conference, “Libraries and Open Knowledge: from vision to implementation” was deepened in 12 parallel sessions.

Professor Silvio Peroni, Director of OpenCitations, participated in Session #5 ‘How Can Open Infrastructures Support the Role of Research Libraries?’ with a presentation dedicated to the benefits of Open Infrastructures for libraries, dialoguing with James MacGregor (interim Managing Director of the Public Knowledge Project), Joanna Ball (Head of Roskilde University Library), and Niels Stern (director of OAPEN and co-Director of DOAB).  

The session, chaired by Maaike Napolitano (National Library of the Netherlands) opened with a presentation by Fidan Limani (Research assistant at ZBW– Leibniz Information Centre for Economics) about the integration of scholarly artifacts from the domain of economics using Knowledge Graphs (KG), and the creation of a network of entities describing objects of interest and connections, while keeping a library perspective. The use of citation links connecting datasets and citations, and the adoption of ontologies and data exportation in RDF would facilitate a possible beneficial collaboration between ZBW and Open Infrastructures such as OpenCitations (whose data is itself in the form of a Knowledge Graph). 

OpenCitations also shares some common features with the other Open Infrastructures described in the second presentation: the financial support from SCOSS project; the community-based approach; and their promising value for libraries and the entire scholarly community.  

OpenCitations is an independent not-for-profit infrastructure organization dedicated to open scholarship and the publication of open bibliographic and citation data by the use of Semantic Web (Linked Open Data) technologies, engaged in advocacy for open citations and open bibliographic metadata, as a founding member of both the Initiative for Open Citations (I4OC) and the Initiative for Open Abstracts (I4OA). It provides data containing more than 7 hundred million citations that the community can use for any purpose. Such data can be crucial as a vehicle for use in national and international research evaluation exercises to make such activities more transparent and reproducible as compared to other proprietary services. Librarians can use OC citation data (e.g., via our REST API) to enhance or develop tools to support their authors, researchers, students, institutional administrators in different kind of contests, for instance by providing metrics to monitor research at your institution and by improving the discoverability of research products such as publications and data. 

OAPEN is a no-profit foundation dedicated to increase the discoverability of open access books and trust around them. They are running three Open-Source platforms enabling open access to books:  the Directory of Open Access Books (DOAB) – a freely available basic indexing service easy integrable within library catalogues; OAPEN Library – a publication platform dedicated to hosting, preserving and distributing books; OAPEN OA Books Toolkit – public information resource for authors to build trust around open-access books. 

PKP (Public Knowledge Project) is a software and library project, consisting of three applications (Open Journal System, Open Pre-printer System and Open Monograph Press).  

The dialogue during this LIBER session wasn’t a mere presentation of these projects and their technical properties: the speakers emphasized the importance of ensuring the participation and the engagement of the stakeholder community, pointed out the crucial value of the support received – not only financial – from Research Libraries, and discussed how such Open Infrastructures can be beneficial for libraries. 

How can libraries support Open Infrastructures? And what role do they play in a long-term solution? According to Joanna Ball, from a librarian perspective, it’s not only a who-benefits-whom problem, but it’s more about finding a “third way, about developing mutually beneficial partnerships, and going beyond the traditional way of approaching things so that we can really play to each other’s strengths.” 

This approach is fully aligned with OpenCitations’ intentions. As Silvio Peroni underlined, in most of cases the active collaboration between Open Infrastructures and libraries is not only about the financial support, but in cooperatively reach a common goal. In particular, “if infrastructures like OpenCitations provide appropriate and easy-to-use interfaces and tools that allow librarians to contribute appropriate bibliographic metadata, and if librarians are willing to enter such metadata from their own records, libraries may become a significant reliable source of this kind of information”. The result of such a ‘crowd-sourced’ entry of bibliographic metadata by libraries would be an enrichment of the overall global open knowledge graph made available through citational links.  

In the last presentation, dedicated to two services provided by OPERAS, Emilie Blotière, (CNRS) and Tiziana Lombardo (Net7) reiterated the value of scholarly communication. COESO and GO TRIPLE, funded by the European Commission, aim in fact to create a persistent dialogue in the Social Sciences and Humanities community, by tackling the fragmentation and becoming a meeting point among different communities.  

What emerged from the session is the importance of communication, cooperation and networking between Open Infrastructures and Libraries, and this is a message that perfectly matches with the core values of LIBER, collaboration and inclusivity. The next LIBER annual conference is scheduled for June 2022 in Odense, hopefully recreating the physical and enthusiastic gathering of the previous meetings.  

You can find the recording of the full session here: LIBER 2021 Session #5: How Can Open Infrastructures Support the Role of Research Libraries? 

You can find the slides of the session on Zenodo.

More than 733M citations now available in COCI

Today, we have published the bi-monthly release of COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations. In this latest release (dated 6 September 2020), we extended COCI with more than 11 million additional citations. Now, COCI contains more than 733 million DOI-to-DOI citation links between more than 59.4 million bibliographic entities.

These new citations were harvested from the most recent Crossref data dump, downloaded on 19 August 2020, which includes the references of articles deposited in Crossref between 4 June 2020 and 3 August 2020.

We remind you that COCI has been fully described in our open-access article

Ivan Heibi, Silvio Peroni & David Shotton (2019). Software review: COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations. Scientometrics, 121 (2): 1213-1228. DOI: https://doi.org/10.1007/s11192-019-03217-6

and that all the bibliographic and citation data in COCI:

A final remark: COCI and other OpenCitations services will be the topic of a presentation that we will have in the context of the Workshop on Open Citations and Open Scholarly Metadata 2020. The workshop is a 3-hour event for researchers, scholarly publishers, funders, policymakers, and opening citations advocates, interested in the creation, reuse, and improvement, of open citation data and open scholarly metadata, with invited speakers. No registration is needed to follow it – we hope to see you there on 9 September at 15:00 CET!

Where you can follow the Workshop on Open Citations and Open Scholarly Metadata 2020.

OpenCitations updates: interviews, outreach, research positions, citations

Several important recent events have involved OpenCitations directly as a participant. Here we introduce some of the most significant ones:

  • Our interviews during a Fireside Chat with John Chodacki during the Open Publishing Fest;
  • The SCOSS poster about OpenCitations and the other two selected infrastructures during LIBER 2020, which won the LIBER 2020 Peoples Choice Poster Award;
  • The availability of two new short-term research positions with OpenCitations for our Wellcome Trust funded project (application closing deadline: 23 July 2020); and
  • A new release of COCI, bringing the total number of open citations available in this dataset to more than 721 million.

Open Publishing Fest

The Open Publishing Fest was a decentralized on-line public event held from 18 May to 29 May

to bring together communities supporting open source software, open content, and open publishing models.

(from the Open Publishing Fest website, last visited 4 July 2020)

Within this event, John Chodaki and Cameron Neylon organised a series of sessions named “Fireside Chats” in which they invited people from the open publishing community to discuss their careers and projects. David Shotton and Silvio Peroni were the guests in one of these chats with John Chodaki, in which we talked about the origins of OpenCitations and the plans for OpenCitations’ future. The video of our chat (see link below) is available on YouTube.

The recording of the fireside chat that David Shotton and Silvio Peroni had on May 28th 2020 with John Chodaki in the context of the Open Publishing Fest.

SCOSS at LIBER 2020: promoting open infrastructures

SCOSS, the Global Sustainability Coalition for Open Science Services, which selected OpenCitations last December as worthy of community crowd-funding support as part of its second funding cycle, participated in LIBER 2020 with a poster which showcases the infrastructures that they recommend that the community financially supports.

Tweet by SCOSS on the poster presented at Liber 2020.

This poster won the People’s Choice Award and confirms the crucial activity that SCOSS is doing in making the academic community aware of the importance of providing financial support to open infrastructures like OpenCitations for the benefit of the whole of society.

A tweet by SCOSS on the document they wrote on the importance of supporting open infrastructures.

Furthermore, Giannis Tsakonas, Director in the Library & Information Center at the University of Patras (Greece), and a member of the LIBER Executive Board as head of its Innovative Scholarly Communication Steering Committee, shared a wonderful thread on Twitter about OpenCitations and the other open infrastructures (DOAB, OAPEN, and PKP) that have been selected by SCOSS in its second funding cycle.

A thread by Giannis Tsakonas about us and the other open infrastructures selected by SCOSS for their second funding cycle.

Short-term research positions open at OpenCitations in Bologna for our Wellcome Trust funded project

Wellcome Trust announced they have extended all their grants that were due to end in 2020 or 2021, including our Open Research Fund funded project entitled “Open Biomedical Citations in Context Corpus”. This project aims at providing data for each individual in-text reference pointer (aka in-text citation) and its semantic context, making it possible to distinguish references that are cited only once in the text of a paper from those that are cited multiple times, to see which references are cited together (e.g. in the same sentence), to determine in which section of the article references are cited (e.g. Introduction, Methods), and, potentially, to retrieve the function of each citation.

Some preliminary outcomes of the project have already been described in a recent blog post, and a preprint describing some of the activities of the project has also been also made available on arXiv. That paper focuses on the extensions we have made to the OpenCitations Data Model, used for the storage of data in all the OpenCitations datasets, to enable the additional metadata types resulting from the Citations in Context project to be recorded.

In the context of the funding extension to this Wellcome Trust project, the Department of Classical Philology and Italian Studies (FICLIT) at the University of Bologna has just opened two new positions for short-term (5 months) research fellowships from August to December 2020 inclusive, for which the application closing deadline is 23 July 2020. The main goals of these the short-term research fellowship are (a) to develop the software for handling the data stored in the Open Biomedical Citations in Context Corpus, and (b) to develop indexing mechanisms to analyse a large number of documents simultaneously within our local computing environment, without having to use external services.

The net salary for each research fellowship is 1,600 EUR per month, tax free. The minimal requirement to apply for one of these positions is to have a Bachelor degree, although higher qualifications in Computer Science would be beneficial. Since these are University of Bologna positions, the application forms are in Italian. However, the description of the activity plan of the research fellowships is available in both Italian and English.  We would be happy to provide further information, and help in completing the application forms if necessary, so please do not hesitate to email us.

New release of COCI with an additional 18 million citations

Every two months we are able to publish additions to COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations. This latest release (dated 4 July 2020) extended COCI with more than 18 million additional citations, so that COCI now contains more than 721 million DOI-to-DOI citation links between more than 58.8 million bibliographic entities.

These new citations were harvested from the most recent Crossref data dump, downloaded on 8 June 2020, which includes the references of articles deposited in Crossref between 4 April 2020 and 4 June 2020. As before, we will use this new release of COCI to update the Coronavirus Open Citations Dataset, the third release of which will include details about relevant additional references and publications.

We remind you that COCI has been fully described in our open-access article

Ivan Heibi, Silvio Peroni & David Shotton (2019). Software review: COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations. Scientometrics, 121 (2): 1213-1228. DOI: https://doi.org/10.1007/s11192-019-03217-6

and that all the bibliographic and citation data in COCI:

COCI has surpassed 700M citations

We are excited to share that COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations, was on 12 May 2020 extended with more than 47 million additional citations, and has reached a total number of more than 702 million DOI-to-DOI citation links between more than 58 million bibliographic entities.

The citations added in this the fifth release of COCI came from the most recent Crossref dump downloaded on 22 April 2020, which includes the references of the articles deposited in Crossref between 4 October 2019 and 4 April 2020. Such updates to COCI will now occur regularly at bimonthly intervals.

As a consequence, COCI now contains 702,772,530 citations, and also includes publications about the COVID-19 pandemic. We will use this new release of COCI to update the Coronavirus Open Citations Dataset, the second release of which will include details about these additional references and publications.

COCI, which is fully described in our open-access article, was one of the subjects of a multidisciplinary comparison between the major citation indexes recently published on arXiv. In addition, it has been recently mentioned on the Scholix web site as one of the implementors of the Scholix citation data format.

Finally, we wish to remind you that all the bibliographic and citation data in COCI:

Search OpenEdition Search

You will be redirected to OpenEdition Search