If you are a leader of a Library or a Research Institution and would like to learn more about the existing open infrastructures that could help your institution to evolve in the research environment, but you don’t know where to look for, you can now use Infra Finder, a brand-new tool aimed at foster discovery, adoption, and investment for open infrastructure services. Infra Finder is launched by Invest in Open Infrastructure (IOI), and is designed to be the go-to resource for anyone navigating the complex landscape of infrastructure services and standards enabling open research and scholarship.
The tool stems from the feedback collected from the community after the launch, in 2022, of the Catalog of Open Infrastructure Services pilot project in 2022, from which IOI decided to re-envision what outcomes they could affect and rethink what solutions could look like, also by furthering the Open Science principles conveyed by UNESCO’s Recommendation on Open Science, the Principles of Open Scholarly Infrastructure, and other Open Science initiatives and guidelines.
We are happy to announce that OpenCitations is among the 56 infrastructures which have accepted IOI’s invitation to be included in this first release of Infra Finder. During its development, IOI team has been constantly collecting feedback from the infrastructures involved, and has conducted user testing to understand how different people might approach it. In OpenCitations we believe that Infra Finder could serve as an helpful tool to make OpenCitations and the other infrastructure services more visible to a wider and diverse community. Infra Finder allows users to easily find and compare the catalogued services, and its intuitive use could work as a starting point towards the creation of a greater awareness of existing open infrastructures, thus leading users to further explore their features and services and, hopefully, to consider the possibility of support their sustainability.
IOI is currently planning to include new services in Infra Finder: if you are part of an organization that would like to be included, you can apply by filling out their Expression of Interest form. IOI will give two identical webinars to share with the community more details about the background and features of Infra Finder and on how interested infrastructure services can express interest in being listed. Register here to the either slot:
The first month of the new year has almost come to an end, and we at OpenCitations have dedicated these weeks after the holiday season to retrace the progress we reached as an open infrastructure throughout 2023, an activity that has become a tradition in the past few years. Looking back to the achievements has proven to be a great practice for our team not only to honour and celebrate the success, but also to visualize the points to focus on to best serve our community when planning future activities. During 2023 we have reached numerous milestones, which can be summarized under the two main topics: technical developments and new connections.
Technical Developments
1. New sources
At the end of 2022, we had enriched our collection with two more sources, PubMed and DataCite, which were added to Crossref. During 2023 we completed the integration between OpenCitations and the OpenAIRE Graph as concerns open bibliographic and citation data, and in December we included the DOI-to-DOI citation links derived from the Japan Link Center (JaLC). As a result, by the end of 2023, OpenCitations counted almost 2 BILLION open citations available in its collection.
2. A brand-new workflow for an easier consultation
The progressive integration of new sources is one of OpenCitations’ main objectives. However, the recent source increase made it necessary to design a new ingestion workflow to facilitate their management. In November 2023 OpenCitations launched the new workflow, designed to produce just two comprehensive collections: OpenCitations Index, collecting open citation data, and OpenCitations Meta, containing open bibliographic metadata of citing and cited entities involved in citations. This evolution is making it simpler for OpenCitations users to consult the data maintained by our infrastructure without reducing the information content and accuracy.
New connections
3. A great event for the community
In October 2023, OpenCitations hosted the Workshop on Open Citations and Open Scholarly Metadata 2023 (WOOC 2023) in the historical location of the University of Bologna. After two online editions, it was exciting to meet in person the researchers, policy makers and service providers who in the past few years have collaborated with OpenCitations in many ways. For two days, almost 80 people from all around the world gathered to discuss the theme “The value of open scholarly metadata for Research Assessment purposes”. The event gave us a picture of a young and enthusiastic research community, aiming to collaborate among different countries to spread the message “Open Science is Better Science”. If you couldn’t attend the workshop, but are interested in the theme, you can find all the presentations from WOOC 2023 on Zenodo. Stay tuned on the WOOC X account and website, we will announce the 2025 edition later in 2024!
4. New and old partnerships
OpenCitations’ main activities, besides the technical development, are tied to its participation in R&D and funded projects. Since 2019, we have started building a network of collaborations all around the world with institutions which share the same Open Science values. In 2023, our involvement in the SCOSS 2nd funding cycle has come to an end. In the last four years SCOSS has worked as an international showcase for OpenCitations, making it possible for us to get known by many institutions on an international level. We are now proud to keep on collaborating with SCOSS within the SCOSS Family network, a community of practice gathering all the OS infrastructure involved in the SCOSS funding cycles. During 2023, the SCOSS Family was involved in four internal sprints dedicated to planning four actions – collectively advocating the importance of open infrastructure, communications kit for ‘the post-SCOSS phase’, collective self-assessment towards POSI, and map open science infrastructure ecosystem including SCOSS infrastructures – and the initial outcomes were presented in the Scoss Family Public Webinar 2023. The SCOSS family will continue its activities throughout 2024.
In 2023, we also celebrated the update of the Principles of Open Scholarly Infrastructure (POSI), as a result of the work conducted since 2022 with the “POSI-Adopters” group to make the principles clearer and more inclusive. In June, the OpenAIRE-Nexus project, started back in 2021, formally came to an end, with the release of the index of citations from OpenAIRE. OpenCitations is part of the OpenAIRE Catalogue, which also links to the exhaustive and engaging materials we produced within the project. However, we continue to meet the familiar and friendly faces from the OpenAIRE team in the meetings of GraspOS, a project which started in January 2023 involving several European Open Science institutions, services and infrastructures. In GraspOS, OpenCitations is involved as one of the Data Asset Sources, which provides both the raw and the structured data that can be used to directly support OS-aware RRA processes or to calculate indicators or collect evidence that can be of value in these processes. The researchers from OpenCitations and the University of Bologna are contributing to GraspOS by developing a tool for the extraction of citation data from PDFs, enriched with additional information on the reasons for citing (because the citing entity reuses a method described in the cited entity, for agreeing with the claims of the cited entity, etc.) through the implementation of machine-learning systems. Also, together with GraspOS, OpenAIRE, and many other Coalition members, the OpenCitations personnel at the University of Bologna is among the participants of the CoARA Working Group Towards Open Infrastructures for Responsible Research Assessment (OI4RRA), to work on enabling the research community to be in full control of the data and indicators it relies on in assessments. OpenCitations is always seeking interest from new institutions, and we are proud that our visibility is boasted by some reliable catalogues including the OpenAIRE catalogue, the EOSC Marketplace and, from 2023, also the JISC Catalogue. And soon you’ll find us in a new tool – stay tuned to find out!
5. Our way towards sustainability
OpenCitations is increasingly gaining a prominent role in the community, establishing itself as a key player in changing research evaluation practices towards greater openness and equity. In recent years, OpenCitations has grown a lot both technically and in popularity, and for this we want to thank the number of members and donors who, even in the last year, have decided to support us and believe in our infrastructure. During 2023, our two Internal Working Groups “Governance Evolution” and “Community Building”, thanks to the participation of some members of the International Advisory Board for OpenCitations, reflected on the sustainability of OpenCitations, discussing strategies both at the management and communication level to ensure the integration of new actors into its subsistence. The groups will continue to meet throughout 2024, and some of the results will be released in the coming months. However, there is still a long way to go towards sustainability, and OpenCitations’ existence still relies on community funding, which is why OpenCitations still needs support: if you are looking for more information on why and how to support OpenCitations, you can read our recent call for action here. Thank you!
6. Enlarging our team
It is thanks to the support received that OpenCitations was able to develop its yearly activities, and we are happy to announce that in 2023 a new software developer, Elia Rizzetto, and a new PhD student, Francesca Cappelli, have joined our team to make this progress possible. The OpenCitations team now has a total of 12 members, including research fellows, PhD students, MA students, a Reseach Manager, an Assistant Professor and OpenCitations’ Directors. The number is expected to grow soon, as OpenCitations has recently released a call for the hiring of a System Administrator for technical infrastructure management.
As can be seen from this brief (and detailed) report of some of the developments that have marked our 2023, OpenCitations is increasingly emerging as a community-based and community-guided infrastructure organization, in which there is an indissoluble link between technical progress and the network of people who guide the planning, work on implementation, and use or promote our services. Throughout 2024, OpenCitations aims to focus even more on harmonizing the relationship between the technical infrastructure and the community, developing more intuitive ways of accessing services and communications, and continuing to weave relationships with new research groups. We look forward to seeing what the year ahead will bring with it, and we hope you will continue to be with us.
2023 has arrived, and it’s the time to put aside the old agendas and to fill the freshly printed organizers with new tasks. However, it is also worth looking back over the achievements of the past year.
So, before consigning 2022 to the archives, we want to take a moment to recall and celebrate all that OpenCitations has achieved over the past year, in terms of technical developments, community building, and our own internal organization.
Clarifying our objectives
Since its foundation, OpenCitations has been presented to the scholarly community as a infrastructure organization guided by the values of openness, equity and accessibility. In 2022, we publicly clarified our mission, our aims and our unique points by publishing these three documents:
In addition, we continued to monitor OpenCitations’ compliance with the Principles of Open Scholarly Infrastructure. Because one area in which we were still falling short involved stakeholder involvement in OpenCitations’ governance, in 2022 OpenCitations started reflecting on its organizational structure by creating a Governance Evolution Working Group. In addition, we explored the further development of our stakeholder community by establishing a parallel Community Building Working Group. These two small working groups, involving members of the OpenCitations International Advisory Board, will continue to meet throughout 2023 to develop strategic plans concerning OpenCitations open governance structure and outreach.
More people, more tools
In 2022, OpenCitations welcomed two new full-time developers to its team: Arianna Moretti as software developer, and Ivan Heibi as the person responsible for OpenCitations’ technical infrastructure. As of January 2023, the OpenCitations team consists of our two directors Silvio Peroni and David Shotton, our administrator and research manager Claudio Fabbri, myself as our communications director and community development manager, and our four developers: Giuseppe Grieco (software and systems developer), Arcangelo Massari (developer of OpenCitations Meta), Arianna Moretti (software and indexes developer), and Ivan Heibi (technical infrastructure manager). This team meets weekly at the Research Centre for Open Scholarly Metadata within the University of Bologna.
This enlarged technical workforce has been crucial in the development of the two new OpenCitations indexes, DOCI (citations from DataCite) and POCI (citations from PubMed), and our new database of bibliographic metadata, OpenCitations Meta, all of which were released in December 2022:
DOCI, the OpenCitations Index of DataCite open DOI-to-DOI citations, is an RDF dataset containing details of all the citations that are specified in the most recent dump of DataCite metadata. The citations available in DOCI are treated as first-class data entities, with accompanying properties including the citations timespan, modelled according to the OpenCitations Data Model. Its first release (December 2022) contains information on 169,822,752 citations from 358,277 bibliographic resources.
POCI, the OpenCitations Index of PubMed open PMID-to-PMID citations, is an RDF dataset containing details of all the citations from publications bearing PubMed Identifiers (PMIDs) to other PMID-identified publications, harvested from the National Institutes of Health Open Citations Collection (NIH-OCC). As for DOCI, the citations available in POCI are treated as first-class data entities, with accompanying properties including the citations timespan, modelled according to the OpenCitations Data Model. Its first release contains 717,654,703 citations from 26,024,862 bibliographic resources.
OpenCitations Meta database stores and delivers bibliographic metadata for all the publications involved as citing or cited publications in the citations recorded in the OpenCitations Indexes. For each publication, the metadata exposed by OpenCitations Meta includes the publication’s title, type, venue (e.g. journal name), volume number, issue number, page numbers, publication date, and identifiers such as Digital Object Identifiers (DOIs) and PubMed Identifiers (PMIDs). In addition, OpenCitations Meta includes details of the main actors involved in the document’s publication, i.e., the names of the authors, editors, and publishers, each with its own additional metadata and identifier (e.g. ORCID). Its first release (December 2022) contains information on 87,321,593 bibliographic entities, 277,750,235 authors and 2,359,301 editors (counted by their roles, without disambiguating individuals), 710,226 publication venues, and 17,268 publishers.
As can be seen on OpenCitations roadmap, we have many tasks already scheduled for 2023, and for this reason OpenCitations team needs to expand further: we now seek applicants for a new Research Fellow position to work on the further development of the OpenCitations technical infrastructure from April 2023.
Our role in the community
Throughout 2022, the OpenCitations team has participated both online and in person in a number of international events, presenting details of OpenCitations to potential stakeholders and to Open Science experts, and connecting with similar organizations.
In October, OpenCitations organized our third workshop, the Workshop on Open Citations and Open Scholarly Metadata 2023, as a three-hour online event, to gather together and connect researchers, publishers and policy makers interested in the creation, reuse and improvement of open citation data and open scholarly metadata. The invited speakers provided a broad overview of the current state and future development of open bibliographic metadata and the scholarly record, and the role of OpenCitations in research assessment.
We have participated in two major European research and development project: In the context of the EC-funded project RISIS2we released the new database OpenCitations Meta and the index DOCI, while within the OpenAIRE-Nexus project network we continue to provide open bibliographic citations as part of the open data components of OpenAIRE and the European Open Science Cloud. During 2022, the OpenAIRE team supported our activities in many areas, with a particular attention to communication and dissemination, as we explained in our blog post OpenCitations and EC funding: OpenAIRE Nexus and RISIS2.
The end of 2022 brought with it two new exciting collaborations that will be developed during the coming year: the inclusion of OpenCitations in the Jisc Collections, thereby permitting UK institutions to support OpenCitations through JISC; and our involvement in the GraspOS project (starting date January 1, 2023) as an open and trusted federated infrastructure for next-generation research metrics and indicators.
Continuing support
At OpenCitations we are deeply grateful to all our members and donors that have supported us financially so far, thus making it possible to enlarge our team, maintain our technical infrastructure, and work on our new data releases.
Much of our success has been due to our selection as an infrastructure worthy for inclusion in the SCOSS second pledging round in 2019. SCOSS helped us get in touch with a wider community, and enabled OpenCitations to have an increasingly significant role in the research environment. OpenCitations is now part of the SCOSS family, the community of all SCOSS-supported infrastructures, that provides a framework of mutual support for the the Infrastructures during their growth and a safety net should troubles occur along the path.
However, OpenCitations still has a long way to go along the path from being a sustainable open infrastructure to becoming a sustained open infrastructure providing reliable and invaluable services to our global stakeholder community. There are still many long-planned activities that we hope to initiate in the near future, given sufficient resources. This is why OpenCitations requires ongoing support from the scholarly community as our three-year period of SCOSS sponsorship draws to a close. All the information you may require to start helping us financially is available on the OpenCitations website: https://opencitations.net/membership.
2023 has just started. With your support, you can help us in making it an impactful year for OpenCitations and the Open Science community as a whole.
Posted on August 10th 2022 by Chiara Di Giambattista
More than a year ago, Ginny Hendricks, Director of Member & Community Outreach for Crossref, and a valued member of the OpenCitations International Advisory Board, published on the Crossref blog the post “The road ahead: our strategy through 2025”. In order to describe all Crossref’s principles and activities, Ginny presented the Crossref strategic planning framework as a diagram summarizing Crossref’s statements, key messages and truths. The clarity and immediacy of the diagram were such that we adapted it to present OpenCitations’ own statements and goals. The resulting poster “OpenCitations – what does the future hold?” was presented by our Director David Shotton at the OASPA2021 conference, and can be found in this blog post.
Although the poster offered a wide overview of OpenCitations values, unique traits, benefits and plans, it differed slightly from Ginny’s original diagram, in particular because it lacked a “Mission Statement”, scattering the relevant information within the “Values” and “Principles” boxes. Indeed, at that time (September 2021), we didn’t have a clearly defined Mission Statement.
Nevertheless, the creation of that poster was crucial in helping us start to articulate more clearly the purpose and meaning of OpenCitations. As David underlined in his post “From little acorns…a retrospective on OpenCitations”, since 2018 OpenCitations activities have progressively increased and, with them, the number of related journal articles, conference papers and technical definitions. OpenCitations’ involvement in international networks and collaborations (such as SCOSS and the OpenAIRE-Nexus project), together with our need of identifying and reaching out to new stakeholders to assure OpenCitations’ development and sustainability, has made it necessary to publicly define OpenCitations’ mission, unique strengths and next developmental steps.
After numerous revisions, aided by wise advice from members of the OpenCitations Advisory Board members, we’re now happy to publish the following three OpenCitations documents:
which together provide a summary of why we exist and where we are heading.
We are particularly proud of the definition of OpenCitations’ primary mission, namely
“to harvest and openly publish accurate and comprehensive metadata describing the world’s academic publications and the scholarly citations that link them, and to preserve ongoing access to this information by secure archiving.”
The Mission Statement also presents brief descriptions of the OpenCitations context, our vision, our value proposition and our relationship with the community and stakeholders.
The Uniqueness of OpenCitations provides the answer to the question ‘Why choose to use OpenCitations?’, and is a detailed presentation of OpenCitations’ benefits.
OpenCitations – Present Status and Future Plans summarizes OpenCitations’ ongoing activities, that can be quickly visualized on our public roadmap. It also introduces the OpenCitations Working Groups, served by the members of the OpenCitations International Advisory Board, which are currently working on the themes of governance evolution and community building, with the common purpose of driving OpenCitations along the path from being a ‘sustainable infrastructure’ (in POSI terms) to being an enduring community led andfinancially sustained infrastructure.
In fulfilling our mission and reaching our goals, the support and vital interest of our community members is fundamental. We request that you, as a member of our community, provide us with feedback on these documents and the ideas they contain, or indeed to ask for clarifications, to help us improving our mission and our communications to explain it. You can reach us here: contact@opencitations.net.
In March, The Global Sustainability Coalition for Open Science Services (SCOSS) celebrated, together with the generous funders and the projects involved (including OpenCitations), the achievement of an amazing milestone: a total of 4 million Euros raised so far for supporting the growth and development of Open Science Infrastructures. This significant sum is not just a number, but a concrete sign of the commitment of numerous institutions all over the world to ensure the success of vital organs of the Open Science ecosystem. Thanks to the pledges recently made, the new infrastructures selected for the third SCOSS pledging round have now started developing their services and can now look to the future with more surety.
At the end of 2019, OpenCitations was selected by SCOSS for its second pledging round, and since then much progress has been made. As the OpenCitations’ founder and director David Shotton recently stated:
“OpenCitations is growing, thanks to the generous support from our members and donors, and we thank SCOSS sincerely for bringing us into contact with them. The citation coverage provided by OpenCitations is now approaching parity with that of the leading commercial citation indexes, and our ambition, within the next five years, is for OpenCitations to be routinely used by our worldwide stakeholders as their primary source of comprehensive scholarly citation information”.
In 2020, OpenCitations monitored the achievements of the first year of SCOSS support and shared the most important updates in a blog post. After a review from SCOSS Advisory Group and the SCOSS Board, the OpenCitations 2021 report to SCOSS is now available in the SCOSS May newsletter. We’re proud of the successful developments that 2021 brought with it in many areas, from the technical enhancements to OpenCitations to its new supporters and partnerships.
After a two-year-long collaboration, we in OpenCitations recognise that one of the most precious benefits of being part of SCOSS is working within a community: SCOSS not only provides a framework but also a real family of supportive institutions that support the Infrastructures during their growth and provide a safety net if troubles occur along the path. The bi-monthly meetings organized by the SCOSS team enable dialogue with the other infrastructures within the same pledging round, while presentation and promotion to the institutions worldwide are fostered by participation in webinars and conferences. During 2021, OpenCitations participated in 16 events, and in 5 of them (LIBER Webinar 2021, JISC Webinar 2021, LIBER Annual Conference 2021, Open Science Fair 2021, and Open Access Tage 2021) OpenCitations’ director Silvio Peroni gave talks together with the representatives of others infrastructures involved in the SCOSS second pledging round.
In compliance with the POSI principles, SCOSS encouraged OpenCitations to set up an open governance structure. As described here, the organizational bodies included in the OpenCitations present governance are three: the Directors, the International Advisory Board, and the Council. Although executive power is still currently vested in the hands of the Directors, in the last two years the OpenCitations International Advisory Board, a committee that now comprises nine Open Science experts from different professional and academic backgrounds, has had a crucial role in guiding the OpenCitations activities, and it is now working on strategic developments in terms of collaborations, policies, support and governance. Moreover, last December OpenCitations organized a Webinar in conjunction with its Annual Meeting to present and discuss with the OpenCitations Council members OpenCitations’ recent developments and future plans.
OpenCitations is highly reliant upon the connections we have created and on people working together: thanks to the support we received throughout the last two years, the OpenCitations team has grown and now includes six people employed by the Research Centre for Open Scholarly Metadata at the University of Bologna, the administrative body for OpenCitations. Besides the previously announced appointments of Claudio Fabbri (Research Manager), Chiara Di Giambattista (Communications Director and Community Development Manager), Giuseppe Grieco (Software and Systems Developer), and Arcangelo Massari (Software Developer, working within the OpenAIRE-Nexus project), early in 2022 Ivan Heibi, Ph.D. candidate at the University of Bologna, joined OpenCitations with responsibility for the technical infrastructure, and Arianna Moretti, who recently graduated in Digital Humanities and Digital Knowledge, joined as a software developer. This enlarged team, involving young and motivated researchers, is working on numerous projects to be announced later in 2022. You can learn more about OpenCitations’ ongoing activities on our public roadmap.
OpenCitations’ aim continues to be the publication of open data describing the bibliographic citations linking global scholarly publications, with depth and scope, while maintaining the highest standards of accuracy and provenance. Most importantly, OpenCitations services will always be free, making global scholarly citation data available at zero cost and without restriction for third party analysis and re-use.
With its support, SCOSS has been helping OpenCitations to pursue its mission and spread the benefits of Open Science. However, 2022 will be the last year of the three-year-long support ensured by SCOSS. OpenCitations activities won’t stop in 2023 — indeed there still are many long-planned activities that we hope to initiate in the near future, given sufficient resources. This is why OpenCitations requires ongoing support from the scholarly community. All the information you may require to start helping us financially is available on the OpenCitations website: https://opencitations.net/membership
By supporting OpenCitations, you will embrace and sustain the ideals and vision of Open Science, and you will help in creating a more open, democratic and fair knowledge environment.
On September 27, OpenCitations’ director Silvio Peroni, together with Niels Stern (DOAB/OAPEN) and James MacGregor (PKP), held the online workshop “How Open Infrastructure Benefits Libraries” during the Open Access Tage 2021. Open-Access-Tage (Open Access Days) are the annual central platform for the steadily growing Open Access and Open Science community from Germany, Austria and Switzerland, and are aimed at all those involved with the possibilities, conditions and perspectives of scientific publishing.
The workshop gathered three of the SCOSS-supported infrastructures to discuss how Open Infrastructures (OIs) could encourage the engagement of university libraries, and how they could be beneficial game-changing alternative to commercial infrastructures. This theme, which was also been presented during the last LIBER conference, was here discussed under a new perspective, by involving the specific case of the libraries from the German-speaking region. Their point of view particularly emerged during the second part of the workshop, during which the participants were divided into two breakout rooms to discuss two questions each. These are the answers and comments that emerged from the discussions.
1. What would prevent or encourage libraries in the German-speaking region to support open infrastructures?
The three main concepts held to be crucial in this field were transparency, promotion and governance.
Transparency: German libraries and public institutions often deal with strict funding limitations relating to donations. It is therefore crucial for OIs (a) to present in a clear way how libraries can get involved and the money needed, (b) to communicate what they do and how they can add value to libraries compared to other services, and (c) to clarify the direct return and benefits on investments. These points would make it easier to recommend OIs internally, especially when people from subject-specific institutions are interested in subject-independent OIs. Point (b) leads to the Promotion issue: Open Infrastructures should promote themselves non only on a global level, by communicating their impact in the open research movement as against non-transparent propitiatory services, but also at a local level, by providing information about the usage (and the value) of their services at an institutional and/or national level. This case-by-case narration (with attention to the specific benefits) would make it easier for the institutions to evaluate the sustainability of the investment. An incentive to donate is being actively involved in the community governance, i.e.through a board membership.
Nevertheless, is also necessary for libraries to “take courage” when investing in such OIs, and, when possible, to overcome administrative boundaries by forming consortia. Finally, of particular note was a desire to see locally-managed sub-communities that could speak specifically to the German (or whichever) language environment, much as ORCID arranges itself.
2. Community Governance. What kind of involvement do you want to see, how do you want to be involved?
Some common problems which prevent the institutions from being involved are (a) a general concern about the fact that negotiations with publishers are typically the main focus of OA discussions – leaving little time to focus on OIs and other open initiatives, and (b) a lack of time, or of guidelines, for evaluating the different infrastructures to invest in. This is why SCOSS was appreciated as an intermediary in the decision process, because of its own rigorous evaluation and selection mechanism. The community-funding approach proposed by SCOSS thus seems to be the preferred way by which to support OIs.
Regarding community governance, one idea could be to involve interested scholars in the governance of the open infrastructures (with the library acting as an interface between the open infrastructure and the scholars) rather than only involving library staff – although this idea was argued against in the second group, as researchers are often percieved as too busy to be functional in operational infrastructure groups. What also emerged from this second question is an interest in community involvement on different levels, for example as a community of practices or through discussion boards, mailing lists, periodic meet-ups, workshops, newsletters, etc. The community could also be articulated into local sub-communities, as in the successful case of ORCID and ORCID_DE.
No-one is quite sure of the total number of scholarly publications within the global corpus. Indeed that number will be strongly influenced by the degree to which, in addition to books and journal articles, one includes within the definition of scholarly publications ‘grey literature’ such as reports published by official bodies, patents, etc. Consequentially, the total number of scholarly references within those publications is also unknown, and this number too will vary according to the inclusion criteria chosen. Furthermore, Crossref Event Data and similar datasets recording social media mentions of journal articles in blog posts and tweets extends the concept of a reference beyond that used in ‘conventional’ citation indexes such as COCI.
We celebrate the fact that well over one billion bibliographic citations are now openly available under CC0 waivers in NIH OCC (the National Institutes of Health Open Citation Collection) [1,2] and COCI (the OpenCitations Index of Crossref DOI-to-DOI Citations) [3]. Despite present gaps in their coverage, they include references to all the most important publications within the global corpus, because these will all have been cited multiple times.
Open references available from Crossref and other aggregators and indexes
Crossref, with over 1.6 billion open references, is the largest single source of such bibliographic metadata. Significant numbers of references are also available in a variety of other databases, repositories and indexes.
NIH OCC (the National Institutes of Health Open Citation Collection) is a merger of several citation databases, drawing on PubMed for crucial article metadata, and augmenting this with information from full-text articles that have been made freely available on the internet [1]. The CiteSeerX database, the arXiv preprint repository, and the Dryad data repository are examples of different types of infrastructure that also publish open bibliographic references, while there is further availability of article references from open aggregators such as DataCite and Wikidata. These may either use their own DOIs, DOIs from the Crossref DOI registration agency, or no DOIs at all. Either way, those references will not appear in Crossref.
What is lacking is semantic coherence and interoperability between these sources, permitting federated queries across them. This makes difficult the task of obtaining a comprehensive overview of the availability of open bibliographic references.
However, there are even more citations that are not yet freely and easily available anywhere in bulk, relating to the reference lists within publications of a number of distinct types. This blog post explores academia’s missing references – those that have not yet been documented within open freely accessible citation indexes – and what might be done to bring these into the public domain.
1 References that are closed at Crossref
Eight years ago, I wrote
“In this open-access age, it is a scandal that reference lists from journal articles — core elements of scholarly communication that permit the attribution of credit and integrate our independent research endeavours — are not readily and freely available for use by all scholars.” [4]
I stand by that statement, and, through OpenCitations [5], I have been working with colleagues to rectify the situation, as I described in my previous post. The Initiative for Open Citations (I4OC) can rightly be applauded for its part in encouraging almost all the major academic publishers who deposit references at Crossref to make them open.
The only major scholarly publisher not to be listed as an I4OC Participating Publisher is the Institute of Electrical and Electronics Engineers (IEEE), that, having deposited at Crossref reference lists for 58% of its preprints and publications, persists in keeping these deposited references closed and unavailable for indexing and re-use. It is to be hoped that IEEE will now realize that what it looses by not having references openly available outweighs any benefits it might have received from keeping them closed, and will join its fellow publishers in ensuring that its Crossref-deposited references are made open, both for its current issues and for its back-number journal articles. I thus call again upon IEEE to change its present position, as both Elsevier and the American Chemical Society had the courage to do recently, and to instruct Crossref to open all IEEE references. A single email to Crossref Support, with the instruction “Open all references”, is all it would take!
References in Crossref are either open, ‘limited’ or closed. Limited references are available to those subscribing to Crossref Metadata Plus, which includes OpenCitations, but not to the general public. Closed references are not freely available to anyone. The following table shows the number of Crossref works in each category, and the number of references within those categories.
Works
References
Average references per work
Total
126,627,618
Without references
70,477,843 (55.7% of total)
With references
56,149,775 (44.3% of total)
1,734,831,311
30.9
Open
49,274,155 (38.9% of total)
1,605,120,229
32.6
Limited
2,933,323 (2.3% of total)
66,459,704
22.7
Closed
3,942,297 (3.1% of total)
63,251,378
16.0
Table 1. Numbers of works and their references recorded in Crossref on 31 July 2021.
2 References not deposited at Crossref for publications with Crossref DOIs
The number of works with Crossref DOIs that lack submitted references is surprisingly large. As of 31 July 2021, 70,477,843 publications (55.7% of all works recorded at Crossref) lacked deposited references (Table 1).
Crossref classifies all types of journal content, including editorials, book reviews and letters, as “journal articles”, thus some of these works without deposited references genuinely lack them. However, the majority are conventional journal articles and books with reference lists that the publishers have simply not deposited at Crossref along with the other metadata for these works.
The average number of references per Crossref work with deposited references is 30.9 (Table 1). If, to make allowance for those works that genuinely lack references, we assume a conservative average of 25 references per work for the 70,477,843 works lacking deposited references, this means that there are over 1.75 billion references within these works that have not been deposited at Crossref, and thus are not conveniently available for indexing and reuse.
These missing references relate both large publishers that have submitted references for only some of their publications, and small publishers that perhaps lack, or think they lack, the resources to deposit any reference lists in addition to the other metadata they are already sending to Crossref for each of their DOIs. However, there are several easy methods for depositing reference lists, as detailed by Crossref here. So I encourage all publishers who are supportive of Open Science to update their procedures and commence or complete the deposition of their publication reference lists at Crossref, starting with their current issues. Crossref Support will provide assistance if required. Note that a publisher does not have to subscribe to the Crossref Cited-by service to deposit its references!
3 Citations missing in COCI: open references in Crossref to publications lacking DOIs
COCI is the OpenCitations Index of Crossref DOI-to-DOI Citations, and, as the name suggests, it indexes Crossref open references from works with Crossref DOIs to other works that have DOIs [3]. It therefore does not index open references in Crossref to works that, for whatever reason, lack DOIs.
The most recent (September 2021) release of COCI, based on the August Crossref dump, contains 1,186,958,898 citations between 69,074,291 unique work, comprising 51,103,720 citing bibliographic resources bearing Crossref DOIs and 56,105,783 cited bibliographic resources. Of the cited bibliographic resources, 38,135,212 bear a DOI issued by Crossref and have open or limited references (thus also being COCI citing resources), while 17,970,571 either have a Crossref DOI but lack open or limited references or have a DOI issued by another DOI registration agency such as DataCite (thus not being COCI citing resources).
Note that in Crossref, the ratio of works with open or limited references to works without open or limited references is 0.7:1 (Table 1). However, in COCI, the ratio of cited works with Crossref DOIs containing open or limited references to all other cited works is 2.1:1. Thus works with Crossref DOIs containing open or limited references are three times more likely to be cited than works that either have a Crossref DOI but lacking open or limited references or have a DOI issued by another DOI registration agency. This is most likely because the most important journals from almost all the larger publishers now have open references. However, it is still a remarkable ratio.
Because references to works lacking DOIs are not included in COCI, the average number of bibliographic references per citing article in COCI is only 23.2, in contrast to the numbers of references to works of all types given in Table 1.
From these data, it can be seen that there are over 480 million open Crossref references to a wide variety of works lacking DOIs that OpenCitations does not index in COCI. This is because of an intentional and fundamental limitation in the structure of the Open Citation Identifier (OCI) [6], requiring both citing and cited publications to have identifiers of the same type, that lies at the heart of the functionality of our OpenCitations Indexes.
OpenCitations is currently developing a solution that, without compromising that intentional design limitation in OCIs, will nevertheless permit us to index and publish these ‘missing’ references as Linked Open Data. We will report on this development in due course.
Crossref Event Data is a Crossref service / database that records mentions of publications bearing Crossref DOIs in social media including tweets and blog posts, and in other non-traditional citation sources such as news items and Wikipedia articles. From today (Thursday 23rd September 2021), Crossref Event Data will start to include in its holdings open references from publications bearing Crossref DOIs to other publications bearing DOIs. Limited and closed references will not be included. Initially, open references from current publications will be included in Crossref Event Data, with open references from older works with DOIs being added later. In that respect it will come to resemble COCI, except that COCI also included ‘limited’ references, treats citations as first-class data entities with their own identifiers, and makes its citation data available in RDF as Linked Open Data, as well as via a REST API. Subsequently, Crossref Event Data will also record references to publications with other forms of identifier, as OpenCitations also plans to do.
4 References available on publishers’ web sites
Many publishers, particularly those of Open Access works, already make their publication reference lists openly available on their own web sites. While this is commendable, it is not sufficient, since, if scholarly references are not made available in a centralized aggregator such as Crossref from which they can be conveniently harvested in bulk for analysis and re-use, they are much more difficult to access.
Scraping references from the HTML of individual web sites is difficult, time-consuming and liable to be incomplete. While Microsoft Academic achieved considerable success in scraping references from publishers’ web sites, possible because of the special relationships these publishers have with the Microsoft search engine Bing, this service will unfortunately soon no longer be available, illustrating a problem inherent with scholarly infrastructures provided by commercial companies that do not adopt the Principles of Open Scholarly Infrastructures.
A consequence is that such publications will become increasingly ‘invisible’, as bibliographic and analytical services come to rely more and more on centrally available data.
6 References within PDFs of scholarly works lacking DOIs
There is a large but unknown quantity of reference-containing books, academic reports, patents and journal articles from publishers that, for their own good reasons, choose not to use DOIs. The text of many of these publications is already available in a marked-up machine-readable format such as JATS, used in preparation for publication, from which the reference lists could easily be extracted. Other publications are only available as PDFs, both as published Versions of Record, or as preprints deposited in a variety of preprint repositories such as arXiv or CORE. Mining reference lists out of PDFs required expertise in text mining and AI technologies, and is labour-intensive, since it usually required the tuning of extraction algorithms to handle the particular styles and formatting of individual journals, one at a time. Two stages are involved: first, the recognition and extraction of the text of the individual references from the PDF, and second the parsing of each text string into the component parts of the reference (author names, title, publication year, etc.) A considerable number of the citations within NIH-OCC have been obtained in this manner [1], commercial companies such as Lexical Intelligence specialize in this area, and publicly available software such as GROBID is available for the purpose. However, the overall task of extracting ‘missing’ academic references from the global PDF corpus is daunting in magnitude and would require a well-funded organization.
The correct way to proceed would be for each publisher to take responsibility for liberating the references of their own publications, whether or not the publications themselves are open access, and whether or not these references are already available in a marked-up machine-readable format or only within PDF documents. Then, if the publisher still chose not to use DOIs and to submit these metadata to Crossref, these references could be submitted directly to OpenCitations for aggregation and publication as Linked Open Data.
Conclusion
From the foregoing discussion it is clear that the academic community has a long way to go before the majority of scholarly citations, the products of their own labours, are openly available for analysis and re-use. We at OpenCitations are working to address these issues and to publish more of these missing citations. However, completion of the task will require a coordinated collaborative international effort.
Are you willing to be involved?
References
[1] B. Ian Hutchins et al. (2019). The NIH Open Citation Collection: A public access, broad coverage resource. PLoS Biol.17 (10): e3000385. https://doi.org/10.1371/journal.pbio.3000385
[2] B. Ian Hutchins (2021). A tipping point for open citation data. Quantitative Science Studies2 (2): 433–437. https://doi.org/10.1162/qss_c_00138
[3] Ivan Heibi, Silvio Peroni, David Shotton (2019). Software review: COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations. Scientometrics121 (2): 1213-1228. https://doi.org/10.1007/s11192-019-03217-6
[5] Silvio Peroni, David Shotton (2020). OpenCitations, an infrastructure organization for open scholarship. Quantitative Science Studies, 1(1): 428-444. https://doi.org/10.1162/qss_a_00023
“Infrastructure at its best is invisible. We tend to only notice it when it fails. If successful, it is stable and sustainable. Above all, it is trusted and relied on by the broad community it serves. Trust must run strongly across each of the following areas: running the infrastructure (governance), funding it (sustainability), and preserving community ownership of it (insurance)”.
These areas are fully define the Principles of Open Scholarly Infrastructure (POSI), which provide a set of guidelines by which open scholarly infrastructure organizations and initiatives that support the research community can be run and sustained.
As far as we are aware, Crossref was the first infrastructure to publish its compliance with POSI, detailed in Geoffrey Bilder’s December 2020 blog post
OpenCitations too espouses POSI and, in January 2021, we monitored the extent of our own compliance with POSI, the results of which are shown in the following diagram.
Governance
●
Coverage across the research enterprise
We gather citations from global scholarship
●
Stakeholder governed
Advisory board currently lacks executive power and is not elected
●
Non-discriminatory membership
Membership open to all those espousing open science
●
Transparent operations
Everything is open
●
Cannot lobby
OpenCitations lobbies to achieve open scholarly citations and bibliographic metadata; it does not engage in political or financial lobbying
●
Living will
Since all our data open, others can recreate our service
●
Formal incentives to fulfill mission & wind-down
No formal plan for wind-down has yet been drawn up
Sustainability
●
Time-limited funds used only for time-limited activities
Grant income should be used solely for grantprojects
●
Goal to generate surplus
Goal not yet realized – income so far too limited
●
Goal to create contingency fund to support operations for 12 months
Goal not yet realized – income so far too limited
●
Mission-consistent revenue generation
Membership fees and solicited donations
●
Revenue based on services, not data
All data and services freely given to community, and thus do not generate income
Insurance
●
Open source
All software under open source licenses
●
Open data
All data available under CC0 waiver
●
Available data
All data available via REST APIs, SPARQL endpoints, query interfaces and data dumps
●
Patent non-assertion
We will not patent anything: OpenCitations’ infrastructure is free to replicate
We at OpenCitations are proud of the results reached in the Insurance area, but realise that we still have some was to go in the other areas. Although the general situation is already satisfying, we are working to strengthen our weak points.