Open Citation Corpus – OpenCitations blog

A new revolutionary workflow for a unified collection of citations: say hello to the OpenCitations Index

Blog post by Ivan Heibi (University of Bologna), Arianna Moretti (University of Bologna) and Chiara Di Giambattista (University of Bologna).

In the past five years, the OpenCitations data has been enriched with numerous new indexes of open citation data from different sources. However, the quantity and diversification of the ingested information have raised several issues, which recently made it essential to conduct a complete revision of the ingestion workflow. The result was a revolution in the way OpenCitations data is delivered. In this blog post, we will explain the context and challenges raised by the old procedure. Then, we will present the new ingestion workflow, designed to produce just two comprehensive collections: OpenCitations Index, collecting open citation data, and OpenCitations Meta, for the open bibliographical metadata.

Once upon a time, there were five OpenCitations indexes…

In 2018, OpenCitations released the kickoff version of its first citation index, COCI (citations from Crossref), which contained around 300 million citation links derived from the subset of the reference lists in the Crossref database, where citing and cited entities were identified using Digital Object Identifiers (DOIs). COCI gathered citations with associated metadata in compliance with the recommendations from the Initiative for Open Citations (I4OC) that citation data should be structured, separable, and open, thus marking a turning point by providing a disruptive and free and open alternative to earlier sources such as Google Scholar, which provided freely accessible data although not downloadable, and Web of Science or Scopus, which demanded paid access.

In a short time, COCI became a competitive and trusted index of citation data, used by numerous institutional repositories, including B!son and Optimeta. In 2021, COCI was taken into account in a comparative study with the most relevant sources in the landscape, including the proprietary ones, which showed its coverage approaching parity with those of the other sources involved in the analysis (Microsoft Academic, Scopus, Dimensions, and Web of Science). At the time of its most recent update in January 2023, COCI counted more than 1.4 billion citations. The reason behind this outstanding number lies in several factors, including Elsevier’s endorsement of the Declaration on Research Assessment (DORA) in December 2020, leading to the open release via Crossref of the reference lists of the articles published in all its journals, and confirming the value of initiatives such as the Initiative for Open Citations (I4OC).

However, before this change of heart, in 2019 OpenCitations had tried to narrow the open citations coverage gap by launching its second index, the Crowdsourced Open Citations Index (CROCI). This index allowed publishers and scholars to contribute directly by uploading crowdsourced open citations into the OpenCitations infrastructure.

In December 2022, a new concrete step towards a factual plurality of OpenCitations indexes was taken by the ingestion of new data sources into the infrastructure, with the publication of the inaugural dumps of DOCI (citations from DataCite) and POCI (citations from PubMed). In June 2023, the first version of the OROCI (citations from OpenAIRE) dump was released too, and JOCI (citations from JALC) is expected to be available by the end of November 2023, for a total of five collections from different sources.

Why a new workflow? The issues with multiple sources management and new challenges

While having such a variety and richness of indexes helped present the extent of OpenCitations sources, the recent increment in the number of sources and the diversification of data integrated led to two primary issues:

1. the necessity to handle the ingestion of new identifier types in a DOI-based software infrastructure, and
2. the consequent possibility of encountering the same citation expressed by several sources with different identifiers.

Moreover, it soon became evident the need to optimize the reuse of the already developed software components to facilitate the metadata crosswalk processes between the new sources’ data models and the OpenCitations Data Model, with the aim to define a functional and easily extendable workflow to be easily reused when it comes to incorporating new data sources, which should be:

1. sufficiently generic to establish a globally unique procedure;
2. customizable enough to capture the necessary information within each of the specific data models and formats.

As a solution, we decided to use OpenCitations Meta, the new OpenCitations database and tool for managing bibliographic data related to the publications involved in the citations. OpenCitations Meta makes it possible to assign each entity involved in a citation an internal identifier, nominally the OpenCitations Meta Identifier (OMID), to which all the associated persistent identifiers of the same publication are redirected.

As a result, the allocation of an OMID for each bibliographic resource also enabled the unambiguous identification of each citation, regardless of the persistent identifier schema originally used by the data source to identify the resources. This approach allowed us to perform data deduplication and finally make all the sources’ contributions converge into a unified index containing all the unique citations managed by OpenCitations, expressed as OMID to OMID citation links.

The revised workflow

The new workflow is based on three main components with the benefit of optimizing the process both in terms of computational cost and in terms of flexibility. As shown in Fig. 1, in a preliminary step, source-specific software converts the input dataset – structured according to the source data model – to extract two OpenCitations Data Model compliant data collections in tabular format for bibliographic metadata and citation data, respectively.

The following steps are common to the process of each dataset.

STEP 1: The bibliographic metadata collection is used as input for the META software. At this stage, it is checked whether or not the bibliographic entities have been previously integrated into our infrastructure (coming from other data sources). If so, the existing OMID is linked also to the new alternative identifiers of the new bibliographic resources. New metadata values, if any, are also integrated. A new OMID identifier is produced for entities never previously encountered, uniquely representing the bibliographic resource in OpenCitations. The outputs of the process are: (I) an updated version of the OpenCitations Meta collection that also includes the metadata of the bibliographic entities provided by the new source, and (II) a collection of provenance data. An internal database is constantly refreshed to preserve correspondence between IDs and the associated internal OMIDs.

STEP 2: Starting from the collection of citations expressed as directional links between identifiers of potentially any type (e.g., DOI-DOI, PMID-PMID, PMC-PMID, etc.), the INDEX software queries the internal database mapping IDs to OMIDs to produce an updated version of the OpenCitations Index: unique citations expressed as OMID-OMID links in different formats, accompanied by their corresponding provenance data.

Fig. 1: An overview of the data ingestion workflow, starting from the data source-specific conversion and production of citations and bibliographic metadata tables, progressing through the META process and the assignation of an OMID identifier to each bibliographic record involved in a citation, and culminating with the exposition of the OpenCitations Index collection of OMID-OMID unique citations.

What we have now: The OpenCitations Index

From now on, OpenCitations will no longer display an index of citation data for each source. Instead, we will publish a single collection of citations into which the contributions from each of the sources will flow, which we will simply call ‘The OpenCitations Index‘. The first version of this unified index of OMID-OMID citations is posted on Figshare. It was produced in RDF, CSV, and SCHOLIX formats, together with a collection of its provenance information, provided in RDF and CSV formats. For each citation, it is possible to trace the source of the information by consulting the Provenance data collection, thanks to the http://www.w3.org/ns/prov#atLocation property, which defines the location of each citation.

This new solution has the benefit of simplifying the consultation of the data maintained by our infrastructure without reducing the information content. In addition, by including efficient handling of the deduplication problem, the new Index not only provides accurate data on the exact number of unique citations exposed by the framework but also verifies the individual contribution of each source, as well as their overlapping data (Fig. 2).

Fig. 2: An overview of the number of citations stored in the OpenCitations Index as of October 31, 2023. The diagonal cells in the table (highlighted in yellow) show the unique contribution of each collection to the OpenCitations Index, while the other cells represent the citations that are shared between the collections. More in detail, the green cells show the overall input of each source, while the pink cells represent the number of overlapping citations between two data sources.

Currently, the Index contains almost 2 billion unique citations. By the end of November, a new version of the collection will be published, including the contribution of the new Japan Link Centre (JaLC) source.

How to access the OpenCitations Index data

To maximize the reuse of the exposed information and to ensure the greatest possible interoperability, the collection will always be published on Figshare in all formats listed above. In addition, the data will be accessible via an API, a SPARQL endpoint, and a web interface.

The redesign of the ingestion workflow marks a fundamental step for OpenCitations towards a more intuitive and simple access to our services while always preserving and improving the quality of our data. If you need further information on how the new workflow works, please visit our website, contact us at contact@opencitations.net or leave feedback and/or suggestions in the dedicated card on our public roadmap to help us improve our services and communications. Thank you!

OpenCitations described

OpenCitations is an infrastructure organization for open scholarship dedicated to the publication of open bibliographic and citation data. We at OpenCitations are proud to announce the publication, in the first issue of Quantitative Science Studies, of a canonical paper in which we introduce and describe OpenCitations and outline its achievements and goals [1].

Here, I outline the contents of our paper, and provide definitive links on the topics described. Many of these topics have been the subjects of earlier blog posts.

This paper appears in the first Special Issue of QSS, dedicated to the description of the bibliometric data sources that lie at the heart of scientometric research, which aims to characterize the most important data sources currently available and to show how they differ in various dimensions, for instance in the data they provide, their level of openness, and their support for making research reproducible. The first three papers in this special issue cover the most important commercial bibliographic data sources: Web of Science (Clarivate Analytics), Scopus (Elsevier), and Dimensions (Digital Science), while the remaining three articles describe open data sources: Microsoft Academic, Crossref and OpenCitations.

In the introduction to our own paper, we describe the origins of OpenCitations, discuss the growth and benefits of open science, and introduce the Semantic Web techniques used at OpenCitations for recording and publishing our data. We then go on to describe OpenCitations’ services and data, namely Open Citation Identifiers, the OpenCitations Data Model, the SPAR (Semantic Publishing and Referencing) Ontologies, the OpenCitations Corpus, and the OpenCitations Indexes of citation data, of which the first and largest is COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations, that currently holds information on over 624 million citations. We conclude our survey of OpenCitations’ services and data by outlining the generic open source software developed at OpenCitations, including OSCAR, the OpenCitations RDF Search Application for searching over RDF datasets, LUCINDA, OSCAR’s associated OpenCitations RDF Resource Browser, and RAMOSE, OpenCitations’ application for creating REST APIs over SPARQL endpoints, thus opening Semantic Web datasets to those not familiar with SPARQL, the RDF query language.

In the second half of the paper, we describe OpenCitations as an organization in terms of its compliance with the principles for the sustainability of open infrastructures proposed by Bilder, Lin and Neylon (2015) [2], and report the selection of OpenCitations by the Global Sustainability Coalition for Open Science Services (SCOSS) as an open infrastructure organization worthy of crowd-funding support by the stakeholder community. We then provide usage statistics for our datasets and web site, and describe the adoption of OpenCitations data and services by the community, before concluding with a forward look at our proposed developments of OpenCitations activities.

References

[1] Silvio Peroni and David Shotton (2020). OpenCitations, an infrastructure organization for open scholarship. Quantitative Science Studies 1 (1): 428-444. https://doi.org/10.1162/qss_a_00023

[2] Geoffrey Bilder, Jennifer Lin and Cameron Neylon (2015). Principles for open scholarly infrastructures. Figshare. https://doi.org/10.6084/m9.figshare.1314859

Open Citations is dead. Long live OpenCitations.

In October 2015, I asked Silvio Peroni, my long-term colleague in the development of the SPAR Ontologies, to become Co-Director of the Open Citations Project, and to work with me in taking forward the prototype Open Citations Corpus (OCC), originally developed at the University of Oxford with the support of Jisc, with the aim of developing it into a production service of real use to scholars.

The result is OpenCitations, a new instantiation of the OCC hosted by the Department of Computer Science and Engineering of the University of Bologna, based on a new metadata schema and employing several new technologies to automate the ingestion of fresh citation metadata from authoritative sources.

Since the beginning of July 2016, OpenCitations has been ingesting and processing accurate bibliographic references harvested from the reference lists of scholarly papers available in Europe PubMed Central, enriched by metadata from Crossref. These scholarly citation data are described using the SPAR Ontologies according to the new OpenCitations metadata document [1], and are published under a Creative Commons public domain dedication (CC0), so that others may freely build upon, enhance and reuse them for any purpose, without restriction under copyright or database law. We have described the new OpenCitations Corpus, and the new software developed by Silvio to create it, in [2].

OpenCitations is being continuously populated from the scholarly literature, and, as of 30th March 2017, has ingested the references from 123,989 citing bibliographic resources, and contains information about 5,307,857 citation links to 3,469,648 cited resources.

The whole OCC is now available for querying (via SPARQL), and for browsing by means of a very simple Web interface that shows only the data about bibliographic entities (e.g. https://w3id.org/oc/corpus/br/1). Additional more user-friendly interfaces will be available in the coming months. The entire contents of the OpenCitations Corpus (OCC) are also archived every month as data dumps that are made available online through Figshare. Each dump comprises several zip archives, each containing either data or provenance information of a particular sub-dataset of the OCC.

Despite the fact that OpenCitations presently contains only a small proportion of global citation data, it is important to realize that, because of the very nature of scholarly citation, even this partial coverage includes citations of the most important papers in every biomedical field, these critical papers being characterized by the high number of their inward citation links.

[1] Silvio Peroni, David Shotton (2016). Metadata for the OpenCitations Corpus. figshare. https://dx.doi.org/10.6084/m9.figshare.3443876

[2] Silvio Peroni, David Shotton, Fabio Vitali (2016). Freedom for bibliographic references: OpenCitations arise. Proceedings of 2016 International Workshop on Linked Data for Information Extraction (LD4IE 2016): 32-43.
https://w3id.org/oc/paper/occ-lisc2016.html

Open Citations Extension Project

I am pleased to announce that the JISC have funded an extension to the Open Citations Project to run from 1st August 2012 until 31st January 2013, during which we will review and revise the technology used to create the Open Citations Corpus, will update the content provided by PubMed Central, and will improve its presentation. We will also prototype value-added services over the open citations data, to demonstrate their usefulness and to justify further funding that will permit a major expansion of the corpus in 2013 to include reference lists from subscription-access journals, including those from Nature Publishing Group, Science and other AAAS publications, and Oxford University Press. In preparation for this, we will collaborate with CrossRef to determine how best to ingest reference data into the Open Citations Corpus in an on-going manner.

Full details are given in the Case for Support. Watch this space!

Oxford University Press to support Open Citations

I am delighted to announce that Cathy Kennedy, OUP’s Senior Publisher for Journals, has just written to me as follows:

“Oxford University Press is delighted to support the Open Citation Corpus initiative in the interest of furthering and disseminating scholarship. We are making the reference lists of articles in a number of journals we own available for inclusion in the Open Citations Corpus, and will be consulting further with our partner societies on extending the initiative to include their journals in due course”.

OUP thus becomes the third publisher of subscription-access journals, and the first university press, to declare its willingness to make the reference lists in journal articles publicly available. Congratulations, OUP!

Are we starting to see the beginning of a sea change in attitudes? Watch this space.

Open Citations and Semantic Publishing

Given the renewed interest among publishers in the Open Citations Corpus, following the decisions by Nature Publishing Group, publisher of Nature, and by the American Association for the Advancement of Science, publisher of Science, to open their citation data for inclusion in the corpus, I thought it would be helpful to provide links to videos of two conference presentations I gave that describe the Open Citations Corpus in the context of our other work in the area of semantic publishing.

The first of these was an invited contribution with the title Enriching Scientific Citations to Facilitate Knowledge Discovery, given to publishers at the Scientific, Technical and Medical Publishers Innovations Seminar 2010 entitled “Flows in Flux: how publishing technologies change the researcher’s life”, held in London, UK, on 3rd December 2010:

lecture: http://river-valley.tv/media/conferences/stm-innovation-2010/0202-David-Shotton/

slides: http://imageweb.zoo.ox.ac.uk/pub/2010/Presentations/SHOTTON_Citations_STM-Innovations-Seminar-03Dec2010.pdf

abstract: http://imageweb.zoo.ox.ac.uk/pub/2010/Presentations/STM_Innovations_Seminar_2010_ABSTRACTS.pdf

The second presentation, given almost a year later, was an invited contribution with the title The SPAR (Semantic Publishing and Referencing) Ontologies and the Open Citation Corpus, given to librarians at SWIB11 (Semantic Web in Bibliotheken; Semantic Web in Libraries Conference 2011) “Scholarly Communication in the Web of Data”, held in Hambrug, Germany on 30th November 2011:

lecture: http://www.scivee.tv/node/39208

slides: http://imageweb.zoo.ox.ac.uk/pub/2011/presentations/Shotton_SWIB11_SPARandOpenCitationCorpus_30Nov2011.pptx.pdf

abstract: http://imageweb.zoo.ox.ac.uk/pub/2011/presentations/Shotton_Abstract_for_Presentation_at_SWIB11.pdf

Further details of the Open Citations Corpus and the SPAR ontologies, and their applications are, of course, given in this Open Citations and Semantic Publishing blog, for which the following two posts provide the best introduction for those coming here for the first time:

JISC Open Citations Project – Final Project Blog Post

Introducing the Semantic Publishing and Referencing (SPAR) Ontologies

Science joins Nature in opening reference citations

Hot on the heels of my announcement on Monday that Nature Publishing Group has agreed to open its articles’ reference lists, I am delighted to announce that Science Magazine, the American Association for the Advancement of Science’s premier global science weekly, together with the other AAAS journals Science Signalling and Science Translational Medicine, will also open their articles’ reference lists, and contribute the bibliographic citations contained within these lists as open linked data to an expanded Open Citations Corpus, where they will be freely available for everyone to use in whatever manner they choose.

Preparations to expand the corpus in this manner, by integration with the reference processing pipeline of the CrossRef Cited-By Linking service, will be undertaken over the last six months of this year, and incorporation of the references from the AAAS journals into the expanded Open Citations Corpus is planned to commence in the first half of 2013.

Now that the world’s two most presigeous scientific journals have declared their hands, it is time for other subscription-access publishers to step up and join them, for the reasons given in my earlier post. I am presently in discussion with a small number of other major STM publishers, and hope to be able to make further announcements in the near future. However, there are many other publishers with whom I have not yet been able to make personal contact. Those who would also like to become ‘early adopters’ of the Open Citations publishing model should contact me at <david.shotton@zoo.ox.ac.uk>.

Access to Citation Data

The JISC, in response to its invitation to tender, has recently funded Curtis+Cartwright Consulting Ltd, a research and strategy consultancy, to undertake an independent study entitled Access to Citation Data: A Cost-Benefit and Risk Review and Forward Look. Evidence gathering for the study has just started, and the consultants are due to produce a report on this subject by next February. This report will be one of a series of high profile reports that the JISC is commissioning to inform the UK FE/HE sector on key issues relating to digital infrastructure.

The JISC has invited me to serve on the steering group for this study, which will in particular be reviewing the Open Citations Corpus, both to provide intelligence and to act as a case study and as a potential recommended corpus of citation data.

I look forward to working with the review team, starting later this month, and anticipate that we will benefit from this investigation as well as contribute to it, since we anticipate that Curtis+Cartwright will be able to advise on sustainable business models for such open access citation data corpora.

Why publishers should open their references

Why should the publishers of subscription-access journals, who presently generate income from the sale of access to peer-reviewed full text scholarly articles, be willingly open the reference lists of these articles, and contribute these to the Open Citations Corpus for publication as open linked data? I would like to suggest the following reasons:

1. There is a general move towards open data, which is widely regarded as a common good. This includes citation data, i.e. bibliographic references from one article to another (in RDF Turtle format: A cito:cites B . ).
2. The reference lists at the end of journal articles are works of scholarship by the authors, who have chosen to include certain references and exclude other potentially citable papers from the reference list. However, the references themselves are simply items of bibliographic data, formatted according to the journal style, and do not benefit from the author’s creative input.
3. The reference list, together with the front matter (including the bibliographic information about the article itself) and the abstract, has traditionally been included within the copyright protection enjoyed by the article as a whole. However, the bibliographic information about the article and the article’s abstract are commonly made freely available, for example through PubMed. This same openness should now be afforded to the reference list within each article.
4. There is a home for such reference citation data: the Open Citations Corpus has been specifically created to house and publish scholarly bibliographic citation data, and is now preparing to welcome article reference lists from subscription-access journals, to supplement those already contributed from open-access journals.
5. For those publishers who already contribute their reference information to CrossRef as part of its Cited-By Linking service, this can be accomplished without any change to the publisher’s own publishing workflows, just by giving permission for CrossRef to flag the articles of certain journals as having open references. Open Citations intends to collaborate with CrossRef by harvesting the reference lists from such flagged articles, parsing them into RDF, and adding them to the Open Citations Corpus. Provided that the references are already being submitted to CrossRef, no work will have to be done by the publisher, and no changes in publishing procedure will be involved.
6. Open Citations will publish each reference list as an independent RDF Named Graph, with a unique URI, thereby protecting the integrity of the article reference list as a unit of scholarship, the source of which will be explicitly acknowledged.
7. The open citations data will then be offered back to publishers to use as they wish, e.g. for visualization of citation networks, calculation of metrics, etc., providing easier and more usable access to their own citation data than is currently afforded by commercial providers, who do not provide such data in linked data format.
8. Publishers will also be free to host their own open citations data, should they wish to do so.
9. For the majority of publishers, who would still receive subscriptions on the full articles themselves, opening their article reference lists in this way will cost nothing in terms of lost revenue.
10. Indeed, participation in the Open Citation Corpus will bring the following benefits to subscription-access publishers:
– Access to services to be built over the aggregated open citations data, for example an automated reference correction service available to editors upon receipt of a manuscript, for the automated pre-publication correction of errors in reference lists prior to article publication.
– Increased exposure to users of the references to the publisher’s own journal articles – a form of advertising. While at first coverage among subscription-access publishers will be incomplete, this expanding Open Citation Corpus will, in true Web 2.0 style, become more useful the more publishers participate.
– Even while coverage is incomplete, the Open Citations Corpus by its very nature contains reference citations to all the key papers published in every field covered – currently to all the key papers published in every biomedical field, enabling readers more easily to identify and find the most highly cited papers of each contributing publisher.
– Opening citations data will result in white-listing and general good-will from funding agencies, government and other advocates of open data, who might otherwise mandate publication by grantees in alternative open-access journals.
– Opening citations data will lead to support from scholars and researchers themselves, who wil be more inclined to publish in that publisher’s journals, feeling that at last the publishers would be giving back to them some of their own data, rather than selling it back to them as at present.

As my next blog post shows, one leading subscription-access publisher is now willing to open its journal article references in the way I have suggested. Others who would like to so the same should contact me at <david.shotton@zoo.ox.ac.uk>.

Comments on IS DATA PUBLICATION THE RIGHT METAPHOR?

Is Data Publication the Right Metaphor? is an essay by Mark Parsons and Peter Fox to be published in the Data Science Journal, for which a preprint has been provided for open pre-publication community peer review at http://mp-datamatters.blogspot.com/2011/12/seeking-open-review-of-provocative-data.html.

To supplement the discussion this excellent essay has already prompted, I would like to add some belated comments relating to DOIs for dataset versions, peer review of datasets, metadata as first class scientific objects, and linking the data publication and linked data metaphors.

First, DOIs can easily be used to provide unique identifiers to datasets that undergo updating and versioning, as exemplified in the Dryad Data Repository, a repository for the publication of small heterogeneous biological datasets linked to peer-reviewed journal articles. Dryad DOIs permit the version numbers of each data package and of each of the data package’s constituent data files to be specified explicitly (see http://wiki.datadryad.org/wiki/DOI_Usage).

Second, relating to the problems of applying traditional models of peer review to datasets, it is important to realize a fundamental difference between a journal article and a dataset, namely that the journal article is an exercise in rhetorial persuasion, with the authors presenting selected data to convince the reader of the correctness of particular hypotheses – see, for example, papers on this topic by Anita de Waard et al. [1-3] – while datasets lack this rhetorial structure, and thus cannot be judged by the same standards of logical persuasiveness, fit between data and hypothesis, etc. Rather, datasets contain ‘mere facts’, albeit organized according to some underlying data model, and thus their review and quality control have to be “more like an audit that assures that a data set adheres to best practices of documentation, format, error characterization, etc.”. For this reason, citation links between datasets and the journal articles upon which they are based, as routinely given in the Dryad Data Repository, are of particular value, since the latter provide the peer-reviewed contextual framework within which to undestand the data.

Third, as mentioned by John Milner in the discussion of the essay at https://www.jiscmail.ac.uk/cgi-bin/webadmin?A1=ind1112&L=RESEARCH-DATAMAN, there is increasing recognition that the metadata describing a dataset should themselves also be recognised as independent first class scientific objects, with the possibility of publishing such a metadata file with its own DOI, either as a supplementary file within a data package, or as a ‘metadata paper’ within a ‘metadata journal’.

Finally, one way of linking the ‘data publication’ metaphor with the ‘linked data’ metaphor is to ensure that such metadata describing datasets are made available not only in human-readable form but also in machine-readable form, by publishing them as RDF, either as a separate named graph with a unique URI or embedded as RDFa within the human-readable metadata document.

Hope these comments help.

[1] de Waard, A., Breure, L., Kircz, J.G., Oostendorp, H. van (2006). Modeling Rhetoric in Scientific Publications. Current Res. in Inf. Sci. and Techn. pp. 352-356.

[2] de Waard, A., (2007). A Pragmatic Structure for the Research Article, in: Proceedings ICPW’07: 2nd International Conference on the Pragmatic Web, 22-23 Oct. 2007, Tilburg: NL. (Eds.) Buckingham Shum, S., Lind, M. and Weigand, H. Published in: ACM Digital Library & Open University ePrint 9275.

[3] de Waard, A. and Kircz, J.G. (2008). Modeling scientific discourse – shifting perspectives and persistent issues, ELPUB2008. Open Scholarship: Authority, Community, and Sustainability in the Age of Web 2.0 – Proc. of the 12th Int. Conference on Electronic Publishing, June 2008, Eds. L. Chan and S. Mornati, pp. 234-245.