jiscmrd – OpenCitations blog

Open Citations Extension Project

I am pleased to announce that the JISC have funded an extension to the Open Citations Project to run from 1st August 2012 until 31st January 2013, during which we will review and revise the technology used to create the Open Citations Corpus, will update the content provided by PubMed Central, and will improve its presentation. We will also prototype value-added services over the open citations data, to demonstrate their usefulness and to justify further funding that will permit a major expansion of the corpus in 2013 to include reference lists from subscription-access journals, including those from Nature Publishing Group, Science and other AAAS publications, and Oxford University Press. In preparation for this, we will collaborate with CrossRef to determine how best to ingest reference data into the Open Citations Corpus in an on-going manner.

Full details are given in the Case for Support. Watch this space!

Comments on IS DATA PUBLICATION THE RIGHT METAPHOR?

Is Data Publication the Right Metaphor? is an essay by Mark Parsons and Peter Fox to be published in the Data Science Journal, for which a preprint has been provided for open pre-publication community peer review at http://mp-datamatters.blogspot.com/2011/12/seeking-open-review-of-provocative-data.html.

To supplement the discussion this excellent essay has already prompted, I would like to add some belated comments relating to DOIs for dataset versions, peer review of datasets, metadata as first class scientific objects, and linking the data publication and linked data metaphors.

First, DOIs can easily be used to provide unique identifiers to datasets that undergo updating and versioning, as exemplified in the Dryad Data Repository, a repository for the publication of small heterogeneous biological datasets linked to peer-reviewed journal articles. Dryad DOIs permit the version numbers of each data package and of each of the data package’s constituent data files to be specified explicitly (see http://wiki.datadryad.org/wiki/DOI_Usage).

Second, relating to the problems of applying traditional models of peer review to datasets, it is important to realize a fundamental difference between a journal article and a dataset, namely that the journal article is an exercise in rhetorial persuasion, with the authors presenting selected data to convince the reader of the correctness of particular hypotheses – see, for example, papers on this topic by Anita de Waard et al. [1-3] – while datasets lack this rhetorial structure, and thus cannot be judged by the same standards of logical persuasiveness, fit between data and hypothesis, etc. Rather, datasets contain ‘mere facts’, albeit organized according to some underlying data model, and thus their review and quality control have to be “more like an audit that assures that a data set adheres to best practices of documentation, format, error characterization, etc.”. For this reason, citation links between datasets and the journal articles upon which they are based, as routinely given in the Dryad Data Repository, are of particular value, since the latter provide the peer-reviewed contextual framework within which to undestand the data.

Third, as mentioned by John Milner in the discussion of the essay at https://www.jiscmail.ac.uk/cgi-bin/webadmin?A1=ind1112&L=RESEARCH-DATAMAN, there is increasing recognition that the metadata describing a dataset should themselves also be recognised as independent first class scientific objects, with the possibility of publishing such a metadata file with its own DOI, either as a supplementary file within a data package, or as a ‘metadata paper’ within a ‘metadata journal’.

Finally, one way of linking the ‘data publication’ metaphor with the ‘linked data’ metaphor is to ensure that such metadata describing datasets are made available not only in human-readable form but also in machine-readable form, by publishing them as RDF, either as a separate named graph with a unique URI or embedded as RDFa within the human-readable metadata document.

Hope these comments help.

[1] de Waard, A., Breure, L., Kircz, J.G., Oostendorp, H. van (2006). Modeling Rhetoric in Scientific Publications. Current Res. in Inf. Sci. and Techn. pp. 352-356.

[2] de Waard, A., (2007). A Pragmatic Structure for the Research Article, in: Proceedings ICPW’07: 2nd International Conference on the Pragmatic Web, 22-23 Oct. 2007, Tilburg: NL. (Eds.) Buckingham Shum, S., Lind, M. and Weigand, H. Published in: ACM Digital Library & Open University ePrint 9275.

[3] de Waard, A. and Kircz, J.G. (2008). Modeling scientific discourse – shifting perspectives and persistent issues, ELPUB2008. Open Scholarship: Authority, Community, and Sustainability in the Age of Web 2.0 – Proc. of the 12th Int. Conference on Electronic Publishing, June 2008, Eds. L. Chan and S. Mornati, pp. 234-245.

The plate tectonics of research data publication

In biology, the fields of macromolecular structural biology and sequence bioinformatics have, since the 1970s, had established international databases for the deposition of data, and journal policies mandating such deposition prior to acceptance for publication of manuscripts describing the data. Similar good practices have developed more recently in other disciplines, notably astronomy. But these are the exceptions, and over the majority of scientific fields data publication remains a minority activity. For the most part, this is because the technical barriers to publication of research datasets remain so high, and the academic rewards so low, that such publication is undertaken only by the few who regard it as a moral imperative. However, new policies are combining with new technological capabilities to bring significant change to this publication landscape.

An analogy can perhaps be made with geological plate tectonics. The new policies of funders and journal publishers towards the open publication of research datasets arising from publicly funded research can be likened to a tectonic plate slowly moving forward with inexorable force, that is colliding with the massive stationary continental plate of established scientific practice, in which data are traditionally regarded as belonging to the research group that generated them, in which data sharing occurs only between trusted colleagues on the basis of personal request, and in which the only publications that are truly valued are those of journal articles.

This traditional position is reinforced by the metrics employed in the assessment of research quality, which, while giving lip service to the value of data publication, regard articles in high impact factor journals as of paramount importance. And there is good reason why the scholarly research article is so highly regarded, since it is a rhetorical construct in which the authors attempt by the selective presentation of evidence to convince the readers that particular hypotheses are proven. As such, it can both demonstrate the competence and achievements of the authors, and be evaluated against other such publications by peer review. In contrast, the publication of a research dataset is primarily a presentation of facts, with no rhetorical content, that can only be validated on the basis of internal self-consistency, information about instrument calibrations, resolution and error estimates, and the possession of adequate descriptive metadata in appropriate formats – much more pedestrian stuff.

However, to return to our analogy, these tectonic plates are colliding, with the traditional plate of data retention facing ultimate subduction beneath the advancing plate of open data publication. At present there is friction between them, and along much of the sheer zone there seems to be little movement, leading to a build-up of pressure. Small projects that result in some local movement towards better data management can be likened to minor earthquakes having only local impact. But these are the precursors to an inevitable major re-positioning of the plates to relieve the mounting pressure for change. This will result in a major general realignment of attitudes along the whole plate boundary, resulting in a tsunami of open data publication. We are thus on the cusp between the traditional status quo and a new dramatically reshaped scientific publication landscape, in which open data publication will take its proper place as underpinning the publication of ideas and evidences supporting hypotheses.

Technological projects such as the JISC ADMIRAL Project and its successor the JISC UMF DataFlow Project serve to facilitate this transition by ‘lubricating’ the plate boundary and enabling movement. In particular, the two-tiered federated data management infrastructure they provide, with local services to meet the private data management needs of individual research groups (DataStage filestore instances) being linked to institutional repositories (DataBank repository instances) by automated procedures for the easy archiving and publication of selected datasets, makes the whole process easier, as illustrated by the following figures taken from the original ADMIRAL Project grant application.

Figure 1: The conventional research data lifecycle

Four phases mark the activities undertaken in traversing the conventional data lifecycle: formulation, experimentation, interpretation and publication. The publication outputs from one cycle provide the input to the next. However, only selected research data are conventionally published. The original research datasets are frequently abandoned on local hard drives or CD-ROMs, and neither datasets nor papers are submitted to institutional repositories.

Figure 2: The ADMIRAL enhanced research data lifecycle

Raw research data are first organized and annotated in a local research data filestore. From there they can be shared and used to support publications, and can be automatically archived to institutional repositories, from which they can optionally be published as Linked Open Data on the Semantic Web for public dissemination and reuse. This figure differs from the DCC Data Cycle model by emphasizing the importance of the local research data filestore.

Figure 3: The effort involved in submitting data to an institutional repository

As investment is made in the local organization and annotation of research datasets, the effort involved in data submission to an institutional repository reduces to the point where it becomes feasible on a routine basis.

IBRG projects to facilitate data publication and data citation

In the previous post, I outlined reasons why researchers don’t publish data, presented as evidence to the Royal Society’s Policy Study “Science as a Public Enterprise” Call for Evidence. Here, I summarize activities by members of my Image Bioinformatics Research Group (IBRG) at Oxford University to facilitate data publication and data citation, and thus to help catalyze a cultural shift to a situation in which data publication is as natural a part of research life as is undertaking experiments.

= = =

Data management services and data repositories

We are developing tools and services to assist researchers in their local data management, for their own personal benefit, while facilitating automated data submission to appropriate institutional or subject-specific data repositories, in ways that fit with their normal working practices and impose as little as possible in terms of cognitive overhead – what we term sheer curation. These include the two-stage data management services we are currently funded to develop by the University Modernization Fund through the JISC DataFlow Project, namely (a) DataStage, a private local data management file system, with automated backup, Web access, and security access control, for use by individual research groups, and (b) DataBank, a cloud-deployable data repository for use by universities, research institutes or large research consortia. These open source services will be made available for installation by third parties on the Eduserv academic cloud and elsewhere, as required by research groups, institutions and universities both in the UK and internationally. We seek early adopters!

Curation by addition

For automated data submissions from DataStage to DataBank, that will use the SWORDv2 repository submission protocol to standardize data package ingest, we are intentionally lowering the barriers in terms of metadata requirements for initial data submission, with the the possibility of enriching the metadata at a later date – what we call curation by addition – in order to kick-start the cultural sea change required for data deposition to become routine. We are trying to avoid the best – the requirement for perfect and complete metadata – becoming the enemy of the good – data publication by any means.

Dryad

We are, through the JISC Dryad-UK Project, working to promote the Dryad Data Repository, a domain-specific repository for biological datasets linked to peer-reviewed journal articles, by bringing additional publishers and journals on board, and enabling Dryad metadata to be published as open linked data.

SWORD

We are also promoting the adoption of SWORDv2 repository communication protocol for data package wrapping, to permit automated deposit to DataBank, Dryad or other SWORD-compliant repositories, and the exchange of metadata between them.

SPAR (Semantic Publishing and Referencing) Ontologies

To enable Dryad, DataBank and similar repository metadata to be published as open linked data, we are creating appropriate data description and data citation ontologies, including FaBiO and CiTO4Data, as part of our suite of SPAR Ontologies, and are using them to provide mappings from the DataCite XML Metadata Kernel to RDF.

Data citation

We are working with DataCite to assign DOIs to Dryad and DataBank datasets, so that data publications become citable, gaining academic credit for the data depositor.

These data citations, when they exist, will fit naturally within the Open Citations Corpus, a collection of some 3.4 million bibliographic citations from within PubMed Central that we have recently established as open linked data, as part of the JISC Open Citations Project.

We have also worked to establish best practice for citing data publications from within the literature, and with one open access journal publisher to influence their Data Publishing Policies and Guidelines to Authors regarding data citation, as detailed in earlier posts on this blog.

Tools for metadata curation

The above tools and services are generic. Specifically in the biomedical area, we are developing MIIDI, a Minimal Information standard for reporting an Infectious Disease Investigation, to specify the metadata that should for completeness accompany such an investigation, and have recently developed MIIDI Forms, a web tool that facilitates the entry of such metadata, that involves interaction with appropriate web services to enable autocompleting of bibliographic information and specification of geo-coordinates for place names, and permits automated look-up of ontology terms from the NCBI BioPortal.

Open Research Reports

We are working to create Open Research Reports, open access structured digital abstracts in both human- and machine-readable form that describe datasets or journal articles that relate to infectious disease, based on MIIDI and to be published in an instant data journal format with DOIs to permit referencing and citation.

Tools for creating data management plans

We have recently started working with the Digital Curation Centre to help improve their DMPonline data management planning tool for creating the data management plans increasingly required to accompany grant applications, and useful for managing the flow of data from funded projects. If our current funding application is successful, this work will be carried forward in the OXFORD DMPonline Project, in which, in addition to adoption, adaption, customization and integration of the tool for use by University of Oxford researchers, we will develop the following generic improvements to the tool that will be fed back to the DCC as open source enhancements for general use across UK academia and internationally:

a) creation of DaMO, a simple data management ontology,

b) use of DaMO to create RDF metadata for data management plans,

c) SWORDv2-wrapping of data management plans for repository submission, and

d) creation of DMPBank, a DataBank instance specifically tailored for archiving and publishing data management plans.

Why researchers don’t publish data

Evidence submitted by David Shotton in response to the Royal Society’s Policy Study “Science as a Public Enterprise” Call for Evidence, addressing the following two topics raised by that call:

Getting Researcher buy-in. How do we get researchers to be more willing to share data? What is there to be learned from disciplines such as genomics which have norms which favour wide sharing of data?

Ensuring we generate useful metadata. For open data to be useful, it needs to be sufficiently well described. The researchers creating the dataset are in the best position to create the metadata; but as things stand, the incentives for them to do a thorough job of this are not always very strong. Do we need to change incentives?

= = =

“I guess I have been invited to contribute evidence to the evidence session on digital curation at the Royal Society on 5th August 2011 to present the view from the shop floor – or rather from the laboratory bench. I would like to mention three pressures that presently combine to prevent researchers from publishing their data.

Pressure one: Information volume

When I started research, you could, if you were very fortunate (as I was), solve a protein structure to low resolution within six months and to medium resolution within three or four years, and you could hope to know something about all the protein structures that had so far been determined. Today, you can collect the crystallographic structure factor data for a new protein in a few minutes at the Diamond Light Source, and can compute its 3D structure on your laptop during the train ride home. PDB currently contains the structures of about 74,000 macromolecules, and you are unlikely to know the structures of more than a handful of these.

Looking at the same problem from a different perspective, PubMed currently received a million articles per year. If you imagine there might be a thousand biomedical specialisms – if you slice the salami thinly enough -, as a specialist you can expect to have on average twenty new papers in your field each week – an impossible number to carve out time for, from your other activities, if you wish to read them properly.

Thus you will never catch up – there is just too much scientific information around now. You would like to know about it all, to keep abreast of your field, but the task is impossible. Researchers are thus under overwhelming pressure, and have to run just to stand still. They have no spare time to undertake data curation activities for which they receive little or no academic reward in terms of peer esteem, tenure or promotion.

Pressure two: Institutional pressures

The principal pressures researchers are under from their departments and institutions are (a) to win grants and (b) to publish in high impact journals, because these things influence departmental income both (a) directly through full economic costs from funding agencies, and (b) through high RAE/REF scores that in England determine funding from HEFCE. From the viewpoint of a Head of Department trying to establish or maintain his department’s reputation and financial health, nothing else matters. I have known these factors as the deciding ones in academic appointments. Nobler concepts of scientific excellence and of scientific altruism in the form of data publication become submerged beneath these pressures.

Pressure three: Cognitive overheads of data management

Appropriate ontologies and technical infrastructures for data preservation increasingly exist, but the concepts surrounding metadata creation, repository deposit and data accessibility are foreign to most biomedical researchers, leading to cognitive and skill barriers that prevent them from undertaking routine best-practice data management.

Put crudely, the large amount of effort involved in preparing data for publication, coupled with the negligible incentives and rewards, prevents researchers in most biomedical specialisms from doing so.

Having said that, research scientists are perfectly able to provide structured metadata when it is necessary to do so. With the switch to on-line journal article submission, publishers have devised lengthy web forms that require completion with details of co-authors and their affiliations, funding agencies, etc. before you are permitted to upload your manuscript – forms that for certain publishers can take the best part of an afternoon to complete for a new submission involving many authors, figures and supplementary files. Since researchers have no choice but to comply with the metadata requests, they do so, since this is the only way in which to achieve their desired goal of publication in the chosen journal.

That the fields of genomics and macromolecular structures are exceptions to the rule that data are not widely published is due to two factors:

First, their datasets are relatively simple, homogeneous and well-defined – linear nucleotide or amino acid sequences, lists of structure factors, and lists of atomic coordinates – in comparison with the heterogeneity of data in fields such as ecology or animal behaviour, simplifying the tasks of data management and metadata creation.
Second, and more important, is the fact that in the early 1970s journals such as Nature started to mandate database accession numbers as a precondition of publishing sequence or structure papers – this brought about an almost instantaneous change in attitudes among our research community!

For other disciplines, while I commend journals’ and research councils’ recent policies regarding data publication, I believe we will only achieve radical change when funders and publishers mandate data publication as a pre-condition of applying for a further grant or of article submission. Toothless research council data policies, however laudable, are of little use unless backed up by some policing. ‘Sticks’ are required to achieve desired policy aims, as well as the ‘carrots’ of better personal data management and data security obtained by employing easy-to-use tools and systems.”

= = =

The following post describes what we are doing, with funding help from the JISC, to help mitigate these pressures and provide tools and services to assist researchers in data publication.

Current Projects at the Image Bioinformatics Research Group in Oxford

Alistair Miles, of SKOS fame, who formerly worked in our research group, spent yesterday afternoon catching up with us, and has written a nice blog post on the MalariaGEN Informatics Blog describing our current activities, including our work on the Open Citations Corpus, and how they might intersect with the data management activities of the MalariaGEN, the Malaria Genome Epidemiology Network for which he now works.

He has also written a separate blog post reflecting on the work he did when in our group working on the JISC FlyWeb Project to develop openflydata.org, as an update on his earlier post, which is well worth a read if you wish to understand how semantic web techniques can be used to integrate data from heterogeneous and non-compatible databases in distributed locations.

Graham Klyne, who was part of that FlyWeb Project, has been instrumental in taking those same ‘data web’ data integration techniques and re-applying them to data integration in the classical arts, using the CIDOC-CRM data model developed for the museum community, to underpin CLAROS, The World of Art on the Semantic Web. It now looks as though Graham’s current work with Jun Zhao and others on the provenance of bioinformatics workflows, within the EU Workflows4Ever Project, might be of assistance to Al in his current work with Plasmodium SNP discovery and genotyping pipelines for MalariaGEN.

It’s nice to stand back from time to time and see all these activities interlinking as a whole. Thanks, Al.

My next two posts follow on from this, in that the first explores reasons why the majority of researchers don’t presently publish the datasets underlying their research articles , and the second summarizes the various JISC projects in which we are involved as a research group, including the Open Citations Project, that seek to mitigate this problem and provide tools and services that make data publication and subsequent data citation easier.