Digital resources in the Social Sciences and Humanities OpenEdition Our platforms OpenEdition Books OpenEdition Journals Hypotheses Calenda Libraries OpenEdition Freemium Follow us

Barriers to comprehensive reference availability

Two significant barriers prevent comprehensive reference availability through Crossref.

The first barrier

First, two-thirds of Crossref’s publisher-members, in particular the smaller ones, do not submit references along with the other details of their publications. Many of these published works are of types (e.g. abstracts, editorials and news items) that lack any references.  However, while the number of non-submitted references associated with other publications from these publishers is not known, it is likely to be substantial.

Ironically, quite a number of publishers have their Crossref reference status option set to ‘Open’, and yet fail to submit any references!

All publishers who use Crossref DOIs and submit metadata describing their works to Crossref should be strongly encouraged to start submitting associated reference lists if these exist.  Crossref have confirmed that it is easy to do, with or without membership of Crossref’s free and beneficial Cited-by Service that provides publishers with statistics on the citations of their own publications.  Help can be provided by Crossref Support (support@crossref.org).

The second barrier

The second barrier to full reference availability is created by publishers that submit references to Crossref, but do not presently make them open. Elsevier is by far the largest member of this group, which also includes the American Chemical Society, IEEE and Wolters Kluwer Health.

It is both quick and easy for a publisher to change its preference setting and request that all the references associated with its DOI prefixes are made open – all it requires is an email request to support@crossref.org.  But without such a request, the references will be remain in the default ‘Limited’ status.

References that are not associated with Crossref

There are, of course, many scholarly publications, for example preprints in repositories such as arXiv, and journal articles and monographs from small academic publishers in the Humanities, that do not have Digital Object Identifiers issued by Crossref.  There are also an increasing number of datasets in repositories such as Dryad that have associated references to the scholarly literature, but whose DOIs are issued by DataCite.  None of these submit references to Crossref where they can be made available via the Crossref API, and separate additional measures will be required to capture and share their references with the community.

 

 

 

 

The new Crossref reference distribution policy

Since 1st January 2018, Crossref has had a new reference distribution policy, described at https://www.crossref.org/reference-distribution/.

There are three possible options for setting the reference distribution preference from which a publisher can choose, these being ‘Closed’, ‘Limited’ and ‘Open”.

If the ‘Closed’ option is chosen, the references will only be used for the Crossref Cited-by service, and are not distributed via any of the other Crossref interfaces.

If the ‘Limited’ option is chosen, the references will additionally be made available to Crossref metadata subscribers who have signed an agreement for the new Crossref Metadata APIs ‘Plus’ service which also came into effect on 1st January 2018.  This is the new Crossref default option.

If the ‘Open’ option is chosen, the references will be openly available to anyone using the Crossref APIs.

Publishers will no longer be able to select the reference distribution preference for individual publications on a case-by-case basis, but rather the preference will be set for all publications appearing under a particular DOI prefix, with the new default option being ‘Limited’.

To date, more than 60 scholarly publishers, including most of the major ones, have responded to the Initiative for Open Citations and have instructed Crossref to set their references to open, as describe in a previous post.

Openness of non-Elsevier references

For completeness, this post, also based on analyses performed by Daniel Ecer of eLife (d.ecer@elifesciences.org) on data he downloaded from Crossref in September 2017 (Ecer, 2017), complements the two preceding posts, and details the openness of references from scholarly publishers other than Elsevier.

 The main conclusion is that, of the 650,093,489 references stored in Crossref from journal articles published by publishers other than Elsevier, 486,041,671 (74.76%) are open.

The detailed statistics derived from the Crossref data at the time of sampling relating to all publishers except Elsevier are as follows:

Number of works recorded at Crossref from publishers other than Elsevier

Crossref has records of 93,184,372 works with DOIs, of which 69,699,633 (74.80%) are journal articles and 23,484,739 (25.20%) are works that are not journal articles (i.e. book chapters, proceedings articles, datasets, etc.).

Of the 93,184,372 works, 76,795,932 (82.41%) were published by publishers other than Elsevier.

Of the 69,699,144 journal articles, 54,440,761 (78.11%) were in journals with publishers other than Elsevier.

Of the 23,484,739 works that are not journal articles, 22,355,171 (95.19%) were published by publishers other than Elsevier.

Numbers of non-Elsevier works with references

Of all 76,795,932 works with DOIs recorded in Crossref from publishers other than Elsevier, 27,609,963 (35.95%) have accompanying references and 49,185,969 (64.05%) lack references.

Of the 54,440,761 journal articles recorded in Crossref from publishers other than Elsevier, 23,459,805 (43.09%) have accompanying references, and 30,980,956 (56.91%) lack references.

Of the 22,355,171 works that are not journal articles recorded in Crossref from publishers other than Elsevier, 4,150,158 (18.56%) have accompanying references, and 18,205,013 (81.44%) lack references.

Number of non-Elsevier references at Crossref

Of the 1,075,133,743 references stored in Crossref from all works, 732,513,350 (68.13%) are from works published by publishers other than Elsevier.

Of the 956,050,193 references stored in Crossref from journal articles, 650,093,489 (68.00%) are from journals published by publishers other than Elsevier.

Of the 119,083,550 references stored in Crossref from works that are not journal articles, 82,419,861 (69.21%) are from works published by publishers other than Elsevier.

Average numbers of references per non-Elsevier work

The 732,513,350 non-Elsevier references stored in Crossref come from 27,609,963 works of all types with accompanying references, giving an average of 26.53 references per work.

650,093,489 non-Elsevier references come from 23,459,805 non-Elsevier journal articles with accompanying references, giving an average of 27.71 references per journal article.

82,419,861 non-Elsevier references come from 4,150,158 non-Elsevier works with accompanying references that are not journal articles, averaging 19.86 references per work.

Proportion of non-Elsevier works that have open references

Of the 27,598,963 non-Elsevier works of all type documented in Crossref that have accompanying references, 18,228,221 (66.05%) have open references.

Of the 23,459,805 non-Elsevier journal articles documented in Crossref that have accompanying references, 17,072,801 (72.77%) have open references.

Of the 4,139,158 non-Elsevier works documented in Crossref that are not journal articles and that have accompanying references, 1,155,420 (27.91%) have open references.

Proportion of non-Elsevier references that are open

Of the 732,513,350 references stored in Crossref from all works published by publishers other than Elsevier, 523,186,205 (71.42%) are open, and 209,327,145 (28.58%) are not open.

Of the 650,093,489 references stored in Crossref from journal articles published by publishers other than Elsevier, 486,041,671 (74.76%) are open, and 164,051,818 (25.24%) are not open.

Of the 82,419,861 references stored in Crossref from works published by publishers other than Elsevier that are not journal articles, 37,144,534 (45.07%) are open, and 45,275,327 (54.93%) are not open.

Proportion of references which are not open that are published by publishers other than Elsevier

Of the 551,932,682 references from all works stored at Crossref that are not open, 209,327,145 (37.93%) are from works published by publishers other than Elsevier.

Of the 470,008,522 references from journal articles stored at Crossref that are not open, 164,051,818 (34.90%) are from journal articles published by publishers other than Elsevier.

Of the 81,924,160 references from works that are not journal articles stored at Crossref that are not open, 45,275,327 (55.26%) are from works published by publishers other than Elsevier.

 

 Details for all publishers combined, and for Elsevier separately, are given in the two previous posts.

 

Reference

Ecer, D. (2017). Crossref Data Notebook. Available at https://elifesci.org/crossref-data-notebook

 

Elsevier references dominate those that are not open at Crossref

Yesterday (November 23rd 2017) I was working with Daniel Ecer of eLife (d.ecer@elifesciences.org) to dig some hard facts out of the analyses he undertook on data he downloaded from Crossref in September 2017 (Ecer, 2017).  Because of its dominant position in the scholarly publishing world, in this, the second of two related posts, I report the results for references from works published by Elsevier.

These show that, of all 956,050,193 references from journal articles stored at Crossref, 305,956,704 (32.00%) are from journal articles published by Elsevier, none of which are in the Crossref “Open” category, freely available for others to use.

Put another way, of the 470,008,522 references from journal articles stored at Crossref that are not open, 305,956,704 (65.10%) are from journals published by Elsevier.

On behalf of I4OC, I appeal to Elsevier to join the other major academic publishers and to submit and open all its references without delay.

The detailed statistics derived from the Crossref data at the time of sampling relating to Elsevier publications are as follows:

Number of Elsevier works recorded at Crossref

Crossref has records of 93,184,372 works with DOIs, of which 69,699,633 (74.80%) are journal articles and 23,484,739 (25.20%) are works that are not journal articles (i.e. book chapters, proceedings articles, datasets, etc.).

Of the 93,184,372 works of all types, 16,388,440 (17.59%) were published by Elsevier.

Of the 69,699,633 journal articles, 15,258,872 (21.89%) were published in Elsevier journals.

Of the 23,484,739 works that are not journal articles, 1,129,568 (4.81%) were published by Elsevier.

Numbers of Elsevier works with references

Of all 16,388,440 Elsevier works with DOIs recorded in Crossref, 10,835,273 (66.12%) have accompanying references, and 5,553,167 (33.88) lack references.

Of the 15,258,872 Elsevier journal articles recorded in Crossref, 10,212,958 (66.93%) have accompanying references, and 5,045,914 (33.07%) lack references.

Of the 1,129,568 Elsevier works that are not journal articles recorded in Crossref, 622,315 (55.09%) have accompanying references, and 507,253 (44.91%) lack references.

Number of Elsevier references at Crossref

Of the 1,075,133,743 references stored in Crossref from all works, 342,620,393 (31.87%) are from works published by Elsevier.

Of the 956,050,193 references stored in Crossref from journal articles, 305,956,704 (32.00%) are from journals published by Elsevier.

Of the 119,083,550 references stored in Crossref from works that are not journal articles, 36,663,689 (30.79%) are from works published by Elsevier.

Average numbers of references per Elsevier work

The 342,620,393 Elsevier references stored in Crossref come from 10,835,273 works of all types with accompanying references, giving an average of 31.62 references per work.

305,956,704 Elsevier references come from 10,212,958 Elsevier journal articles with accompanying references, giving an average of 29.96 references per journal article.

36,663,689 Elsevier references come from 622,315 Elsevier works with accompanying references that are not journal articles, averaging 58.92 references per work.

Proportion of Elsevier works that have open references

Of the 10,846,273 Elsevier works of all type documented in Crossref that have accompanying references, 417 (0.0038%) have open references.

Of the 10,212,958 Elsevier journal articles documented in Crossref that have accompanying references, none (0.0000%) have open references.

Of the 633,315 Elsevier works documented in Crossref that are not journal articles and that have accompanying references, 417 (0.0658%) have open references.

Proportion of Elsevier references that are open

Of the 342,620,393 references stored in Crossref from works of all types published by Elsevier, 14,856 (0.0043%) are open, and 342,605,537 (99.9957%) are closed.

Of the 305,956,704 references stored in Crossref from journal articles published by Elsevier, none (0.0000%) are open, 100% being closed.

Of the 36,663,689 references stored in Crossref from works published by Elsevier that are not journal articles, 14,856 (0.0405%) are open, and 36,648,833 (99.9595%) are closed.

Proportion of references which are not open that are published by Elsevier

Of the 551,932,682 references from all works stored at Crossref that are not open, 342,605,537 (62.07%) are from works published by Elsevier.

Of the 470,008,522 references from journal articles stored at Crossref that are not open, 305,956,704 (65.10%) are from journal articles published by Elsevier.

Of the 81,924,160 references from works that are not journal articles stored at Crossref that are not open, 36,648,833 (44.74%) are from works published by Elsevier.

 

Details for all publishers combined are given in the previous post, and those for all publishers other than Elsevier in the following post.

 

[Note: As a result of further calculations undertaken by Daniel Ecer on 27th November 2017, which are recorded in his updated Crossref Data Notebook (Ecer, 2017), the figures in this blog have been expanded to show the average number of references for Elsevier works submitting references.  At the same time, very minor corrections have been made the total numbers of works in each category, which have not altered the percentages and main conclusions presented in this post.]

 

Reference

Ecer, D. (2017). Crossref data notebook. Available at https://elifesci.org/crossref-data-notebook

 

Milestone for I4OC – open references at Crossref exceed 50%

Yesterday (November 23rd 2017) I was working with Daniel Ecer of eLife (d.ecer@elifesciences.org) to dig some hard facts out of the analyses he undertook on data he downloaded from Crossref in September 2017 (Ecer, 2017).  In this, the first of two related posts, I report the results for all publishers.

The analyses show that, of the 33,672,763 journal articles documented in Crossref that have accompanying references, 17,072,801 (50.70%) have open references, and of the 956,050,193 references from journal articles stored at Crossref, 486,041,671 (50.84%) are now classified as “Open”, and are freely available for third parties to download and use for any purpose.

This is a significant milestone for the Initiative for Open Citations (I4OC, https://i4oc.org/), which since early 2017 has been campaigning for scholarly publishers to open their reference lists, and a major gain for the world of open scholarship.

The academic community is deeply indebted to all those publishers whose references are now open (https://i4oc.org/#publishers), and to Crossref itself (https://www.crossref.org/), for making these references freely available, providing a tremendous resource for bibliometric analysis.

However, 51.7% of the journal articles recorded in Crossref lack accompanying references, and of the references that are submitted together with the metadata for the remaining journal articles, 49.16% are yet not open.

On behalf of I4OC, I strongly encourage those publishers who are not yet submitting references to Crossref with their article metadata to start to do so, and those other publishers who are submitting references but have not yet made them open to open them without delay, by sending a message requesting this to support@crossref.org.

The detailed statistics derived from the Crossref data at the time of sampling relating to all publishers are as follows:

Works with DOIs documented at Crossref

Crossref has records of 93,184,372 works with DOIs, of which 69,699,633 (74.80%) are journal articles and 23,484,739 (25.20%) are works that are not journal articles (i.e. book chapters, proceedings articles, datasets, etc.).

Numbers of works with references

Of all 93,184,372 works with DOIs recorded in Crossref, 38,445,236 (41.3%) have accompanying references and 54,739,136 (58.7%) lack references.

Of the 69,699,633 journal articles recorded in Crossref, 33,672,763 (48.3%) have accompanying references, and 36,026,381 (51.7%) lack references.

Of the 23,484,739 works that are not journal articles recorded in Crossref, 4,772,473 (20.32%) have accompanying references, and 18,709,979 (79.68%) lack references.

Numbers of references at Crossref

Crossref stores 1,075,133,743 references from all 93,184,372 works.

Of these references, 956,050,193 references (88.92%) are from journal articles, and 119,083,550 references (11.08%) are from works that are not journal articles.

Average numbers of references per work

The 1,075,133,743 references stored in Crossref come from 38,445,236 works of all types with accompanying references, giving an average of 27.97 references per work.

956,050,193 references come from 33,672,763 journal articles with accompanying references, giving an average of 28.39 references per journal article.

119,083,550 references come from 4,772,473 works with accompanying references that are not journal articles, averaging 24.95 references per work.

Proportion of works that have open references

Of the 38,445,236 works of all type documented in Crossref that have accompanying references, 18,228,638 (47.41%) have open references.

Of the 33,672,763 journal articles documented in Crossref that have accompanying references, 17,072,801 (50.70%) have open references.

Of the 4,772,473 works documented in Crossref that are not journal articles and that have accompanying references, 1,155,837 (24.22%) have open references.

Proportion of references that are open

Of the 1,075,133,743 references stored in Crossref from works of all types, 523,201,061 (48.66%) are open, and 551,932,682 (51.34%) are not open.

Of the 956,050,193 references stored in Crossref from journal articles, 486,041,671 (50.84%) are open, and 470,008,522 (49.16%) are not open.

Of the 119,083,550 references stored in Crossref from works that are not journal articles, 37,159,390 (31.20%) are open, and 81,924,160 (68.80%) are not open.

The majority of the references that are not yet open are from works published by Elsevier, as detailed in the next post.

 

[Note: As a result of further calculations undertaken by Daniel Ecer on 27th November 2017, which are recorded in his updated Crossref Data Notebook (Ecer, 2017), the figures in this blog have been expanded to show the average number of references for works submitting references.  At the same time, very minor corrections have been made the total numbers of works in each category, which have not altered the percentages and main conclusions presented in this post.]

 

Reference

Ecer, D. (2017). Crossref Data Notebook. Available at https://elifesci.org/crossref-data-notebook