How accurately does output reflect the nature and design of transdisciplinary research programmes?

Elizabeth Koier, E. Horlings

Research output: Contribution to journal/periodicalArticleScientificpeer-review

1 Downloads (Pure)


Many of today’s societal problems are wicked problems that require a new, transdisciplinary approach in which knowledge of scientists and stakeholders from different disciplines is integrated. The evaluation of transdisciplinary science requires a multi-method approach. Bibliometric analysis is consistently among the methods in multi-method evaluations. We analyse the accuracy of bibliometric evidence for the evaluation of transdisciplinary research by examining two large climate adaptation research programmes in the Netherlands. The assessment of accuracy involves a comparison of different approaches to defining and measuring involvement, output, and quality. We draw three conclusions with regard to accuracy. First, scientific output covers a fairly high amount of the scientific activities of the programmes, though information on funding agencies is not yet sufficiently accurate to reconstruct a programme’s output through the Web of Science (WoS). Second, scientific output does not accurately reflect the nature and design of the programmes. The WoS appears to underestimate locally oriented and practically oriented research, non-academic actors rarely co-author scientific publications, and the contributions of non-academic organizations to projects could not be recognized from author affiliations. Third, our exploration of two alternative reproducible metrics (non-scientific output and download statistics) shows that it is too early to introduce such metrics into evaluation practices. The research agenda for transdisciplinary output metrics should focus on the development of a common definition of transdisciplinary research output and a typology of non-scientific outputs, as well as a discussion and assessment of the relative value of such outputs for the integration of knowledge.
Original languageEnglish
Pages (from-to)37-50
JournalResearch Evaluation
Issue number1
Early online date11 Nov 2014
Publication statusPublished - 2014


  • transdisciplinary research
  • evaluation
  • bibliometrics
  • large research programmes
  • Web of science


Dive into the research topics of 'How accurately does output reflect the nature and design of transdisciplinary research programmes?'. Together they form a unique fingerprint.

Cite this