<abstract xmlns="http://www.w3.org/1999/xhtml"><p>This paper proposes a <italic>holistic</italic> framework for the development of <italic>models</italic> for the assessment of research activities and their impacts. It distinguishes three dimensions, including in an original way, <italic>data</italic> as a main dimension, together with <italic>theory</italic> and <italic>methodology</italic>. Each dimension of the framework is further characterized by three main building blocks: education, research, and innovation (<italic>theory</italic>); efficiency, effectiveness, and impact (<italic>methodology</italic>); and availability, interoperability, and “unit-free” property (<italic>data</italic>). The different dimensions and their nine constituent building blocks are attributes of an overarching concept, denoted as “quality.” Three additional quality attributes are identified as implementation factors (tailorability, transparency, and openness) and three “enabling” conditions (convergence, mixed methods, and knowledge infrastructures) complete the framework. A framework is required to develop models of metrics. Models of metrics are necessary to assess the meaning, validity, and robustness of metrics. The proposed framework can be a useful reference for the development of the <italic>ethics</italic> of research evaluation. It can act as a <italic>common denominator</italic> for different analytical levels and relevant aspects and is able to embrace many different and heterogeneous streams of literature. Directions for future research are provided.</p></abstract>

This paper proposes a holistic framework for the development of models for the assessment of research activities and their impacts. It distinguishes three dimensions, including in an original way, data as a main dimension, together with theory and methodology. Each dimension of the framework is further characterized by three main building blocks: education, research, and innovation (theory); efficiency, effectiveness, and impact (methodology); and availability, interoperability, and “unit-free” property (data). The different dimensions and their nine constituent building blocks are attributes of an overarching concept, denoted as “quality.” Three additional quality attributes are identified as implementation factors (tailorability, transparency, and openness) and three “enabling” conditions (convergence, mixed methods, and knowledge infrastructures) complete the framework. A framework is required to develop models of metrics. Models of metrics are necessary to assess the meaning, validity, and robustness of metrics. The proposed framework can be a useful reference for the development of the ethics of research evaluation. It can act as a common denominator for different analytical levels and relevant aspects and is able to embrace many different and heterogeneous streams of literature. Directions for future research are provided.

A Framework for the Assessment of Research and Its Impacts

Department of Computer, Control and Management Engineering A. Ruberti (DIAG)

Journal of Data and Information Science

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

{"article-title":"A Framework for the Assessment of Research and Its Impacts"}

This paper proposes a holistic framework for the development of models for the assessment of research activities and their impacts. It distinguishes three...

A Framework for the Assessment of Research and Its Impacts

Article Category: Perspective

Published Online: Dec 29, 2017

Page range: 7 - 42

Received: Jul 30, 2017

Accepted: Sep 06, 2017

DOI: https://doi.org/10.1515/jdis-2017-0018

Keywords
Evaluation of research, Efficiency, Effectiveness, Impacts, Modeling, Responsible metrics, Ethics of research evaluation

© 2017 Walter de Gruyter GmbH, Berlin/Boston

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

A Framework for the Assessment of Research and Its Impacts

Article Category: Perspective

Published Online: Dec 29, 2017

Page range: 7 - 42

Received: Jul 30, 2017

Accepted: Sep 06, 2017

DOI: https://doi.org/10.1515/jdis-2017-0018

KeywordsEvaluation of research, Efficiency, Effectiveness, Impacts, Modeling, Responsible metrics, Ethics of research evaluation

© 2017 Walter de Gruyter GmbH, Berlin/Boston

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Keywords
Evaluation of research, Efficiency, Effectiveness, Impacts, Modeling, Responsible metrics, Ethics of research evaluation