Applications of, and Experiences with, the Norwegian Model in Finland

Open access

Abstract

The purpose of this article is to describe the development, components and properties of a publication indicator that the Ministry of Education and Culture in Finland uses for allocating direct core funding annually to universities. Since 2013, 13% of the core funding has been allocated on basis of publication indicator that, like the Norwegian model, is based on comprehensive national level publication data that is currently provided by the VIRTA publication information service. In 2015, the publication indicator was complemented with other components of the Norwegian model, namely, quality-weighted publication counts based on national Publication Forum authority list of the publication channels with ratings established by experts in the field. The funding model allocates around 1.6 billion euros annually to universities with the publication indicator annually distributing over 200 million euros. Besides the funding model, the indicator provides comparable data for monitoring the research performance of Finnish universities, fields and subunits. The indicator may also be used in the universities’ local funding models and research management systems, sometimes even at individual level evaluation. Positive and negative effects of the indicator have been extensively discussed and speculated. Since 2011, the Finnish universities’ productivity appears to have increased in terms of both quantity and quality of publications.

Abstract

The purpose of this article is to describe the development, components and properties of a publication indicator that the Ministry of Education and Culture in Finland uses for allocating direct core funding annually to universities. Since 2013, 13% of the core funding has been allocated on basis of publication indicator that, like the Norwegian model, is based on comprehensive national level publication data that is currently provided by the VIRTA publication information service. In 2015, the publication indicator was complemented with other components of the Norwegian model, namely, quality-weighted publication counts based on national Publication Forum authority list of the publication channels with ratings established by experts in the field. The funding model allocates around 1.6 billion euros annually to universities with the publication indicator annually distributing over 200 million euros. Besides the funding model, the indicator provides comparable data for monitoring the research performance of Finnish universities, fields and subunits. The indicator may also be used in the universities’ local funding models and research management systems, sometimes even at individual level evaluation. Positive and negative effects of the indicator have been extensively discussed and speculated. Since 2011, the Finnish universities’ productivity appears to have increased in terms of both quantity and quality of publications.

1 Background and motivation

University funding by government in Finland has been partly based on performance since the establishment in the 1990s of performance agreements, funding models and the KOTA-database for input and output statistics. The main policy goals have been to enhance efficiency, internationalisation, quality and impact of educational and research activities, gaining insights into the performance of the Finnish higher education institutions, as well as creating accountability and transparency regarding the use of public funds. Adaptation of the Norwegian model is embedded in the funding model that the Ministry of Education and Culture uses for allocating direct core funding annually to universities.

The funding model determines what performance indicators are used, and what is their share of core funding, in a fixed funding formula. The next years’ funding is calculated on the basis of three previous year’s performance outputs (average). Over the years, performance-based funding criteria have undergone several changes. In general, the share of core funding allocated on basis of indicators, and the weight of research indicators compared to education, has been increased. Publication statistics have been collected since 1994. The number of scientific publications related to the number of academic staff was introduced to the model in 2007–2012 as an indicator of research performance.

In the current model used since 2017, 41% of core funding is allocated on basis of indicators measuring performance in education, 33% is based on research performance indicators, and 28% on other policy considerations (Table 1). One of four research indicators is the quality-weighted count of scientific publications (share 13%). This indicator was adapted with components of the Norwegian model gradually in 2013–2015.

Table 1

Universities’ funding model 2017–2020.

Performance indicatorsShare (%)
Education41
  • Master’s degrees

13
  • Bachelors’ degrees

6
  • Study credits in open university, specialization studies, studies based on cooperation and in non-degree programmes

2
  • Number of students who have gained at least 55 study credits

10
  • Student feedback

3
  • Number of employed graduates

2
  • Master’s degrees awarded to foreign nationals

1
  • Student mobility to and from Finland

2
Research33
  • PhD degrees

9
  • Scientific publications

13
  • International teaching and research staff

2
  • Competed research funding

9
Other education and science policy objectives28
  • Strategic development

12
  • Field-specific funding

9
  • National duties

7

2 Organization of (re)design, implementation and operations

Development of a publication-based indicator for measuring research performance was discussed and planned since 2005 in several ministry working-groups and reports that investigated publication practices, information sources and international examples. In 2007, a KOTA working-group reviewed the performance-based research funding systems (PRFSs) used at the time in Australia, Flanders, Norway, Sweden and UK. Following the Norwegian model, the KOTA working-group recommended the development of a comprehensive national publication information system to improve the quality of publication data, as well as the development of a national authority list of publication channels to take quality of publications adequately into account in the funding model.

In 2009, the Finnish Council of University Rectors (now Universities Finland UNIFI) set up a working-group to develop the evaluation of quality of publications. Citation analysis, which is well established in science and medicine, was not chosen because of the inadequate coverage of the social sciences and humanities publications in the international databases (Web of Science and Scopus). Based on the Norwegian model adopted already in Denmark, also the UNIFI working-group recommended the development in Finland of a system based on a publication forum, in which peer-reviewed publication channels are identified and classified in quality levels by the experts in the field. Several advantages and disadvantages were identified with this model (Table 2).

Table 2

UNIFI working-group’s SWOT analysis of a publication forum-based system.

StrengthsWeaknesses
  • promotes quality

  • burdensome expert-panel organisation

  • quality defined by research community

  • panels tie human resources

  • sensitive to different publishing practices

  • requires harmonization of publication data

  • could be established e.g. only for SSH fields

  • renewal of field-classification (in Finland)

  • transparent indication of quality

  • validity may differ between fields

  • takes into account quantity and quality

  • comprehensive information for all fields

  • dynamic system that can be updated



OpportunitiesThreats
  • current data about research quality

  • experts privilege channels of national interest

  • supports research assessments

  • over quality in ratings

  • steering and management

  • ensuring commensurability

  • field and subfield specific analyses

  • difficult to reach consensus about ratings

3 Data sources and indicator design

There is a clear distinction of responsibilities with regard to the design of the publication indicator and the development of its different components: Federation of Finnish Learned Societies (TSV) produces together with the research community the publication channel authority list with level ratings, CSC – IT Center for Science Ltd. collects the comprehensive national level publication data from universities, while the funding model working-groups constituted by the Ministry Education and Culture decide if and how the publication data and channel ratings are used in the funding model indicator.

TSV, an independent organisation that represents the research community via its over 250 member societies, has been entrusted with the responsibility to produce the national authority list of peer-reviewed publication channels. Funding for this and other TSV activities comes from the Ministry. In 2010, TSV appointed a steering-group for a two-year project, which established a secretariat (2 persons fulltime) and appointed 23 expert panels covering all fields specifically for the task of evaluating the outlets. Panels have around 250 members from Finnish universities and public research institutes. The first authority list was published in the beginning of 2012, and after the project ended, the Publication Forum has become a fixed part of the TSV activities.

The most notable differences between the Norwegian and the Finnish authority lists concern the number of panels, the number of quality levels, the rating of national language channels, and the interval of updates. The rating has three levels for peer-reviewed publication series and book publishers: level 3 = top, level 2 = leading and level 1 = basic. There is also a level 0 for channels not qualifying as level 1. Both foreign and national journals/series, conferences and book publishers can be admitted to level 1 if they meet the following four criteria:

  • specialised in the publication of scientific or scholarly research outcomes

  • editorial board constituted by experts

  • entire manuscripts of scientific or scholarly articles or books subject to peer review

  • registered ISSN or ISBN number

As a main rule, however, even a publication channel meeting these criteria should not be included in Level 1 if they are local (mainly used by researchers of a single research organisation) or the quality and relevance to Finnish research community is questionable (e.g. predatory journals).

The authority list contains more than 30 000 publication series and book publishers, of which levels 2 and 3 include 10% of the most widely respected international journals and book publishers in different fields (table 3). In the social sciences and humanities fields, however, level 2 has also included since 2012 three book publishers and over 20 journals and book series publishing in the national languages (Finnish and Swedish). To balance the ratings across fields, in each expert-panel the publication channels selected for level 2 can represent in total up to 15%, and in level 3 up to 5%, of the world’s publications.

Table 3

Publication Forum authority list of publication channels in 2018.

Publication Forum levelJournals/seriesBook publishers
365415
22,24291
119,8371,203
05,0471,710
All27,7803,019

The authority list of publication channels is updated regularly, new channels being evaluated for addition to level 1 every year, and levels 2 and 3 are updated every four years. Also, the panels and the steering-group are appointed for four-year periods. Publication Forum ratings are openly available at the JUFO-portal, which the members of research community can also use for suggesting new additions and changes to the ratings.

CSC, a company partly owned by the state of Finland (70%) and Finnish higher education institutions (30%), has developed technical solutions for the national level integration of publication metadata from the universities since 2009. Finland faced the challenge that many universities had strongly invested in their own information systems, so it was not considered feasible to replace those systems with a national CRIS (such as CRISTIN in Norway). Integrating data from diverse local information systems has involved development of the data model, the publication type and field classifications, as well as the data collection guidelines and procedures to ensure quality and comparability of the publication data. Full publication data from 2011 to the present is currently stored in the VIRTA publication information service, an advanced solution launched in 2016.

In VIRTA, publications are divided in scientific, professional and general publications, and for the scientific publications separate publication types are used for the refereed and not-refereed articles in journals, conferences and books, monographs and edited work (Table 4). Publications are assigned to the publication types in universities at the time of registering of publications in the local information systems, from which these classifications are transferred with publication metadata to VIRTA. VIRTA stores records also for artistic and design activities, theses, patents and innovation announcements, as well as audiovisual material and ICT applications, but these have not been taken into account in the publication indicator.

Table 4

Publication type classification in VIRTA publication information service.

Scientific publications
A1 Peer-reviewed journal article, original research
A2 Peer-reviewed journal article, review
A3 Peer-reviewed article or chapter in book
A4 Peer-reviewed article in conference proceedings
B1 Non-peer-reviewed journal article
B2 Non-peer-reviewed article or chapter in book
B3 Non-peer-reviewed article in conference proceedings
C1 Monograph (peer-reviewed)
C2 Edited book or special issue (peer-reviewed)
Professional publications
D1 Article in a trade journal
D2 Article in a professional book
D3 Article in professional conference proceedings
D4 Published development or research report or study
D5 Textbook, professional manual or guide
D6 Edited professional book
Publications for the general public
E1 Popularised article, newspaper article
E2 Popularised monograph
E3 Edited popularised book

According to the data collection guidelines published by the Ministry of Education and Culture, scientific publications need to fulfil the following three conditions:

  1. The publication must produce new information in relation to previous research data on the same subject.

  2. The publication must be presented in a format that enables the verification of the research results and/or use of the research results in a new research, thus allowing other researchers to assess the research results and use them in their own work.

  3. The publication channel for the publication specialises in publishing scientific research results, and it has an editorial staff consisting of experts in the field of science as well as a peer review practice.

A pre-publication peer-review, focusing on research quality, by independent experts in the field is required for inclusion of publications among the peer-reviewed publication types (A and C). To support the identification of peer-reviewed publications, TSV introduced in 2014 a national label for peer-reviewed publications inspired by the example of the GPRC-label in Flanders. Ten Finnish book publishers and 167 journals currently use the label to indicate articles and books that have undergone peer-review according to the label requirements.

The funding model is developed by working-groups appointed periodically by the Ministry to consider adjustments to the performance indicators and their share of the core funding. These groups have representatives from the Ministry, the higher education institutions, and the Academy of Finland (the main research funding agency). A funding model proposal is published in a report and sent out by the Ministry for consultation to a wide group of stakeholders. A decree concerning the universities’ core funding criteria is amended according to the new model approved by the Ministry.

The publication information now stored in VIRTA has been used as the data source of publication indicator since 2013. Weighting of publications according to the publication type was also introduced already in 2013, as monographs (C1) were counted with four times larger weight than articles (A1–A4, B 1–B3). The Publication Forum levels have been identified for all peer-reviewed articles and monographs published in 2011 or later. In the period 2013–2014, however, the funding model did not yet use the Publication Forum levels to differentiate publications according to quality. Instead, 9% of core funding was allocated on basis of an undifferentiated count of international refereed publications and 4% based on the other publications. The former category included publication types A1–A4 and C1 with country of publication other than Finland, and the latter category included all the other publications belonging to types A, B and C.

The funding model working-group originally planned that in 2015, international refereed publications would be replaced with the count of publications in Publication Forum level 2 and 3 channels, and other publications with publications in level 1 channels. Eventually, a weighting of publications by type and channel was introduced in 2015, and later adjusted for the period 2017–2020 (Tables 5 and 6). The Finnish publication indicator differs somewhat from the Norwegian model as it assigns similar weight to peer-reviewed articles in journals, books and proceedings, and takes into account with small weight not-refereed publications intended for scholarly, professional and general audiences (publication types D and E).

Table 5

Weight of publications in the funding model 2015–2016.

Publication typeLevel 3Level 2Level 1Level 0
Peer-reviewed monograph (C1)121264
Peer-reviewed article in journal (A1–2)331.51
Peer-reviewed article in book (A3)331.51
Peer-reviewed article in proceedings (A4)331.51
Not-peer-reviewed monographs0.4
Not-peer-reviewed articles0.1
Table 6

Weight of publications in the funding model since 2017–2020.

Publication typeLevel 3Level 2Level 1Level 0
Peer-reviewed monograph (C1)161240.4
Peer-reviewed article in journal (A1–2)4310.1
Peer-reviewed article in book (A3)4310.1
Peer-reviewed article in proceedings (A4)4310.1
Peer-reviewed edited work (C2)4310.1
Not-peer-reviewed monographs0.4
Not-peer-reviewed articles and edited work0.1

Institutional level whole-counts are used, meaning that co-publications with authors from several Finnish universities are counted more than once in the funding model. Fractionalization at institutional or author level has been discussed but not adopted in order not to discourage inter-university collaboration.

The Universities Act and the Polytechnics Act require all higher-education institutions, including 14 universities and 23 universities of applied sciences, to supply publication information to Ministry of Education and Culture. Also five university hospital districts, each consisting of several hospitals, and six public research institutes have agreed to provide their publication data to VIRTA. The publication information is openly available at the JUULI-portal for browsing and searching, and at the VIPUNEN-portal for statistics.

4 Funding implications

The funding model allocates around 1.6 billion euros annually to universities, and this core funding covers almost two-thirds of the universities’ budgets. The share of core funding allocated on basis of scientific publications has increased considerably, being 0.3% in 2007–2009, 1.7% in 2010–2012, and 13% since 2013. Since 2013, the publication indicator distributes annually over 200 million euros. In a report published in 2018 by Economic Policy Council on the university reform and financial Incentives, it was estimated that one publication point (equivalent e.g. to one peer-reviewed article in level 1 journal) is worth approximately 4200 euros to universities (Seuri & Vartiainen, 2018).

5 Other uses of the data

According to the UNIFI working-group, the development of the Publication Forum based publication indicator supported with national level publication data would serve three potential interests:

  • to provide quality indicator of publications for the universities’ funding model

  • to provide universities with comparable data for benchmarking with other universities, monitoring the publication profiles of units, and developing internal funding models

  • to increase the awareness of the researchers about quality of publication channels, thereby raising their level of ambition in publishing

The working-group also pointed out that Publication Forum could be used as information supporting research evaluation, for example in research assessment exercises the universities conduct themselves every few years, and the state of research in Finland reports produced by the Academy of Finland. Publication Forum ratings have indeed been used to supplement Web of Science based bibliometric analyses in some recent research assessment exercises.

Besides the funding model, the Ministry of Education and Culture uses the information based on the VIRTA publication data and the Publication Forum ratings for monitoring the research performance of the universities. Since 2010, the Ministry’s spending proposals submitted to the Ministry of Finance for the formulation of the state budget includes quantitative research performance targets for the universities collectively. Since 2016, targets have been set for the number of scientific level 2 and 3 publications per academic staff, as well as the international co-publications’ share of scientific level 1–3 publications. The same indicators are used by the Ministry as positioning statistics, openly available at the VIPUNEN-portal, for the performance agreements with individual universities.

The Ministry of Education and Culture has emphasized that the internal allocation of funds to faculties and departments should be based on universities’ own strategic priorities rather than replicate the funding model criteria. Nevertheless, universities are autonomous in using data and indicators in evaluation, development and management of their research activities. While the UNIFI working-group envisioned the potential use of the indicator in universities, appropriate local models and uses of the Publication Forum rating have not been defined. Already in 2012, the Publication Forum steering-group published a user-guide, in which it recommended not to use the rating in cross-field comparisons or in the evaluation of individual researchers.

In the fall 2015, TSV and UNIFI conducted a survey among university managers concerning the local use of the Publication Forum rating. The sample consisted of 10 rectors, 19 deans and 68 heads of department. Most Finnish universities make use of the channel rating for monitoring and developing their publishing activities, and in some universities the rating is used also for funding allocation to faculties and/or departments. The use of the ratings for evaluation purposes (recruitment, promotion, personal performance, bonuses) at the individual level is also attested, especially in social sciences and humanities. It is not clear, however, to what extent the rating supports, rather than supplants, qualitative expert assessment.

VIRTA publication data, including the Publication Forum ratings, has provided the source of information for bibliometric research and analyses concerning for example publications patterns and research productivity.

6 Experiences and effects

In Finland, as in the other Nordic countries using the Norwegian model, the positive and negative effects of the publication indicator have been discussed and speculated extensively. The effect of the publication indicator on research performance has been investigated in two recent reports, one evaluating the impact of the Universities Act reform in 2010 (2016), and another by Economic Policy Council assessing specifically the incentives created by the funding model (2018). In both reports, the development of Finnish universities publication activities was analyzed using the VIRTA publication data and Publication Forum levels. Both reports conclude that the research productivity has increased since 2011. Most notably, there has been a clear movement from level 0 channels toward level 1–3 channels. It seems that the publication channel rating has increased the Finnish research community’s awareness of quality of publication channels, even though the quality-weights were formally introduced to the funding model only in 2015.

The Economic Policy Council’s report also established that the average Publication Forum rating of peer-reviewed publications has increased in all universities, and more so in those universities where the average level was weaker in 2011. The differences in performance have decreased between universities, suggesting that the model has been most effective in those environments where full potential in productivity have not yet been reached. When related to the number of teaching and research personnel, it seems that the publication productivity of academic staff has increased but this has not happened at the expense of quality. But also the question is raised, to what extent the level ratings indicate increase in quality, as the selection of channels to level 2 and 3 can be influenced by local interests. The report also pointed out field specific differences in publication performance, making some fields seem more productive to universities.

Although it is regarded an advantage of the publication indicator that research community itself is entrusted the task of rating the publication channels, subjectivity of the expert-ratings is a common concern in Finland. The indicator has been developed for the purpose of providing comprehensive coverage of research outputs across fields, but equal treatment of different fields and specialties in level 2 and 3 nominations is much debated. The indicator is sensitive to the specificity of social sciences and humanities publishing culture, yet the funding models’ incentives are more often linked with decline rather than promotion of book and national language publishing. Other topics of discussion have included the effect of the publication indicator on interdisciplinarity and open access. Many of the concerns probably stem from the the publication indicator’s use for research management and evaluation purposes at universities, rather than from the indicators’ use in the universities’ funding model.

References

Description of the Finnish higher education system, performance agreements and funding model in English

Aarrevaara, T., Wahlfors, L., & Dobson, I. (2018). Higher Education Systems and Institutions, Finland. In Teixeira, Pedro et al. (eds.), Encyclopedia of International Higher Education Systems and Institutions, Springer 2018.

De Boer, H.F., Jongbloed, B., Benneworth, P., Cremonini, L., Kolster, R., Kottmann, A., Lemmens-Krug, K., & Vossensteyn, H. (2015). Performance-based funding and performance agreements in fourteen higher education systems—Report for the Ministry of Education, Culture and Science, Enschede: CHEPS, University of Twente. Retrieved from https://ris.utwente.nl/ws/portalfiles/portal/5139542/jongbloed+ea+performance-based-funding-and-performance-agreements-in-fourteen-higher-education-systems.pdf

Jonkers, K., & Zacharewicz, T. (2016). Research Performance Based Funding Systems: a Comparative Assessment. European Union. Retrieved from http://publications.jrc.ec.europa.eu/repository/bitstream/JRC101043/kj1a27837enn.pdf

Reports related to the development of the funding model and the publication indicator

Ministry of Education (2005). Yliopistojen tulosohjauksen kehittämistyöryhmä III (in English: Committee on the development of university performance management III). Helsinki: Opetusministeriö. Retrieved from https://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/80321/tr24.pdf?sequence=1

Ministry of Education (2007). Bibliometristen aineistojen käytettävyys yliopistojen julkaisujen laadun ja tuottavuuden arvioinnissa (in English: Usability of bibliometric data in evaluation of quality and productivity of universities’ publications). Helsinki: OPetusministeriö. Retrieved from http://www.minedu.fi/export/sites/default/OPM/Julkaisut/2007/liitteet/opm02.pdf?lang=fi

Ministry of Education (2008). Julkaisukäytännöt eri tieteenaloilla (in English: Publishing patterns in different fields). Helsinki: Opetusministeriö. Retrieved from http://www.minedu.fi/export/sites/default/OPM/Julkaisut/2008/liitteet/opm33.pdf?lang=fi;

Ministry of Education (2008). TiedeKOTA-työryhmän raportti (In English: Report of the TiedeKOTA working-group). Helsinki: Opetusministeriö. Retrieved from http://docplayer.fi/14481463-Tiedekota-tyoryhman-raportti.html

Ministry of Education and Culture (2011). Laadukas, kansainvälinen, profiloitunut ja vaikuttava yliopisto – ehdotus yliopistojen rahoitusmalliksi vuodesta 2013 alkaen (In English: High-quality, profilised and effective international university - Proposal for a reform of the university financing model from 2013). Helsinki: Opetus- ja kulttuuriministeriö. Retrieved from http://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/75464/tr26.pdf?sequence=1&isAllowed=y

Ministry of Education and Culture (2014). Vahvemmat kannusteet koulutuksen ja tutkimuksen laadun vahvistamiselle: ehdotus yliopistojen rahoitusmallin tarkistamiseksi vuodesta 2015 alkaen (In English: Greater incentives for strengthening quality in education and research: A proposal for revising the funding model for universities as of 2015). Helsinki: Opetusja kulttuuriministeriö. Retrieved from https://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/75264/tr07.pdf?sequence=1

Ministry of Education and Culture (2015). Ehdotus yliopistojen rahoitusmalliksi 2017 alkaen (In English: Proposal for the funding model of universities as of 2017). Helsinki: Opetus- ja kulttuuriministeriö. Retrieved from https://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/75157/tr19.pdf?sequence=1

UNIFI (2010). Julkaisujen laadunarvioinnin kehittäminen (In English: Development of evaluation of publication quality). Retrieved from http://www.helsinki.fi/halvi/srno/raportit_ja_julkaisut/Julkaisujen_laadunarviointi.pdf

Ministry of education and culture: Retrieved from https://minedu.fi/en/steering-financing-and-agreements

Budget spending proposals submitted to the Ministry of Finance: Retrieved from http://budjetti.vm.fi/indox/index.jsp

Documents describing the development and properties of the publication channel rating

Auranen, O., & Pölönen, J. (2012). Classification of scientific publication channels: Final report of the Publication Forum project (2010–2012). Helsinki: Federation of Finnish Learned Societies. Retrieved from http://www.julkaisufoorumi.fi/sites/julkaisufoorumi.fi/files/publication_forum_project_final_report_0.pdf

Pölönen, J., & Ruth, A. S. (2015). Final report on 2014 review of ratings in Publication Forum. Helsinki: Federation of Finnish Learned Societies. Retrieved from http://www.julkaisufoorumi.fi/sites/julkaisufoorumi.fi/files/publication_forum_final_report_on_2014_review_of_ratings.pdf

Publication Forum website: www.julkaisufoorumi.fi

JUFO-portal: Retrieved from http://jfp.csc.fi:8080/en/

Label for peer-reviewed publications: Retrieved from https://www.tsv.fi/en/services/label-for-peer-reviewed-scholarly-publications

Description of the properties of the national publication data

Sile, L., Guns, R., Sivertsen, G., & Engels, T. C. E. (2017). European Databases and Repositories for Social Sciences and Humanities Research Output. Antwerp: ECOOM & ENRESSH. Retrieved from https://doi.org/10.6084/m9.figshare.5172322.v2

Sile, L., Pölönen, J., Sivertsen, G., Guns, R., Engels, T., Arefievd, P., Duškováe, M., Faurbækf, L., Hollg, A., Kulczycki, E., Nelhans, G., Petr, M., Piskk, M., Soós, S., Stojanovskil, J., Stone, A., Šušolo, J., & Teitelbaump, R. (2018). Comprehensiveness of national bibliographic databases for social sciences and humanities: findings from a European survey, Research Evaluation doi: 10.1093/reseval/rvy016.

Puuska H. M., Guns, R., Pölönen, J., Sivertsen, G., Mañana-Rodríguez, J., & Engels, T.C.E. (2018). Proof of concept of a European database for social sciences and humanities publications: Description of the VIRTA-ENRESSH pilot. Helsinki: CSC & ENRESSH. Retrieved from https://doi.org/10.6084/m9.figshare.5993506

VIPUNEN-portal for publication statistics. Retrieved from https://vipunen.fi/en-gb/

JUULI-portal for browsing publication information. Retrieved from http://www.juuli.fi/

Funding implications, other uses, effects and experiences

Auranen, O., Leino, Y., Poropudas, O., & Pölönen, J. (2013). Julkaisufoorumi-luokitus ja viittausindeksit tieteellisten julkaisujen laadun mittarina. Web of Science -aineistoon perustuva vertailu (In English: Publication Forum ratings and citation scores as indicators of quality of scientific publications. A comparison based on Web of Science data), TaSTI Työraportteja 8/2013. Retrieved from http://www.uta.fi/yky/tutkimus/tasti/Julkaisut/Sahkoinenkirjasto/Julkaisufoorumi-luokitus%20ja%20viittausindeksit%20tieteellisten%20julkaisujen%20laadun%20mittareina%2019.pdf

Auranen, O., & Pölönen, J. (2014). Julkaisufoorumi-luokitus ja kansallinen julkaiseminen (In English: Publication Forum rating and national publishing). In Muhonen Reetta, Puuska, Hanna-Mari (eds.) Tutkimuksen kansallinen tehtävä. Tampere: Vaystapaino, 153–175.

Hakala, J., & Roihuvuo, J. (2014). Research Assessment Exercise at The University of Tampere 2014: Final Report. Tampere: The University of Tampere. Retrieved from http://www.uta.fi/rae2014/index/UTA_RAE_2014_Final_Report(Web).pdf

Lyytinen, A., Muhonen, R., Koskinen, O., & Eskola, O. (2016). Ulkomaiset tutkijat ja julkaisuaktiivisuus (In English: Foreign researchers and their publishing frequency), Opetusja kulttuuriministeriön julkaisuja 2016:16. Retrieved from http://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/75337/okm16.pdf

Meriläinen, S. (2015). Julkaisufoorumin tasoluokitus: ovatko vaihtoehtoiset lähestymistavat ja poikkitieteellinen tutkimus uhattuina?: Puheenvuoro (In English: Publication Forum ratings: are alternative approaches and interdisciplinary research threatened?). Sukupuolentutkimus 28:3, 45–50. Retrieved from https://sunsseura.files.wordpress.com/2015/01/puheenvuoro_merilainen.pdf

Ministry of Education and Culture (2015). Suomalaisen yliopistotutkimuksen tuottavuus ja vaikuttavuus: opetus- ja kulttuuriministeriön profiiliryhmän raportti (In English: Productivity and impact of Finnish university research. Report by the profile working group of the Ministry of Education and Culture), Opetus- ja kulttuuriministeriön julkaisuja 2015:5. Retrieved from http://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/75132/okm05.pdf

Ministry of Education and Culture (2016). Yliopistolakiuudistuksen vaikutusten arviointi (In English: Impact evaluation of the Universities Act reform). Helsinki: Opetus- ja kulttuuriministeriö. Retrieved from https://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/75416/okm30.pdf?sequence=1

Muhonen, R., Eskola, O., Leino, Y., & Pölönen, J. (2016) Tutkimuksen monitieteisyys ja laatu (In English: The multidisciplinary nature and quality of research), Opetus- ja kulttuuriministeriön julkaisuja 2016:12. Retrieved from http://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/64992/okm12.pdf?sequence=1&isAllowed=y

Pihlström, S. (2014). Academic publishing and interdisciplinarity: Finnish experiences. Human Affairs 24, 40–47. DOI: 10.2478/s13374-014-0204-2.

Poropudas, O. (2018). Miksi suomalaisen tutkimuksen tuottavuus kasvoi, mutta laatu heikentyi? (In English: Why the productivity of Finnish Research Increased but quality decreased?). Informaatiotutkimus 37:1, 5–16. Retrieved from https://journal.fi/inf/article/view/70169/31174

Puuska, H. M. (2014). Scholarly Publishing Patterns in Finland. Tampere: University of Tampere. Retrieved from https://tampub.uta.fi/bitstream/handle/10024/95381/978-951-44-9480-2.pdf?sequence= 1.

Pölönen, J., & Auranen, O. (2017). Julkaisupaine suomalaisessa tiedeyhteisössä (In English, Publishing pressure in Finnish research community). Informaatiotutkimus 36(2), 23–41. Retrieved from https://doi.org/10.23978/inf.65186

Pölönen, J., Auranen, O., Engels, T., & Kulczycki, E. (2018). Taking national language publications into account: the case of the Finnish performance-based research funding system. STI 2018 Conference Proceedings. Retrieved from https://openaccess.leidenuniv.nl/bitstream/handle/1887/65223/STI2018_paper_43.pdf?sequence=1

Pölönen, J., & Sivertsen, G. (2017). Experiences with the rating of publication channels for the Nordic Model. With a response to a proposal for automated ratings from Saarela et al. (2016). Presentation at the 22nd Nordic Workshop on Bibliometrics and Research Policy. Retrieved from https://figshare.com/articles/Experiences_with_the_rating_of_publication_channels_for_the_Nordic_Model_With_a_response_to_a_proposal_for_automated_ratings_from_Saarela_et_al_2016_/5624731

Pölönen, J., & Wahlfors, L. (2016). Local use of a national rating of publication channels in Finnish universities. Poster presented at the 21st Nordic Workshop on Bibliometrics and Research Policy. Retrieved from https://figshare.com/articles/Local_Use_of_a_National_Rating_of_Publication_Channels_in_Finnish_Universities_NWB_2016_poster_/4246541

Saarela, M., Kärkkäinen, T., Lahtonen, T., & Rossi, T. (2016). Expert-based versus citation-based ranking of scholarly and scientific publication channels. Journal of Informetrics 10, 693–718. Retrieved from https://doi.org/10.1016/j.joi.2016.03.004

Seuri, A., & Vartiainen, H. (2018). Yliopistojen rahoitus, kannustimet ja rakennekehitys (In English: University funding, incentives and structural development). Helsinki: Economic Policy Council. Retrieved from https://www.talouspolitiikanarviointineuvosto.fi/wordpress/wp-content/uploads/2018/01/Seuri_Vartiainen_2018-1.pdf

Sivula, A., Suominen, J., & Reunanen, M. (2015). A1 alkuperäisartikkeli tieteellisessä aikakauslehdessä. Uusien julkaisukäytänteiden omaksuminen ihmistieteissä 2000-luvulla (In English: A1 original research article in scientific journal. Embracing new publishing practices in social sciences and humanities in the 21st century). Kasvatus & Aika 9(3), 149–171. Retrieved from http://www.kasvatus-ja-aika.fi/dokumentit/a10_sivulaal_2910151558.pdf

Tähtinen, H., Rautava, P., & Happonen, R. P. (2014). EVO-pisteiden ja Julkaisufoorumin tasoluokituksen vertailu terveystutkimuksen arvioinnissa (In English: Comparison of EVO-points and Publication Forum level ratings for evaluation of health research). Suomen Lääkärilehti 69(10), 714–719.

Wahlfors, L., & Pölönen, J. (2018). Julkaisufoorumi-luokituksen käyttö yliopistoissa (In English: The use of Publication Forum ratings in universties. Hallinnon tutkimus 37(1), 7–21.

Wang, L., Vuolanto, P., & Muhonen, R. (2014). Bibliometrics in the research assessment exercise reports of Finnish universities and the relevant international perspectives, University of Tampere 2014. Retrieved from http://tampub.uta.fi/handle/10024/96266

Journal Information

CiteScore 2018: 1

Source Normalized Impact per Paper (SNIP) 2018: 0.413

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 603 603 30
PDF Downloads 155 155 6