The Total Efficiency of Teaching Activity of Polish Higher Education Institutions

The higher education system plays an extremely important role in the development of many countries. It enables the acquisition of necessary qualifications and skills (provides human capital of adequate quality) for the needs of economies. Therefore, many countries are involved in financing the activities of higher education institutions. In Poland, the Ministry of Science and Higher Education (MSHE), since the beginning of the reform process has emphasized in its documents the need to spend public funds effectively and improve the quality of educational services.

It was already indicated in the assumptions to the Act on Higher Education of 2011 (MSHE, 2009, p.22) that “the proposed changes are primarily of a pro-quality nature and lead to an improvement in the efficiency of spending public funds on higher education”. “By 2030, development goals were adopted, including, among others (Ministry of Science and Higher Education, 2015a, p. 6) increasing the quality of education (...), improving the functioning of the higher education system through changes in the areas of organization, management and financing of teaching activities”. Nucińska (2017, p.106) states that “public entities need efficient management more than commercial entities. This is due to the lack of market stimuli affecting them which would spontaneously force effective and efficient action.”

The aim of the study was to estimate the level of efficiency of the total teaching activity of Polish public higher education in 2015 using the data envelopment analysis (DEA) method. In addition, it was decided to analyze the revenues and costs of teaching activity in the context of total efficiency indicators.

The added value of the article is determining the total efficiency of teaching activity, of various forms of education using the epsilon-based measure (EBM) model, which has not been used so far, either in Poland or abroad.

The article consists of seven parts. The first has the character of an introduction, the second part reviews the literature on the applied research variables and DEA models. The next, methodical part characterizes the EBM model used in empirical research. In the fourth and fifth parts respectively, the research sample was defined and the variables used for the efficiency analysis were defined. The results of the research and their interpretation are presented in the sixth part. The final conclusions and directions of further research are presented in the seventh part of the article in the form of a summary.

The studies of Brzezicki (2018a) and Brzezicki and Prędki (2018) show that the non-parametric DEA method is most often used to measure the efficiency of higher education. However, it should be noted that not all of the analytical possibilities of this method have been exhausted, nor all problems connected with the functioning of higher education in Poland (especially in the field of teaching) have been solved. Researchers involved in the evaluation of the process of teaching universities using the DEA method focused primarily on first and second cycle studies and uniform master’s studies (Brzezicki, Pietrzak, 2017; Ćwiąkała-Małys, 2010; Wolszczak-Derlacz, 2013). Occasional attention was paid to doctoral studies (Brzezicki and Pietrzak, 2018; Flégl, Tichá and Stanislavská, 2013).

Usually the issues of doctoral studies, if they were raised at all, were discussed only in the context of first and second cycle studies (Chodakowska, 2015; Nazarko and Šaparauskas, 2014; Szuwarzyński, 2005; Mikušová, 2015). There are also very few analyses in which post-graduate studies were considered (Johnes, 2006; Flegg, et al., 2003). In foreign literature, only in a few studies on the efficiency of higher education, the authors discussed a number of various forms (first and second cycle studies, doctoral and postgraduate studies) of academic education in the same work (Johnes, 2006; Visbal-Cadavid, et al., 2017). And as far as Polish literature is concerned, no assessment of total effectiveness (concerning all forms of higher education) has yet been made.

Depending on the purpose of the study, the following diagnostic variables were used in measuring the efficiency of teaching activities: the number of students, the number of doctoral students, or the number of graduates, the number of doctors or both.

As indicated by Wolszczak-Derlacz (2013, p.35) “the problematic variable is the number of students

Including doctoral students.

A similar situation was in the case of doctoral studies. The number of participants in doctoral studies was adopted as a result of teaching activities by Chodakowska (2015) and Nazarko and Šaparauskas (2014), and the aggregate number of degrees awarded, including the Ph.D. degrees as a result, but of a scientific activity (Brzezicki and Prędki, 2018). Pietrzak (2016a, b), however, placed the number of doctoral students on the input side. And Johnes (2006) assumed on the input side the number of first-cycle students and post-graduate students, and classified as a result the number of professional titles awarded at first-cycle studies and the combined number of doctoral degrees awarded and other higher titles.

Three groups of variables were usually assumed as expenditures, that is, personal, financial and tangible. The most commonly used variable was the number of academic teachers (Ćwiąkała-Małys, 2010), sometimes also the number of other employees who are not teaching staff (Brzezicki and Prędki, 2018). In terms of financial data, the value of teaching subsidies (Brzezicki, 2016), total teaching revenues (Brzezicki, 2017) or total revenues of the university (Visbal-Cadavid, et al., 2017) were taken into account. In the case of tangible variables, the value of assets (e.g., Ćwiąkała-Małys, 2010), fixed assets (Brzezicki and Prędki, 2018) and various types of costs, in the form of, for example, consumption of materials and energy, external services, gross wages and depreciation (Świtłyk, 2012) were considered.

To test the efficiency of higher education using the DEA method, two standard radial models are most often used, namely CCR (proposed by Charnes, Cooper and Rhodes) and BCC (proposed by Banker, Charnes and Cooper) (Ćwiąkała-Małys, 2010). However, within the DEA method has created many different models and their modifications that have new analytical capabilities previously unavailable. Some authors involved in the study of higher education recognize the potential and possibilities of new DEA models, an example of which are subsequent attempts to use them. These include: the non-radial SBM model (Brzezicki, 2017; Visbal-Cadavid et al., 2017), non-radial network slacks-based measure (SBM) model (Pietrzak and Brzezicki, 2017), network model (Chodakowska, 2015), relational model (Brzezicki and Pietrzak, 2017).

The above summary shows that the studies can be divided into those carried out using radial and non-radial models. Yet, Brzezicki and Pietrzak (2018) made the first Polish attempt to use these two approaches in one hybrid model. The authors of this article decided to develop and deepen the analysis of the efficiency of higher education using a model combining radial and non-radial approach.

As noted in the previous part of the article, the DEA method is very often used to examine the efficiency of the higher education system. The first DEA model presented was the radial CCR model, assuming constant scale effects (Charnes, et al., 1978). And the non-radial SBM model was developed as long as 23 years later as a response to the imperfections of radial efficiency measures (Tone, 2001).

In addition, in the non-radial SBM model, there were taken into account mismatched values (gaps) that arise when optimizing the objective function when there are surplus inputs and deficiencies in the results of the entity’s empirical technology in comparison to the optimal technology (Kozuń-Cieślak, 2011). Brzezicki (2017), presenting the differences between the two approaches, pointed out that in the case of efficiency testing using a radial CCR model aimed at minimizing input, all input is proportionally reduced. However, there are no such situations in economic practice, as various inputs have a differentiated impact on the efficiency of the economic entity (Johnes and Tone, 2017).

Therefore, non-radial models are based on the assumption that it is possible to achieve full efficiency of the decision making unit (DMU) with various combinations of its resources. However, the simultaneous use of both measures, that is, radial and non-radial allows comprehensive examination of the efficiency of a given business entity.

Tone and Tsutsui (2010) presented the EBM model that combines the above two approaches in a uniform structure. The EBM model has two parameters: one scalar and one vector. In order to estimate these parameters, Tone and Tsutsui (2010) introduced a new affinity index with inputs or outputs. Based on the principal component analysis (PCA), a relationship matrix is defined. The above two parameters are used to unify the radial approach and non-radial in one coherent EBM model. This model with variable scale effects and output-oriented (EBM-O-V) is defined as follows: (1)1/τ*=maxη+εy∑i=1swi+si+yioXλ≤xo,ηyo−Yλ+s+=0,λ≥0, s+≥0, λn=1,\matrix{{\matrix{{1/\tau^* = \max} \hfill & {\eta + {\varepsilon _{\rm{y}}}\sum\nolimits_{{\rm{i}} = 1}^{\rm{s}} {{{{\rm{w}}_{\rm{i}}^ + {\rm{s}}_{\rm{i}}^ +} \over {{{\rm{y}}_{{\rm{io}}}}}}}} \hfill \cr}} \hfill \cr {{\rm{X}}\lambda \le {{\rm{x}}_{\rm{o}}},} \hfill \cr {\eta {{\rm{y}}_{\rm{o}}} - {\rm{Y}}\lambda + {{\rm{s}}^ +} = 0,} \hfill \cr {\lambda \ge 0,\,\,\,\,\,{{\rm{s}}^ +} \ge 0,\,\,\,\,\,{\lambda _{\rm{n}}} = 1,} \hfill} where:

the input and output matrices are denoted by X and Y,

In contrast, ε_y is a key parameter that combines radial and non-radial clearance values. Values wi+{\rm{w}}_{\rm{i}}^ + and ε_y must be determined before measuring efficiency.

The empirical study adopted the EBM model’s orientation on output, as the purpose of the analysis was to estimate the maximum results of individual forms of education in the form of the number of qualifications and people who received them.

The adopted convention should reflect the overall assessment of teaching activities of universities.

The research sampling was purposeful. The study involved 57 out of 59 academic universities supervised by the Ministry of Science and Higher Education in 2015. Two units were excluded from the study due to a lack of data, that is, the University of Technology and Humanities in Bielsko-Biała (U22), because in 2015, no doctoral degree was awarded in this unit, and the Bronisław Czech Academy of Physical Education in Cracow (U55), where no post-graduate certificates were issued.

Universities accepted for the study (DMU) were given the following codes: U1 – University of Warsaw, U2 – University of Białystok, U3 – University of Gdańsk, U4 – University of Adam Mickiewicz, in Poznań, U5 – Jagiellonian University in Kraków, U6 – University of Lodz, U7 – Maria Curie-Skłodowska University in Lublin, U8 – Nicolaus Copernicus University in Toruń, U9 – University of Opole, U10 – University of Szczecin, U11 – University of Silesia in Katowice, U12 – University of Rzeszów, U13 – University of Warmia and Mazury in Olsztyn, U14 – University of Wrocław, U15 – Cardinal Stefan Wyszyński University in Warsaw, U16 – University of Zielona Góra, U17 – Kazimierz Wielki University in Bydgoszcz, U18 – Jan Kochanowski University in Kielce, U19 – West Pomeranian University of Technology in Szczecin, U20 – Warsaw University of Technology, U21 – Politechnika Białostocka, U23 – Częstochowa University of Technology, U24 – Gdańsk University of Technology, U25 – Silesian University of Technology in Gliwice, U26 – Kielce University of Technology in Kielce, U27 – Koszalin University of Technology, U28 – Tadeusz Kościuszko Cracow University of Technology, U29 – AGH University of Science and Technology in Kraków, U30 – Lublin University of Technology, U31 – Lodz University of Technology, U32 – Opole University of Technology, U33 – Poznan University of Technology, U34 – Kazimierz Pułaski University of Technology and Humanities in Radom, U35 – Ignacy Łukasiewicz Rzeszow University of Technology, U36 – Wroclaw University of Technology, U37 – University of Economics in Katowice, U38 – University of Economics in Krakow, U39 – University of Economics in Poznań, U40 – Warsaw School of Economics, U41 – University of Economics in Wroclaw, U42 – the Maria Grzegorzewska University in Warsaw, U43 – Jan Długosz University in Częstochowa, U44 – The Pedagogical University in Krakow, U45 – Pomeranian University in Słupsk, U46 – University of Natural Sciences and Humanities in Siedlce, U47 – Warsaw University of Life Sciences, U48 – Jan and Jędrzej Śniadecki’s University of Technology and Life Sciences in Bydgoszcz, U49 – University of Agriculture in Krakow, U50 – University of Life Sciences in Lublin, U51 – University of Life Sciences in Poznań, U52 – University of Life Sciences in Wrocław, U53 – Jedrzej Śniadecki Academy of Physical Education and Sport in Gdańsk, U54 – the Jerzy Kukuczka Academy of Physical Education in Katowice, U56 – the Eugeniusz Piasecki Academy of Physical Education Academy of Physical Education in Poznań, U57 – Józef Piłsudski University of Physical Education in Warsaw, U58 – University of Physical Education in Wrocław, U59 – Christian Theological Academy in Warsaw U31.

According to the Ministry of Science and Higher Education classification, all the above-mentioned universities were assigned to specific groups of universities: universities (18 objects, U1–U18), technical universities (17 objects, U19–U36 [without U22]), economic universities (5 objects, U37–41), pedagogical universities (4 objects, U42–U46), agricultural/natural science universities (6 objects, U47–U52), physical education academies (5 objects, U53–U58 [without U55]). Theological colleges were not included in the classification of groups, as there is only one school of that type (U59). Grouping of universities will help to determine whether belonging to a given group affects the level of efficiency.

The selection of appropriate variables characterizing individual forms of education is not trivial, especially in terms of expenditure. Sharing resources during the implementation of various teaching processes, as well as the lack of access to detailed records of universities, only intensifies the problem of determining the level of their use. However, the above problems occur only when analyzing selected forms of study.

The holistic approach to the teaching activity of universities makes it possible to take into account the total expenditure used in the field of educational services, that is, in the form of financial and human resources and tangible (university assets). The first input adopted in the study is the number of academic teachers (full-time and part-time) – X₁, the second is the total value of teaching income – X₂.

Since the system of financing teaching activities in Poland is based on two main financial streams: subsidies from the state budget and fees collected from part-time students, it would be worth assessing the situation of universities in two variants. In the first case, when the expenditure is the total amount of revenue from teaching activities. In the second, when universities would have only a state subsidy and tuition paid by students at their disposal – that is why, the total value of these two financial streams in the form of expenditure was assumed – X₃.

The above procedure aims to check several issues. Firstly, if the value of funds transferred by the state in the form of subsidies is underestimated in relation to the costs of full-time education, and secondly, whether universities duly estimate the costs of part-time studies using appropriate tools (methods), collected accounting data and knowledge about the educational process. The last variable accepted for the study is the value of fixed assets – X₄. The above variables have been used many times in literature by other authors, which was presented in the literature review.

As far as the effects of teaching activity are concerned, the following variables were adopted: the total number of graduates of 1st, 2nd degree and uniform Master’s studies (full-time and part-time, including foreigners) in 2015 (Y₁), the number of doctoral degrees (Y₂) in 2015, the number of post-graduate certificates (2015) issued in 2015 (Y₃).

According to the research assumption, two variants of financial variables assigned to two empirical models were adopted (Table 1). The first M1 model includes the total value of teaching income in 2015, while the second M2, the total value of teaching subsidies and fees for educational services in 2015.

All data used in the study was obtained from the Ministry of Science and Higher Education based on the application on access to public information, from the Ministry of Science and Higher Education (2015b) “Higher education – basic data”, reports on the implementation of the material and financial plan of individual universities. The choice of the research period was dictated by the timeliness and availability of data.

Table 2 presents the synthetic characteristics of the research sample in cross-section of seven diagnostic variables included in the study. Based on the results of two empirical models, the total efficiency of teaching activity of individual universities was determined and their rankings were created on that basis (Table 3).

Fully efficient universities in two EBM models were: University of Warsaw (U1), University of Gdańsk (U3), Jagiellonian University in Kraków (U5), University of Lodz (U6), Nicolaus Copernicus University in Toruń (U8), University of Silesia in Katowice (U11), University of Rzeszów (U12), University of Wrocław (U14), Cardinal Stefan Wyszyński University, in Warsaw (U15), AGH University of Science and Technology in Kraków (U29), University of Economics in Katowice (U37), University of Economics in Krakow (U38), University of Economics in Poznań (U39), Warsaw School of Economics (U40), University of Economics in Wrocław (U41), Maria Grzegorzewska Academy Special Education in Warsaw (U42) and the Christian Academy of Theology in Warsaw (U59).

The lowest result was recorded for the Kielce University of Technology in Kielce (U26).

It is worth noting that those universities that were characterized by full efficiency (index 1.00) in the field of teaching activity in the first empirical model (M1) were also characterized by full efficiency in the second model (M2).

Thus, universities that are efficient did not achieve this result once and by accident, but in fact they repeatedly achieved the best results among the examined academic centers in the analyzed year.

The similarity of the results obtained was influenced by a significant share (in %) of funds obtained in the form of budgetary subsidies and fees paid by part-time students, in revenues from teaching activities (on average at the level of 87%). It is worth noting that some public universities (i.e., the University of Warsaw and the Jagiellonian University) owe their high total teaching efficiency index also to their activity in obtaining additional sources of financing, that is, funds from the budgets of local government units or their associations, or fees collected from training participants, courses – Fig. 1.

Figure 1

In the next step of the analysis, it was decided to check whether there is a correlation between overall effectiveness and the financial result of teaching activity. The authors are aware that public universities are not profit-oriented entities. However, making excessive losses by universities (as part of their activity) is not a positive phenomenon, either. The average financial result of teaching activity (understood as the difference between general revenues and costs from this activity) in 2015 for the analyzed group of universities amounted to −7.9 thousand PLN.

The largest loss was incurred by the University of Warsaw, while the largest profit was recorded by the Jagiellonian University in Krakow. Interestingly, both universities were fully efficient in the field of teaching – Fig. 2.

Figure 2

Within individual types of universities, the highest, average financial result was obtained by economic schools (+2.7 thousand PLN), and the lowest by universities (−12.3 thousand PLN) – Fig. 2. Pearson’s simple correlation coefficient calculated for the average total efficiency (M1 model) and the average financial result from teaching activity for individual types of universities was positive. However, the strength of the relationship can be described as weak

The classification according to Guilford (1956, p. 145) was adopted: | r | = 0 - no correlation; 0.0 < | r | ≤ 0.2 - weak correlation; 0.2 < | r | ≤ 0.4 - low correlation; 0.4 < | r | ≤ 0.7 - moderate correlation; 0.7 < | r | ≤ 0.9 - high correlation; 0.9 < | r | < 1.0 - very high correlation; | r | = 1 - full correlation.

It should be noted that while in the case of the full-time education process, only a part of universities suffered a loss; in the case of part-time studies, the majority did – Fig. 3. This is probably due to the fact that universities use imprecise methods of estimating the cost of education (taking into account only employee labor costs, excluding non-personnel costs such as the cost of using machinery, equipment, energy consumption).

Figure 3

It is worth noting that the largest loss on teaching activity as part of part-time studies was incurred in 2015 by: University of Gdańsk and the Warsaw University of Life Sciences. Of course, it should be remembered that there may be significant disparities between the costs of student education also within individual universities, depending on the course of study.

The following conclusions can be drawn from the research on the total efficiency of teaching activity of selected public universities in Poland.

First of all, the average difference between the results obtained under two empirical models was 0.00, which means that the funds obtained, among others, from the budgets of local government units or their associations have a small share in the total income from teaching activity and have little impact on the efficiency of the university.

Secondly, there is a positive but weak correlation between the total efficiency and the financial result of universities from teaching activity. Thirdly, in most of the analyzed schools, there was a loss from teaching activity from part-time studies, which was probably caused by an incorrect estimation of unit costs of education by universities.

It should be stressed here that the authors are aware of the imperfections of the adopted assumptions and the method used. The undertaken research concerned only a selected group of public universities and was ran for one year (they do not show the change in total efficiency over time). In addition, the university as a whole was adopted as the analytical unit. Within individual departments and even fields of study, there may be significant differences, for example, in terms of unit costs of education. Moreover, the research concentrated only on the quantitative aspect of teaching activity.

Universities that have proved inefficient can be characterized by a very high quality of the education process, and their graduates may have a better position on the labor market (higher salary, shorter time to look for a job and lower risk of being unemployed). It is necessary to conduct more comprehensive and in-depth research in this area, including an attempt to estimate the total efficiency for individual faculties and to supplement the set of diagnostic variables with qualitative variables. For this purpose, data published as part of the Polish Graduate Tracking System - ELA seems to be helpful.

Another interesting research issue would be to use the DEA method to measure allocation efficiency, not just technical efficiency, as has been the case hitherto. It is worth noting, however, that the authors’ query shows that the research methodology presented in this article is novel.

Firstly, various forms of teaching activities, including post-graduate studies, were taken into account in measuring efficiency.

Secondly, the EBM model (combining radial and non-radial approaches) that was not previously used by other researchers was used in the analyses. In conclusion, despite the shortcomings of the methodology adopted, the research results obtained by the authors fill the knowledge gap in the field of the efficiency of public universities.