A novel DEA model for hospital performance evaluation based on the measurement of efficiency, effectiveness, and productivity

Healthcare centres are concerned with the lives of people in the community, and since community health is vital, improving various aspects of the performance of these centres has always been a concern in developing countries (Lupo, 2013). Like all other organisations active in the healthcare sector, hospitals are complex social systems (Buchelt et al., 2017). Hospitals are the most important and costly part of the healthcare system; the key role of hospitals in providing health care

services has shown their profound impact on the proper functioning of the healthcare system. This has forced hospitals to improve their performance (Kohl et al., 2018). Patients regularly experience failed attempts to effectively meet their needs for medical services (Twardowska & Jewczak, 2017). Improved performance requires proper resource allocation, which in turn requires HPE (Yang, 2017). The use of the DEA technique for HPE has attracted researcher attention. Existence of numerous applied studies on the use of DEA for hospital evaluation demonstrates the importance of using this technique (O'Neill et al., 2008). DEA is based on mathematical programming and has been widely used to evaluate the relative performance of similar DMUs. In addition, the capability of analysing the DEA results has consistently increased the applicability of the method in various contexts (Nassiri & Singh, 2009).

There are numerous studies of DEA application in evaluating effectiveness, such as industry (Docekalova & Bockova, 2013; Nazarko & Chodakowska, 2015) companies (Sajnóg, 2015; Grmanová & Pukala, 2018), countries (Chodakowska & Nazarko, 2017a), railway industry (Lan & Lin, 2003; Yu & Lin, 2008), urban transit systems (Karlaftis, 2004), hotel-chain services (Keh et al., 2005), healthcare work (Stefko et al., 2016), Spanish football teams (García-Sánchez, 2007), public higher education institutions (Nazarko & Šaparauskas, 2014), public sector banks (Kumar & Gulati, 2010), couriers and messengers (Chodakowska & Nazarko, 2017b), airlines (Tavassoli et al., 2014), and supply chains (Azadi et al., 2015).

Numerous studies have been conducted on HPE, but almost none of them have simultaneously measured hospital efficiency, effectiveness, and productivity. Traditional DEA models measure the efficiency of DMUs but the DEA-EEP model presented in this study, simultaneously measures the efficiency, effectiveness, and productivity of DMUs. In the related literature, efficiency refers to the inputs-outputs ratio, effectiveness refers to the extent to which outputs align with predetermined goals, and productivity refers to the sum of both efficiency and effectiveness. In addition to simultaneously measuring the efficiency, effectiveness, and productivity, the DEA-EEP model also provides a capability for non-productive units so that they can improve their productivity. The DEA-EEP model provides a new framework for HPE. Efficiency alone cannot reflect the performance of a hospital because the DEA model shows performance compared to similar units. In that way, at least one of DMUs (the best one) will assumedly work perfectly even though its performance is very low; but the proposed approach can create a comprehensive framework for evaluating hospitals by simultaneously measuring the efficiency, effectiveness, and productivity.

In the case study, 11 hospitals in Tehran were evaluated for the period 2016-2018. Once inputs and outputs are identified, experts set goals for the outputs to measure the effectiveness of hospitals in achieving their goals using the DEA-EEP model.

The rest of the article is organised as follows. Section 1 reviews research literature that focuses on the use of DEA in the evaluation of hospitals and treatment centres. Section 2 describes the DEA-EEP model for HPE. Application of the proposed method in evaluating the actual cases of hospitals in Iran is described in Section 3. Section 4 includes a discussion of the evaluation results. The conclusion and directions for future research are presented in the final section.

Literature review

This section reviews the related studies and evaluation methods of healthcare centres. Table 1 shows methods and their applications in the related studies. Table 2 shows the inputs and outputs of the DEA-EEP model, which are selected based on the literature review and expert opinions.

Tab. 1

Summary of the techniques in HPE

AUTHOR(S)	TECHNIQUE(S) USED	APPLICATION
Bannick and Ozcan (1995)	DEA	Hospitals of the US Department of Defence
Ersoy et al. (1997)	DEA	Turkish acute general hospitals
Ouellette and Vierstraete (2004)	DEA	Hospital emergency services in Montreal
O'Neill et al. (2008)	DEA	Systematic review of previous studies
Weng et al. (2009)	DEA	Iowa Hospital Association (IHA)
(Yawe, 2010)	DEA	Hospital Performance Evaluation in Uganda
Chuang et al. (2011)	DEA and regression tree	Taiwan's HOSPITAL
Mitropoulos et al. (2015)	Stochastic DEA and Bayesian analysis	Greek public hospitals
Rezaee and Karimdadi (2015)	Multi-group DEA	Iranian hospitals
Prakash and Annapoorni (2015)	DEA	Hospitals of Tamil Nadu State in India
Gholami et al. (2015)	DEA	US hospitals
Rouyendegh et al. (2016)	DEA-FAHP	Hospitals in Turkey
Chowdhury and Zelenyuk (2016)	DEA with truncated regression	Hospital services in Ontario
Lobo et al. (2016)	DEA	Brazilian hospitals
Khushalani and Ozcan (2017)	Dynamic Network DEA	USA HOSPITAL
Kang et al. (2017)	DEA	USA HOSPITAL
Johannessen et al. (2017)	DEA and panel analysis	Norwegian hospitals
Chen et al. (2017)	DEA	Pennsylvania hospitals
Haghighi and Torabi (2018)	BWM and DEA	A real general HOSPITAL
Zare et al. (2018)	DEA-Game theory	Iran hospitals
Omrani et al. (2018)	Fuzzy Clustering DEA-Game theory	Iran hospitals
This Study	New DEA Model	Hospitals of Tehran

Tab. 2

Definition of sets, parameters, and variables

		DESCRIPTION
Sets	i ∈ I = {1,2,...,m}	Inputs
	r ∈ R = {l,2,...,s}	Outputs
	t ∈ G = {1,2,...,t}	Goals
	j ∈ D = {l,2,...,n}	DMUs
Parameters	$x_{i j}$ ${{x}_{ij}}$	i-th input of j-th DMU
	$y_{r j}$ ${{y}_{rj}}$	r-th output of j-th DMU
	$g_{t j}$ ${{g}_{tj}}$	t-th target of j-th DMU
	$x_{i o}$ ${{x}_{io}}$	i-th input of DMUo
	$y_{r o}$ ${{y}_{ro}}$	r-th output of DMUo
	$g_{t o}$ ${{g}_{to}}$	t-th target of DMUo
Variables	$μ_{j}, λ_{j}$ ${{\mu }_{j}},{{\lambda }_{j}}$	Dual variables of the problem
	$β$ $\beta $	The variable added for linearization
	$t_{i}^{-}$ ${{t}^-_{i}}$	Total maximum of input losses
	$t_{r}^{+}$ $t_{r}^{+}$	Total maximum of outputs that should have been produced with regard to the inputs, but they are not actually produced
	$S_{t}^{-}$ $S_{t}^{-}$	Total maximum of target deficiencies
	$S_{r}^{+}$ $S_{r}^{+}$	Total maximum of outputs that should have been produced with regard to the target values, but they are not actually produced
	$ϕ$ $\phi $	Efficiency of DMU_o
	$θ$ $\theta $	Effectiveness of DMU_o

Bannick and Ozcan (1995) used the DEA model to compare the performance of hospitals of the US Department of Health and the Department of Defence; the results of their study showed that the efficiency of the Department of Defence hospitals was significantly higher than that of the Department of Health hospitals.

Ersoy et al. (1997) evaluated the technical efficiencies of Turkish hospitals using DEA. The results of their study showed that less than 10 percent of these hospitals were efficient compared to other hospitals.

In another study, the effect of semi-constant inputs on the efficiency of the emergency department of a hospital in Montreal was investigated; they proposed modified DEA with semi-constant inputs (Ouellette & Vierstraete, 2004).

A DEA-based performance evaluation model was presented by O'Neill et al. (2008), who classified inputs and outputs in more detail and selected them with regard to the local environment. The authors used window analysis. Their method made it possible to perform HPE over multiple periods.

Yawe (2010) used the Super-Efficiency approach in DEA to analyse and rank the hospital performance and the BSC method criteria in the model to perform HPE.

Chuang et al. (2011) used a combination of DEA-artificial neural network and DEA-assurance region model to analyse hospital data and evaluate hospital performance. Then, they discussed the efficiency and inefficiency of hospitals using the regression.

Mitropoulos et al. (2015) used a combination of stochastic DEA and Bayesian analysis to calculate hospital efficiency scores in Greece. Bayesian analysis was used to generate a statistical model and create a simulation platform to analyse data of alternatives, and then calculated the efficiency scores of hospitals using the DEA model.

Rezaee and Karimdadi (2015) examined the effect of geographical location on the hospital performance. They categorised hospitals into different groups by province. Hospitals of each group were evaluated in a similar geographical environment. Then, they evaluated the performance of hospitals in different geographical locations using the multi-group DEA model.

Prakash and Annapoorni (2015) used the DEA method to calculate the technical efficiency (TE) of hospitals in the state of Tamil Nadu in India. The results of their study showed that only 29% of hospitals were efficient.

Gholami et al. (2015) used the two-stage Bootstrap DEA and two-year information of 187 hospitals in the United States to demonstrate the impact of IT investment on quality and the impact of quality on the operational efficiency of the hospitals.

Rouyendegh et al. (2016) proposed a hybrid HPE approach based on DEA and FAHR The method combined the advantages of both DEA and FAHP methods to obtain optimal weights.

Chowdhury and Zelenyuk (2016) evaluated the performance of hospital services in Ontario, Canada, using DEA with Bootstrap and regression. They estimated the efficiency through the DEA, and then calculated the distribution of efficiency across different geographic locations, educational settings, and sizes using the two-stage Bootstrap method.

Lobo et al. (2016) used the Dynamic Data Envelopment Analysis (DDEA) method to evaluate university hospital performance in different years. Determining the efficiency scores for the annual performance of hospitals and monitoring the variation of these scores were among the benefits of their method.

Khushalani and Ozcan (2017) used the Dynamic Network DEA to evaluate the performance of hospital

subunits, including surgical care, medical care and quality. The results of their study showed that the efficiency of hospitals increased during the studied period.

Kang et al. (2017) used DEA to evaluate the performance of the hospital emergency department. They concluded that many emergency departments needed to re-engineer their processes to improve their performance.

Johannessen et al. (2017) used DEA to evaluate the performance of physicians at a hospital in Norway. Their study showed that more attention should be paid to employees with multiple skills.

Chen et al. (2017) examined the impact of a recession on hospital performance; they concluded that hospital performance declined after the recession.

Haghighi and Torabi (2018) evaluated a hospital information system as one of the most important factors affecting patient satisfaction and health. They used BWM to calculate the weights of the evaluation indicators and then evaluated the performance of each unit using DEA.

Zare et al. (2018) used a combination of DEA and Game theory to measure the performance of health centres in Iran.

Omrani et al. (2018) used DEA based on clustering for DMUs under uncertain condition; they expanded the method to measure the performance of hospitals in different provinces.

In the research literature, different DEA models have been proposed for HPE, but most of these models only seek to measure hospital efficiency. Whether or not predetermined goals of hospitals have been met can be determined by measuring hospitals effectiveness. The DEA-EEP model can measure efficacy and effectiveness simultaneously and provide a more comprehensive assessment of hospitals.

In the evaluation of healthcare centres, selecting efficient inputs and outputs is crucial. In this study, inputs and outputs are selected according to the expert opinions and the review of existing literature. In the case study, which included Iranian hospitals, the authors of the article attempted to select indicators that correctly measured the hospital performance.

Proposed method

This section presents an integrated DEA approach, in which the performance of hospitals in achieving their goals is addressed in addition to the efficiency of hospitals. The framework of the proposed method is shown in Fig. 1. In the first step, the authors of the article use DEA to measure efficiency. The second step shows how DEA can be used to measure the effectiveness of DMUs. Then, the DEA-EEP model is described, which can measure efficiency, effectiveness, and productivity simultaneously. Also, the proposed DEA-EEP model can offer suggestions for improving the inefficient and ineffective units. This model seeks to improve the ability to evaluate performance by using the dual problem and adding auxiliary variables. It also shows the numbers of input losses and output deficiencies. The inputs losses can be calculated by adding an auxiliary variable; if the value of this variable approaches zero, the inefficient unit becomes efficient. Deficiency of each output can also be calculated with regard to various inputs. Also, the DEA-EEP model has an auxiliary variable that represents the deficiency in the outputs relative to the predetermined target values. In this model, because it is assumed that the target value is fixed and optimally selected, the value of the target deficiency auxiliary variable is always zero. As there is no need to improve the target value and it is used only to evaluate the effectiveness of the DMUs, the proposed method uses the principle of "constant returns to scale" to derive relationships for measuring the effectiveness. The DEA-EEP model demonstrates efficient, effective and reference units for inefficient and ineffective units so that they can increase their efficiency and effectiveness. For example, if a DMU is fully efficient and is designated by the model as a reference unit for an inefficient unit, that inefficient unit should consider this reference unit as a model for achieving efficiency.

Russell's model based on auxiliary variables was used to calculate the efficiencies of the units (Pastor et al., 1999). The linear form of this model is shown as Model (1). Table 2 shows the sets, parameters, and variables used in this study.

The DEA model in the input-oriented mode and with constant returns to scale was described in the study by Charnes et al., 1978. In this study, the dual problem was used to calculate the effectiveness, except that the problem was target-oriented rather

(1)

\begin{matrix} Min β - \frac{1}{m} \sum_{i = 1}^{m} \frac{t_{i}^{-}}{x_{i o}} \\ s t . \\ β + \frac{1}{s} \sum_{r = 1}^{s} \frac{t_{r}^{+}}{y_{r o}} = 1 \\ \sum_{j = 1}^{n} μ_{j} x_{i j} = β x_{i o} - t_{i}^{-}, i \in N_{m} \\ \sum_{j = 1}^{n} μ_{j} y_{r j} = β y_{r o} + t_{r}^{+}, r \in N_{s} \\ β, μ_{j}, t_{i}^{-} t_{r}^{+} \geq 0 \end{matrix}

$$\begin{align}& \text{Min}\,\beta -\frac{1}{m}\sum\limits_{i=1}^{m}{\frac{t_{i}^{-}}{{{x}_{io}}}} \\ & st. \\ & \beta +\frac{1}{s}\sum\limits_{r=1}^{s}{\frac{t_{r}^{+}}{{{y}_{ro}}}}=1 \\ & \sum\limits_{j=1}^{n}{{{\mu }_{j}}}{{x}_{ij}}=\beta {{x}_{io}}-t_{i}^{-},\,i\in {{\mathbb{N}}_{m}} \\ & \sum\limits_{j=1}^{n}{{{\mu }_{j}}{{y}_{rj}}}=\beta {{y}_{ro}}+t_{r}^{+},\,r\in {{\mathbb{N}}_{s}} \\ & \beta ,{{\mu }_{j}},t_{i}^{-}t_{r}^{+}\ge 0 \\ \end{align}$$

than input-oriented. Effectiveness shows how a company can meet the set targets (Tavassoli et al., 2014). In this study, effectiveness is defined as the output-to-target ratio. In the DEA-EEP model, it is assumed that the target is always ideal and unchangeable. For measuring the effectiveness, the problem is considered in a target-oriented mode, and the values of auxiliary variables for the targets are always zero. This means that there is no deficiency in any target. The effectiveness is measured using Model (2).

(2)

\begin{matrix} Min θ \\ s t . \\ \sum_{j = 1}^{n} λ_{j} g_{t} j + s_{t}^{-} = θ g_{t o}, t \in ℕ_{T} \\ \sum_{j = 1}^{n} λ_{j} y_{r j} - s_{r}^{+} = y_{r o}, r \in ℕ_{S} \\ λ_{j}, s_{t}^{-}, s_{r}^{+} \geq 0 \end{matrix}

$$\begin{align}& \text{Min}\,\theta \\ & st. \\ & \sum\limits_{j=1}^{n}{{{\lambda }_{j}}{{g}_{t}}j}+s_{t}^{-}=\theta {{g}_{to}},t\in {{\mathbb{N}}_{T}} \\ & \sum\limits_{j=1}^{n}{{{\lambda }_{j}}{{y}_{rj}}}-s_{r}^{+}={{y}_{ro}},r\in {{\mathbb{N}}_{S}} \\ & {{\lambda }_{j}},s_{t}^{-},s_{r}^{+}\ge 0 \\ \end{align}$$

By combining Models (1) and (2), the DEA-EEP model is obtained that can simultaneously measure the efficiency, effectiveness, and productivity of hospital units. The DMUo is the unit that its efficiency and effectiveness are calculated relative to other units at each run of the model. The DEA-EEP model is written as Model (3).

(3)

\begin{matrix} Min ϕ + θ \\ s t . \\ β - \frac{1}{m} \sum_{i = 1}^{m} \frac{t_{i}^{-}}{x_{i o}} - ϕ \leq 0 \\ β + \frac{1}{s} \sum_{r = 1}^{s} \frac{t_{r}^{+}}{y_{r o}} = 1 \\ \sum_{j = 1}^{n} μ_{j} x_{i j} = β x_{i o} - t_{t}^{-}, i \in N_{m} \\ \sum_{j = 1}^{n} μ_{j} y_{r j} = β y_{r o} + t_{r}^{+}, r \in N_{S} \\ \sum_{j = 1}^{n} λ_{j} g_{t j} + s_{t}^{-} = θ g_{t o}, t \in N_{T} \\ \sum_{j = 1}^{n} λ_{j} y_{r j} - s_{r}^{+} = y_{r o}, r \in N_{S} \\ β, μ_{j}, t_{i}^{-}, t_{r}^{+}, λ_{j}, s_{t}^{-}, s_{r}^{+} \geq 0 \end{matrix}

$$\begin{align}& \text{Min}\,\phi +\theta \\ & st. \\ & \beta -\frac{1}{m}\sum\limits_{i=1}^{m}{\frac{t_{i}^{-}}{{{x}_{io}}}-\phi \le 0} \\ & \beta +\frac{1}{s}\sum\limits_{r=1}^{s}{\frac{t_{r}^{+}}{{{y}_{ro}}}=1} \\ & \sum\limits_{j=1}^{n}{{{\mu }_{j}}{{x}_{ij}}}=\beta {{x}_{io}}-t_{t}^{-},i\in {{\mathbb{N}}_{m}} \\ & \sum\limits_{j=1}^{n}{{{\mu }_{j}}{{y}_{rj}}}=\beta {{y}_{ro}}+t_{r}^{+},\,r\in {{\mathbb{N}}_{S}} \\ & \sum\limits_{j=1}^{n}{{{\lambda }_{j}}{{g}_{tj}}+s_{t}^{-}}=\theta {{g}_{to}},\,t\in {{\mathbb{N}}_{T}} \\ & \sum\limits_{j=1}^{n}{{{\lambda }_{j}}{{y}_{rj}}}-s_{r}^{+}={{y}_{ro}},\,r\in {{\mathbb{N}}_{S}} \\ & \beta ,{{\mu }_{j}},t_{i}^{-},t_{r}^{+},{{\lambda }_{j}},s_{t}^{-},s_{r}^{+}\ge 0 \\ \end{align}$$

The DEA-EEP model is solved once for each hospital per each year. The value of productivity for each hospital unit is in the range of 0 to 2.

Suppose that $(ϕ^{*}, θ^{*}, t_{i}^{-}, t_{r}^{+}, s_{t}^{-}, s_{t}^{+}, μ_{j}^{*}, λ_{j}^{*})$ $\left( {{\phi }^{*}},{{\text{ }\!\!\theta\!\!\text{ }}^{\text{*}}},\text{t}_{\text{i}}^{-},\text{t}_{\text{r}}^{+},\text{s}_{\text{t}}^{-},\text{s}_{\text{t}}^{+},{{\text{ }\!\!\mu\!\!\text{ }}_{\text{j}}}^{*},{{\text{ }\!\!\lambda\!\!\text{ }}_{\text{j}}}^{*} \right)$is the optimal solution of model (3) for DMUo, then the following situations can exist:

If (Φ* = 1 and we have $t_{i}^{-^{*}} = 0 and t_{r}^{+^{*}} = 0$ $t_{i}^{{{-}^{*}}}=0\,\,\text{and}\,\text{t}_{r}^{{{+}^{*}}}=0$in the optimal solution, then the DMU_O is "strongly

efficient". If (Φ* = 1 and we have $t_{i}^{-^{*}} \neq 0 and t_{r}^{+^{*}} \neq 0$ $t_{i}^{{{-}^{*}}}\ne 0\,\text{and}\,t_{r}^{{{+}^{*}}}\ne 0$in the optimal solution, then the DMU_O is "weakly efficient".

If Φ* < 1, then the DMU_o is inefficient.

If θ* = 1 and we have $s_{t}^{-^{*}} = 0 and s_{r}^{+^{*}} = 0$ $s_{t}^{{{-}^{*}}}=0\,\text{and}\,s_{r}^{{{+}^{*}}}=0$in the optimal solution, then the DMU_O is "strongly

effective". If θ* = 1 and we have $s_{t}^{-^{*}} \neq 0 and s_{r}^{+^{*}} \neq 0$ $s_{t}^{{{-}^{*}}}\ne 0\,\,\text{and}\,s_{r}^{{{+}^{*}}}\ne 0$in the optimal solution, then the DMU_O is "weakly effective".

If θ* < 1 then the DMUis "ineffective".

If ϕ* = 1 and we have $t_{i}^{-^{*}} = 0 and t_{r}^{+^{*}} = 0$ $t_{i}^{{{-}^{*}}}=0\,\text{and}\,t_{r}^{{{+}^{*}}}=0$in the optimal solution, and If θ* = 1 and we have $s_{t}^{-^{*}} = 0 and s_{r}^{+^{*}} = 0$ $s_{t}^{{{-}^{*}}}=0\,\text{and}\,s_{r}^{{{+}^{*}}}=0$in the optimal solution, then the DMUo is "strongly productive"; otherwise the DMUo is "weakly productive" or it "unproductive".

Research results

In this Section, several Iranian hospitals are evaluated in a real case study using the DEA-EEP model. To make DMUS more homogeneous, 11 hospitals were selected from the same geographical region (Tehran). The hospital names are shown in Table 3. All selected hospitals are under the supervision of the Tehran University of Medical Sciences. In the evaluation of healthcare centres, selecting efficient inputs and outputs is crucial. In this study, inputs and outputs are selected according to expert opinions and the review of existing literature. In the case study, which included Iranian hospitals, the authors of the article attempted to select indicators that correctly measured the hospital performance.

Tab. 3

Names of evaluated hospitals

HOSPITALS	DMU_s
IMAM KHOMEINI HOSPITAL	DMU₁
AMIRALAM HOSPITAL	DMU₂
BAHARLOO HOSPITAL	DMU₃
BAHRAMI HOSPITAL	DMU₄
ARASH HOSPITAL	DMU₅
SHARIATI HOSPITAL	DMU₆
RAZI HOSPITAL	DMU₇
ROOZBEH HOSPITAL	DMU₈
SINA HOSPITAL	DMU₉
ZIYAIYAN HOSPITAL	DMU₁₀
CHILDREN'S MEDICAL CENTRE	DMU₁₁

In this study, the inputs are the number of health workers, the number of other staff, and the number of patient beds; and the outputs are the bed occupancy rate and the bed turnover rate. These indicators are described in Tab. 4.

To measure the hospital performance more comprehensively, 3-year data of hospitals are used. In this study, data on inputs and outputs, including statistical data of the selected hospitals, were obtained from the website of the Statistics Centre of Tehran University of Medical Sciences for the period of 2016-2018. Then, target values were set for the outputs. Selecting the appropriate targets is essential because one of the purposes of this study is to provide a method for improving the performance of health centres. A study demonstrated that the ideal value for the bed occupancy rate was 85%; clinical observations showed that when the bed occupancy rate was greater than 85%, safety and effectiveness of care tasks could be compromised (Keegan, 2010). On the other hand, when the bed occupancy rate is below 85%, it indicates that the resources are used inefficiently. In this study, the ideal bed occupancy rate is set at 85% and this value is considered as the target value for the bed occupancy rate in the DEA-EEP model for all DMUs.

Since the target values of the bed turnover rate are different in specialised and general hospitals, this study does not consider the same bed turnover rate for hospitals, and the ideal value of this indicator for each hospital is determined by experts. Table 5 shows the statistical data of the inputs and outputs as well as the target values of hospitals for 2016. Also, Table 6 shows the data for 2017, and Table 7 — for 2018.

Tab. 4

Input and output indicators according to the research literature and expert opinion

	CRITERIA	DESCRIPTION	AUTHOR(S)
Input	Number of health workers (C₁)	Number of health workers working in the hospital including doctors, nurses and ...	Liao et al. (2019)
	Number of other staff (C₂)	Number of other staff working in hospital sub-units including Food and Drug Staff, Management, Service ...	Chowdhury and Zelenyuk (2016)
	Number of patient beds (C₃)	Numbers of beds specially designed for hospitalised patients or others in need of some form of health care.	Wang et al. (2016)
Output	Bed occupancy rate (C₄)	Calculated as the number of hospital bed days divided by the number of available hospital beds, multiplied by the number of days in a year.	Zhijun et al. (2014)
Output	Bed turnover rate (C₅)	Number of times that hospitalised patients used a hospital bed over a one-year period	Zhijun et al. (2014)

Tab. 5

Statistical data of hospitals for 2016

HOSPITALS	INPUTS			OUTPUTS		GOALS
HOSPITALS	X₁	X₂	X₃	Y₁	Y₂	G₁	G₂
DMU₁	2172	1128	1170	91.02	49.85	85	46
DMU₂	401	232	240	72.83	67.97	85	61
DMU₃	501	156	304	90.05	82.52	85	67
DMU₄	231	110	149	83.34	67.62	85	62
DMU₅	310	118	145	90.60	125.94	85	74
DMU₆	972	563	550	91.36	51.27	85	52
DMU₇	133	90	69	51.32	59.86	85	58
DMU₈	263	154	202	94.26	13.24	85	23
DMU₉	742	398	475	89.58	58.17	85	69
DMU₁₀	294	159	169	86.03	98.30	85	94
DMU₁₁	651	318	370	84.66	53.87	85	68

Source: elaborated by the authors based on (http://sitl.tums.ac.ir,05.02.2019)

Tab. 6

Statistical data of hospitals for 2017

HOSPITALS	INPUTS			OUTPUTS		GOALS
HOSPITALS	$X_{1}$ ${{X}_{1}}$	$X_{2}$ ${{X}_{2}}$	$X_{3}$ ${{X}_{3}}$	$Y_{1}$ ${{Y}_{1}}$	$Y_{2}$ ${{Y}_{2}}$	$G_{1}$ ${{G}_{1}}$	$G_{2}$ ${{G}_{2}}$
DMU₁	2269	1106	1182	91.43	51.89	85	46
DMU₂	416	247	257	73.61	76.92	85	61
DMU₃	539	154	333	81.98	76.02	85	67
DMU₄	239	115	145	78.46	71.90	85	62
DMU₅	339	119	154	96.95	128.19	85	74
DMU₆	1098	603	561	87.48	48.76	85	52
DMU₇	149	90	69	51.94	67.38	85	58
DMU₈	264	185	201	98.41	13.59	85	23
DMU₉	808	418	515	87.76	57.59	85	69
DMU₁₀	296	160	170	82.59	94.83	85	94
DMU₁₁	711	344	393	93.46	76.25	85	68

Source: elaborated by the authors based on (http://sitl.tums.ac.ir,05.02.2019)

Tab. 7

Statistical data of hospitals for 2018

HOSPITALS	INPUTS			OUTPUTS		GOALS
HOSPITALS	$X_{1}$ ${{X}_{1}}$	$X_{2}$ ${{X}_{2}}$	$X_{3}$ ${{X}_{3}}$	$Y_{1}$ ${{Y}_{1}}$	$Y_{2}$ ${{Y}_{2}}$	$G_{1}$ ${{G}_{1}}$	$G_{2}$ ${{G}_{2}}$
DMU₁	2274	1118	1233	88.58	51.57	85	46
DMU₂	421	251	247	73.78	80.64	85	61
DMU₃	542	156	365	76.26	75.31	85	67
DMU₄	240	118	161	71.90	67.77	85	62
DMU₅	347	126	148	88.90	117.13	85	74
DMU₆	1108	607	551	84.07	45.41	85	52
DMU₇	151	94	69	48.26	71.88	85	58
DMU₈	269	189	194	93.29	14.17	85	23
DMU₉	814	424	526	83.15	59.56	85	69
DMU₁₀	298	175	169	74.32	82.32	85	94
DMU₁₁	724	358	408	89.68	66.48	85	68

Source: elaborated by the authors based on (http://sitl.tums.ac.ir,05.02.2019)

Using the data on inputs and outputs and Model(3), the DEA-EEP model was coded in GAMS software version 24.5.4 and the efficiency, effectiveness, and productivity of each hospital were calculated.Table 8 shows the efficiency, effectiveness, and productivity of hospitals for 2016. Also, Table 9 and Table 10 show these values for 2017 and 2018, respectively. For example, the results of the DEA-EEP model for DMU3 for 2016 are as follow:

The values of the efficiency, effectiveness, and productivity for DMU3 (which is the Baharloo Hospital) are 0.488, 0.979, and 1.467, respectively. μ₅ represents the DMU3 efficiency reference unit. λ₅ and λ₈ represent the DMU3 effectiveness reference unit.

DMU3 should consider DMU5 as a reference unit for the efficiency, and DMU5 and DMU8 Units as reference units for the effectiveness.

Tab. 8

Efficiency, Effectiveness and Productivity scores of hospitals in 2016

HOSPITALS	EFFICIENCY	EFFECTIVENESS	PRODUCTIVITY	μ_j	$t_{1}^{-}$ $t_{1}^{-}$	$t_{2}^{-}$ $t_{2}^{-}$	$t_{3}^{-}$ $t_{3}^{-}$	$t_{1}^{+}$ $t_{1}^{+}$	$t_{2}^{+}$ $t_{2}^{+}$	λ_j	$s_{1}^{-}$ $s_{1}^{-}$	$s_{2}^{-}$ $s_{2}^{-}$	$s_{1}^{+}$ $s_{1}^{+}$	$s_{2}^{+}$ $s_{2}^{+}$
DMU₁	0.070	0.978	1.049	μ₅ = 0.568	0	24.41	12.48	44.07	67.84	λ₅ = 0.327 λ₈= 0.651	0	5.80	0	0
DMU₂	0.406	0.792	1.198	μ₅ = 0.646	0	39.62	26.17	22.15	47.39	λ₅ = 0.510 λ₈= 0.282	0	4.10	0	0
DMU₃	0.488	0.979	1.467	μ₅ = 0.790	54.46	0	67.09	17.75	50.16	λ₅ = 0.617 λ₈= 0.362	0	11.61	0	0
DMU₄	1	0.903	1.903	μ₄ = 1	0	0	0	0	0	λ₅ = 0.494 λ₈= 0.409	0	10.04	0	0
DMU₅	1	1	2	μ₅ = 1	0	0	0	0	0	λ₅ = 1	0	0	0	0
DMU₆	0.153	0.982	1.136	μ₅ = 0.580	0	35.70	17.64	35.65	63.56	λ₅ = 0.340 λ₈= 0.643	0	11.17	0	0
DMU₇	1	0.632	1.632	μ₇ = 1	0	0	0	0	0	λ₅ = 0.465 λ₈= 0.097	5.90	0	0	0
DMU₈	0.834	1	1.185	μ₄ = 0.008μ₅ = 0.050μ₇ = 0.281	0	0	14.23	0	20.95	λ₈ = l	0	0	0	0
DMU₉	0.214	0.966	1.180	μ₅ = 0.630	0	30.39	33.65	33.48	63.99	λ₅ = 0.403 λ₈= 0.563	0	23.89	0	0
DMU₁₀	0.758	0.942	1.701	μ₅ = 0.050μ₇= 0.281	0	42.20	28.36	0	19.12	λ₅ = 0.403 λ₈= 0.563	0	28.06	0	0
DMU₁₁	0.242	0.913	1.155	μ₅ = 0.587	0	19.61	18.30	29.51	58.85	λ₅ = 0.371 λ₈= 0.542	0	22.15	0	0
Mean	0.501	0.917	1.478	--	--	--	--	--	--	--	--	--	--	--

$t_{1}^{-}$ $t_{1}^{-}$represents the first input surplus, which equals 54.46;

$t_{2}^{-}$ $t_{2}^{-}$represents the second input surplus, which equals 0;

$t_{3}^{-}$ $t_{3}^{-}$represents the third input surplus, which equals 67.09;

$t_{1}^{+}$ $t_{1}^{+}$represents the first output deficiency, which equals 17.75;

$t_{2}^{+}$ $t_{2}^{+}$represents the second output deficiency, which equals 50.16;

$s_{1}^{-}$ $s_{1}^{-}$represents the first output deficiency, which equals 50.16;

$s_{2}^{-}$ $s_{2}^{-}$represents the second output deficiency, which equals 11.61;

$s_{1}^{+} and s_{2}^{+}$ $s_{1}^{+}\,\text{and}\,s_{2}^{+}$are deficiencies of the first and second output, respectively, and their values are always zero in this model because it is assumed that the target value is already set to its desired value and cannot be changed;

$t_{i}^{-} and t_{r}^{+}$ $t_{i}^{-}\,\,\text{and}\,t_{r}^{+}$are auxiliary variables related to the efficiency and $s_{t}^{-} and s_{r}^{+}$ $s_{t}^{-}\,\text{and}\,s_{r}^{+}$are auxiliary variables related to the effectiveness.

The results of the efficiency evaluation for the period 2016-2018 showed that the efficiency score of Roozbeh Hospital was 0.834 in 2016, but this hospital was fully efficient for remaining years. The efficiency of Ziyaiyan Hospital decreased dramatically over the period. One of the most important issues is the improvement of the efficiency of public health care institutions by changing their legal structure (Lachowska, 2017). Obviously, hospital managers should look for plans to offset this decline. Arash Hospital, Razi Hospital and Bahrami Hospital were also fully efficient during the period. The rest of the hospitals were not fully efficient and had no major changes in their efficiency scores. The efficiencies of these hospitals were fluctuating during this period, and they should look for plans to improve their efficiencies. In addition, the average efficiency of the hospitals during the period was constantly fluctuating.

The results of the effectiveness evaluation for the period 2016-2018 showed that Arash Hospital and Roozbeh Hospital had full effectiveness scores. The effectiveness scores of the rest of the hospitals did not follow a specific trend during the period. Also, the average effectiveness of the hospitals was fluctuating from 2016 to 2018. This shows that the effectiveness of most hospitals did not improve in this period and even declined to some extent.

The results of the productivity evaluation for the period 2016-2018 showed that only Arash Hospital was fully productive. Also, given that the productivity score is the sum of the efficiency score and the effectiveness score, variations of these scores affect the productivity score. Average productivity scores of the hospitals were fluctuating from 2016 to 2018. These results show that the hospitals should seek plans to improve their productivity.

Fig. 2 shows the efficiency scores of the hospitals using the DEA-EEP model. Bahrami Hospital, Razi Hospital and Arash Hospital were fully efficient during the period. Fig. 3 shows the effectiveness scores of the hospitals. Roozbeh Hospital and Arash Hospital were fully effective during the period. Also, Fig. 4 shows the productivity scores of the hospitals. Arash Hospital was fully productive during the period.

Efficiency scores of hospitals in the period 2016 to 2018

Tab. 9

Efficiency, Effectiveness and Productivity scores of hospitals in 2017

HOSPITALS	EFFICIENCY	EFFECTIVENESS	PRODUCTIVITY	μ_j	$t_{1}^{-}$ $t_{1}^{-}$	$t_{2}^{-}$ $t_{2}^{-}$	$t_{3}^{-}$ $t_{3}^{-}$	$t_{1}^{+}$ $t_{1}^{+}$	$t_{2}^{+}$ $t_{2}^{+}$	λ_j	$s_{1}^{-}$ $s_{1}^{-}$	$s_{2}^{-}$ $s_{2}^{-}$	$s_{1}^{+}$ $s_{1}^{+}$	$s_{2}^{+}$ $s_{2}^{+}$
DMU₁	0.073	0.934	1.007	μ₅ = 0.566	0	26.19	12.80	47.17	68.22	λ₅ = 0.342 λ₈= 0.592	0	4.04	0	0
DMU₂	0.418	0.771	1.189	μ₅ = 1.265	3.26	0	31.16	31.76	49.76	λ₅ = 0.581 λ₈= 0.175	1.26	0	0	0
DMU₃	0.433	0.841	1.275	μ₅ = 0.697	54.02	0	72.02	23.42	48.41	λ₅ = 0.564 λ₈= 0.278	0	8.28	0	0
DMU₄	1	0.805	1.805	μ₄= 1	0	0	0	0	0	λ₅ = 0.532 λ₈= 0.273	0	4.27	0	0
DMU₅	1	1	2	μ₅= l	0	0	0	0	0	λ₅= 1	0	0	0	0
DMU₆	0.138	0.894	1.031	μ₇ = 1	15.06	0	14.91	39.36	60.84	λ₅ = 0.320 λ₈= 0.574	0	9.62	0	0
DMU₇	1	0.674	1.674	μ₇ = 1	0	0	0	0	0	λ₅ = 0.524 λ₈= 0.011	11.72	0	0	0
DMU₈	1	1	2	μ₈ = 1	0	0	0	0	0	λ₈= 1	0	0	0	0
DMU₉	0.201	0.898	1.098	μ₅ = 0.600	0	33.85	37.27	36.10	62.46	λ₅ = 0.396 λ₈= 0.502	0	21.09	0	0
DMU₁₀	0.733	0.850	1.583	μ₇ = 1.493	26.13	0	39.75	8.18	20.96	λ₅ = 0.727 λ₈= 0.123	0	23.29	0	0
DMU₁₁	0.298	0.958	1.256	μ₅ = 0.736	0	33.11	24.55	38.54	67.56	λ₅ = 0.552 λ₈= 0.406	0	14.96	0	0
Mean	0.572	0.875	1.447	--	--	--	--	--	--	--	--	--	--	--

Tab. 10

Efficiency, Effectiveness and Productivity scores of hospitals in 2018

HOSPITALS	EFFICIENCY	EFFECTIVENESS	PRODUCTIVITY	μ_j	$t_{1}^{-}$ $t_{1}^{-}$	$t_{2}^{-}$ $t_{2}^{-}$	$t_{3}^{-}$ $t_{3}^{-}$	$t_{1}^{+}$ $t_{1}^{+}$	$t_{2}^{+}$ $t_{2}^{+}$	λ_j	$s_{1}^{-}$ $s_{1}^{-}$	$s_{2}^{-}$ $s_{2}^{-}$	$s_{1}^{+}$ $s_{1}^{+}$	$s_{2}^{+}$ $s_{2}^{+}$
DMU₁	0.071	0.967	1.038	μ₇ = 1.032	41.46	0	35.76	42.10	69.68	λ₅ = 0.368 λ₈= 0.599	0	3.47	0	0
DMU₂	0.437	0.870	1.307	μ₇ = 1.294	8.62	0	30.41	26.69	53.93	λ₅ = 0.670 λ₈= 0.152	4.07	0	0	0
DMU₃	0.449	0.846	1.296	μ₇ = 1.260	221.24		190.18	2.90	33.39	λ₅ = 0.615 λ₈= 0.231	0	5.87	0	0
DMU₄	1	0.797	1.797	μ₄ = 1	0	0	0	0	0	λ₅ = 0.549 λ₈= 0.248	0	3.08	0	0
DMU₅	1	1	2	μ₅ = 1	0	0	0	0	0	λ₅ = 1	0	0	0	0
DMU₆	0.129	0.916	1.045	μ₇ = 0.927	19.08	0	15.14	32.67	60.12	λ₅ = 0.315 λ₈ = 0.601	0	10.50	0	0
DMU₇	1	0.783	1.783	μ₇ = 1	0	0	0	0	0	λ₅ = 0.614	14.39	6.29	0	0
DMU₈	1	1	2	μ₈ = 1	0	0	0	0	0	λ₈ = l	0	0	0	0
DMU₉	0.201	0.913	1.113	μ₇ = 1.119	32.96	0	53.28	33.37	65.66	λ₅ = 0.453 λ₈= 0.460	0	18.88	0	0
DMU₁₀	0.636	0.829	1.465	μ₇ = 1.314	11.91	0	28.60	10.95	36.33	λ₅ = 0.685 λ₈= 0.143	0	23.89	0	0
DMU₁₁	0.264	0.985	1.294	μ₇ = 1.235	48.29	0	47.09	30.52	67.21	λ₅ = 0.510 λ₈= 0.475	0	18.32	0	0
Mean	0.562	0.900	1.467	--	--	--	--	--	--	--	--	--	--	--

Effectiveness scores of hospitals in the period 2016 to 2018

Productivity scores of hospitals in the period 2016 to 2018

Discussion

In real-world issues, measuring effectiveness is as important as measuring efficiency. Whenever we talk about effectiveness, we seek to identify the relationship between activities and goals and programmes; when we can better achieve the goals of programmes, we are more effective. So, making a distinction between effectiveness and efficiency is important in performance evaluation. Efficiency measures the ratio of consumed inputs to outputs of a decision-making unit, but effectiveness shows how effective a decision-making unit is in achieving its predetermined goals. In this study, the authors of the article defined effectiveness as the output-to-target ratio. The measurement of the performance of a system based on efficiency and effectiveness can help managers evaluate it in achieving the goals of the system and community. This can facilitate the process of decision-making.

This study has proposed a new model to calculate efficiency, effectiveness and efficiency simultaneously. The model was used to evaluate several hospitals in Tehran in the period 2016-2018. The auxiliary variables of this model were given suggestions for the improvement of the performance of inefficient and ineffective hospitals.

Although some hospitals experienced an upward trend in the period, the efficiency, effectiveness, and productivity scores of most hospitals fluctuated and did not have a growing trend. This indicates that although most hospitals sought to improve the quality of their services, they needed to take more serious steps. Hospitals can review their 2016-2018 performance scores and identify the causes of their strengths and weaknesses; they can use these data to modify inputs and outputs order to improve their future performance. The improvement of staff skills through organisational training, the evaluation of staff and physicians, the review of hospital strategies, and the utilisation of new approaches and equipment in healthcare systems are among the measures that can enhance hospital performance. Therefore, managers and policymakers of the health system in Iran should provide an appropriate context to change the existing situation to improve the efficiency and effectiveness of hospital services.

Conclusions

This paper presented a novel DEA model that can measure the efficiency, effectiveness, and productivity of DMUs simultaneously. The DEA-EEP model can also improve the performance of DMUs by providing a solution to reduce inputs or increase outputs to a certain extent. This model seeks to improve the performance evaluation by using the dual problem with an auxiliary variables-based measure.

The performance of 11 hospitals in Tehran was evaluated using the proposed model. Decision-makers can use the proposed framework to optimally allocate resources, identify reference units, and identify existing strengths and weaknesses of DMUs. In this paper, first, the factors affecting the efficiency and effectiveness of Iranian hospitals were identified based on literature review and opinions of health and treatment experts. Then, data of selected indicators were collected for 11 hospitals.

The results showed that Razi Hospital and Arash Hospital had the highest efficiency, Roozbeh Hospital and Arash Hospital had the most effectiveness, and Arash Hospital had the highest productivity in the studied 3-year period. This paper showed that the DEA-EEP model could be used to rank hospitals in healthcare systems. The results showed that the average of the hospital productivity scores fluctuated in the three-year period and did not follow a specific trend. This indicates that these hospitals need to improve their performance.

Some of our suggestions for future research include: 1) to use one of the MCDM methods for weighting the inputs, outputs, and targets in the DEA-EEP model; 2) to expand the DEA-EEP model using the Network-DEA approach to measure the performance of hospital subunits; 3) to use the DEA-EEP model to evaluate other organisations and activities, such as sustainable or flexible suppliers; and 4) to identify and use other indicators not used in this study for evaluating healthcare centre performance.

eISSN:: 2543-912X
Language:: English

Publication timeframe:: 4 times per year
Journal Subjects:: Business and Economics, Business Management, Industries, Transportation, Logistics, Air Traffic, Shipping, Engineering, Mechanical Engineering, Production Technology, Process Engineering and Industrial Engineering

Journal RSS Feed

A novel DEA model for hospital performance evaluation based on the measurement of efficiency, effectiveness, and productivity

Published Online: May 10, 2020

Page range: 7 - 19

Received: Sep 15, 2019

Accepted: Jan 15, 2020

DOI: https://doi.org/10.2478/emj-2020-0001

© 2020 Mohammad Ghahremanloo, Aliakbar Hasani, Maghsoud Amiri, Mohammad Hashemi-Tabatabaei, Mehdi Keshavarz-Ghorabaee, Leonas Ustinovičius published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Fig. 1

Fig. 2

Fig. 3

Fig. 4