Examination of published articles with respect to statistical errors in veterinary sciences

Abstract The aim of this present work was to examine statistical errors in published veterinary science articles. A total of 204 published articles (SCI or SCI-Exp) were used in this study. The articles were chosen from among those indexed in PubMed database between the years 2010 and 2014, inclusive. A total of 199 articles had at least one statistical error. The most frequently encountered statistical error among the articles published in journals indexed in SCI and in SCI-E was “errors in summarizing data”. No statistical error was found in 2.45% (n=5) of 204 (SCI: 0.98% (1/102), SCI-E: 3.92% (4/102)) articles. To reduce and prevent statistical errors in publications, the researchers must have a basic knowledge of statistics and during the study process they must consult field experts. While reviewing, the reviewers have to redirect the publications to statistical editors when needed and most importantly during the process of editing, the editors have to direct the publications to a statistics reviewer.


INTRODUCTION
The importance of biostatistics is well acknowledged in veterinary and medical sciences. With the advancement of tools for gaining knowledge, we have access to data with larger complexity and information. To understand the structures of the acquired knowledge, the data have to be analyzed, and this analysis can only be performed using statistical tools. Unfortunately, in the process of employing statistical methods in scientifi c research, inappropriate applications can be encountered. When statistical errors are made, both scientists and users of scientifi c fi ndings are exposed to negative consequences [1,2]. Furthermore, statistical errors found in published articles are likely to cause the author's loss of academic credibility.
While errors can be application-related, they can also occur at different stages of the study such as planning, implementation, analysis, interpretation, and presentation, all of which are related to statistical topics [1,3,4]. Statistical errors in the study process can be categorized as the ones that occur (i) In the research process (before reporting) and (ii) in presentations or publications [1]. Some of the statistical errors in publications can be assessed, while others cannot. In publications, not all of the statistical quantities can be checked; it is only possible to check the accuracy of some of the statistics via reported descriptive values. In addition, some terminology, presentation and interpretation errors can be identifi ed [1].
Although the authors are responsible for the errors made in publications, the journal editors are also responsible for the academic prestige of their journals. The publication of studies with statistical errors will cause the journals a loss of academic credibility. Therefore, no editor would like to publish studies that involve erroneous statistical applications in their journals. Also in order to avoid statistical errors in publications, the publications are supposed to be submitted to biostatistics reviewers in the evaluation process of the publications.
The aim of the present work was to examine the statistical errors in the published articles of veterinary sciences. Published articles in journals that are indexed in Science Citation Index (SCI) were compared to Science Citation Index-Expanded (SCI-E) journals.
One hundred-two articles published in SCI journals and 102 articles published in SCI-E journals were included in this study. The articles were chosen from among those indexed in PubMed database between the years 2010 and 2014 inclusive. Not more than 25 studies were taken from each index group and publication year. Relevant articles included in the evaluation were taken from veterinary science and in SCI or SCI-E. During the investigations, the studies outside the scope of our study (with no statistical analysis and collection studies, such as case reports) were not included in the study.
The reference list of a randomly selected article was used for randomization in article selection. The fi rst article that was ranked as fi rst in the reference list with respect to the author name in the relevant years was selected, and then this process was repeated for the fi rst authors of other articles in the reference list. After the last article in the reference list was used for selection, by going back to the beginning of the reference list the second authors' name was employed as the key word for selection. The names of authors were entered into the search engines of these databases. Randomization was accomplished by repeating the process in article selection. Sample size was considered as approximately equal according to the year. The frequencies and percentages of the examined published articles by years are given in Table 1. In this study, the selected articles were examined by allocating articles among research team members with respect to the type of statistical errors. The examined statistical errors were classifi ed following the description as described by Ercan et al [4,10] and Ercan and Demirtas [1]. Of note, errors assessed by each researcher were confi rmed by all members of the research team. Therefore, there is no difference between researchers according to specifying the error and they are in full (100%) agreement. On this basis, there was no need to calculate inter-rater reliability.
The statistical errors were examined as: "p-values given in a closed form" (e.g., p<0.01, p<0.05, p>0.05), "non-reported p-values", "incorrect p-values (which are related to frequency tables)", "incorrect demonstration of p-values (e.g., p=0.000, p<0.0005 etc.)", "undefi ned statistical test", "incorrect name of a statistical test", "statistical technique defi ned but not used", "use of an incorrect test", "statistical analysis required but not performed", "errors in summarizing data" (it contains incorrect reporting regarding analyses, e.g., reporting mean and standard deviation when nonparametric tests are applied, it contains incorrect or inadequate reporting of descriptive statistics, e.g., not reporting measure of variability with arithmetic mean, errors in percentages, incorrect presentation in table format, etc.), "mathematical demonstration errors (e.g., lacking demonstration of decimals, using ":" rather than "=")", "statistical symbol errors (e.g., using π for a Chi-square value)", "incomprehensible statistical terms", "inappropriate interpretation", "errors in (statistical) terminology", and "presentation of statistical method analysis and results in the incorrect section of the manuscript" [1,4,10].
The percentage of statistical errors was calculated, taking into account the number of articles reviewed. Further, the potential difference between the statistical errors seen in articles indexed in SCI and in SCI-E journals was investigated using the Chi-square test and Fisher's exact test. The results of the study were presented as counts and their corresponding percentage values. Data were analyzed with SPSS software 21.0.

RESULTS
In this study, 204 articles, which included 102 SCI indexed and 102 SCI-E articles were reviewed with regards respect to statistical errors. A total of 199 articles were found with at least one statistical error. The most frequently encountered statistical error was "errors in summarizing data" for articles published in journals indexed as SCI and as SCI-E. No statistical errors were found in 2.45% (n=5) of 204 (SCI: 0.98% (1/102), SCI-E: 3.92% (4/102)) articles. Table 2 gives a detailed account of the distribution of statistical errors among the articles. Also Table 3 gives a detailed account of the distribution of statistical errors in similar studies in medical sciences.

DISCUSSION
In this study were identifi ed statistical errors in published articles in the fi eld of veterinary sciences. In our literature survey we observed a number of studies conducted in the fi eld of medicine in order to identify such statistical errors, but no such studies are carried out in the fi eld of veterinary medicine. While published scientifi c studies are being used as reference by scientists, the fi ndings and decisions at the end of the studies are important for the people that will benefi t from it. For this reason, the accuracy and reliability of the publications is very important. Statistics is one of the most important factors for the accuracy and reliability of a publication which starts from the fi rst stage of the study (planning stage) and follows up to the last stage of the study (reporting stage). Therefore, statistics is the most basic element that makes a study scientifi c or otherwise. For this reason, in this study we examined publications in the fi eld of veterinary science in terms of statistical errors. Errors in the application of statistical methods in publications generally can be grouped under three main categories: (i) errors related to p-value, (ii) errors related to tests and (ii) other statistical errors.
When publications in the fi eld of veterinary science were examined, errors related to p-values seem to be relatively high. This source of error is very important considering the importance of the p-value. The most common error related to p-value is giving the p-value in a closed form. Some researchers may not be able to perceive it as an error; but not giving the p-value in the open form may look as depriving the reader from getting access to the actual information obtained from the result of the applied statistical test [11]. For example, while there is a signifi cant difference between p=0.061 and p=0.984, when this p-value given as p>0.05 in such case it means the information is not transferred to the reader. It will also be of great importance if the p-value is given in the open form, so that during the evaluation process the reviewers can control some of the statistical tests prior to the publication of the fi ndings. In this study, 44.12% of the publications (SCI 47.06% and SCI-E 41.18%) were found to have non-reported p-values, while in the study of Ercan et al. in medical journals, the rate was 22.12% [10]. Some authors gave place for statistical interpretations in their studies without giving the p-values. In this situation it cast doubt on accuracy of the statistical tests and also makes it look like the author is depriving the reader from the information of the p-value.
In the publications we reviewed, the major source of error with the quality that can affect the result directly is the wrong giving of the p-values. In this study 8.82% (SCI 8.82% and SCI-E 8.82%) of the publications examined were found to be with mispresentations of the p-value, while in the study of Ercan et al. in medical journals the rate was 13.36% [10]. Specifi ed rate of incorrect p-values, are the rate of the result of tests reviewed with possible means of control. This rate should be considered higher because some of the statistical tests can't be controlled.
Another source of error among the errors related to p-values is the incorrect presentation of the p-value. In 37.25% (SCI 45.10% and SCI-E 29.41%) of the publications reviewed the p-value was incorrectly presented. While in the study of Ercan et al in medical journals found the rate as low as 18.43% [10]. Incorrect demonstration of the p-value is leading the reader not to understand and also to lose confi dence in the study.
Another error source specifi ed in this study is related to statistical tests. Undefi ned statistical test with the rate of 15.69% (SCI 10.78% and SCI-E 20.59%) is the most common error related to statistical tests in publications. In the study of Ercan et al in medical journals this rate was 11.52%, while in study of Hanif and Ajmal 80 research articles published in indexed and recognized local journals in Pakistan the rate was 26.25% [10,12]. Not defi ning the statistical test performed is denying the evaluation of the study in the review process.
In our study, we detected 9.31% of the publications investigated with a given incorrect name of the statistical test. In study of Ercan et al. in medical journals this rate was 3.23%, while in the study of Hanif and Ajmal this rate was 12.50% [10,12].
In this study 3.43% (SCI 3.92% and SCI-E 2.94%) of the investigated publications were found with an unused statistical technique defi ned. In the study of Ercan et al in medical journal this rate was 2.30%, while in the study of Hanif and Ajmal this rate was 21.25% [10,12].
One of the major errors that can affect the results of the study is the use of incorrect statistical test. In this study, 10.78% (SCI 10.78% and SCI-E 10.78%) of the investigated publications were identifi ed with inadequate statistical tests. In the study of Ercan et al. in medical journal this rate was 7.83%, while in the study of Hanif and Ajmal this rate was 28.75% [10,12].
Sometimes the authors make a subjective interpretation without performing the necessary statistical analysis. Scientifi c conclusions can be reached only by performing statistical tests. If the researcher offered a subjective interpretations such as: different, much, effective etc. without performing the necessary test, it has no scientifi c validity and in this study this error rate was detected in 1.96% publications. Ercan et al. reported this rate to be 17.51% in medical journals [10].
When examining the studies in terms of other statistical errors in publications the most frequent error was identifi ed at the rate of 57.84%, which is the error in summarizing data. In the study of Ercan et al in medical journals they categorized the error types into three groups: thus; errors in summarizing data was identifi ed in 28.11%, insuffi cient data presented for the statistical test was identifi ed in 17.51% and incorrect and insuffi cient demonstration of descriptive statistics also identifi ed in 26.73% [10]. A study of Hanif and Ajmal gave this rate to two categories, as insuffi cient data presented for the statistical test was identifi ed in 47.50% while as incorrect and insuffi cient demonstration of descriptive statistics was identifi ed at 16.25% [12].
When examined the publications errors in terms of notation errors are considered in two groups, thus; as mathematical notation error and statistical symbol error. Due to lack of suffi cient knowledge about mathematical notations as well as statistical symbols of, mathematical presentation and statistical symbol errors are seen in publications. In this study 2.94% of the investigated publications were found with mathematical notation errors and 3.43% of the publications with statistical symbol errors. In the study of Ercan et al. in medical journals the rates of mathematical notation errors and statistical symbol errors were identifi ed in 6.91% and 3.23% respectively [10]. Specifi cally, statistical notation errors in the process of evaluation will mislead the reviewer of the study and affect the understanding of the reader.
In some publications, by not giving the related explanation of the statistical expressions given by the authors, incomprehensive statistical terms were found. In the investigated articles, the rate of incomprehensive statistical terms was 0.49%. In the study of Ercan et al. in medical journals this rate was 4.15% [10]. Some studies were found with confl icting interpretations of statistical analysis. In this case publications were found with contradictory interpretations of especially the signifi cance of statistical tests. In this study 14.71% of the investigated publications were identifi ed with inappropriate interpretations. In the study of Ercan et al. in medical journals the rate was 8.76%, while in the study of Hanif and Ajmal the rate was 13.75% [10,12].
Another type of error discovered in the publications is when the researchers lack suffi cient knowledge of statistics, which lead to improper use of statistical terminology. In this study 7.35% of the investigated publications were identifi ed with errors in statistical terminology. In the study of Ercan et al in medical journals the rate was 9.68% [10].
In some publications the statistical method-analysis and the results were presented in improper sections. In the investigated publications these types of errors were seen when the researchers presented the p-value in the discussion section of the manuscript. In this study the rate of such type of error was identifi ed at 15.69% of the investigated publications. In the study of Ercan et al in medical journals the rate was 6.91% [10].
Statistical errors can be broadly classifi ed as; (i) errors in the presentation and terminology, not affecting the results, (ii) errors that are directly pertinent to the results [1]. When published studies in the fi eld of veterinary science were examined in terms of statistical errors, the rate of errors that either affected or not affected the results were found to be considerably high in number. With inadequate statistical knowledge as the leading cause statistical errors in publications, in the general scientifi c process these causes of statistical errors can be classifi ed into four groups: (a) not consulting a specialist on the topic, (b) falsely assuming that it is known, (c) not having adequate knowledge, and (d) carelessness [1].
Before accepting a publication for publishing, it has to go through a serious evaluation by relevant fi eld reviewers; however in this study, as well as other similar studies, unfortunately was seen that the statistical evaluation process was considered as insuffi cient or completely left out in the publications. Editors should give more consideration to the statistical aspect during the evaluation process of the manuscripts. Therefore, the editors should send the submitted studies to statistical reviewers before relevant fi eld reviewers. Performing a statistical review before sending off the study to relevant fi eld reviewers is more appropriate, at the same time it prolongs the review process of the manuscript, thereby providing enough time for all errors to be properly identifi ed. If the study is sent to the statistical reviewer at the same time with the relevant fi eld reviewer, any type of erroneous application of statistical methodology found by the statistical reviewer can bring up changes in results which also lead to changes in the discussions that can directly affect the evaluation of the fi eld expert, in such case the fi eld expert has to re-evaluate the study, again.
In the process of investigating the statistical error sources the researchers were not able to make one and standard classifi cation for the statistical errors. Therefore, even if the defi nitions of the errors are same, because of the differences that will arise from the classifi cation of the researchers to the respective error classifi cations, a big rate of differences can occur. However, when assessing the statistical errors according to the year of publications in the same journal, the assessment of same group of researchers will be a realistic approach.
To reduce and prevent the statistical errors in publications, the researchers must have a basic statistical knowledge and during the study process, they have to get a statistical consultation from fi eld experts. While in the review process, the reviewers have to send the studies to statistical editors when necessary and most important the editors have to redirect the publications to a statistical reviewer whenever needed.