Measurement of health and social behaviors in schoolchildren: Randomized study comparing paper versus electronic mode

Open access

Abstract

Introduction

Electronic survey mode has become a more common tool of research than it used to be previously. This is strongly associated with the overall digitization of modern society. However, the evidence on the possible mode effect on study results has been scarce. Therefore, the aim of this study is to investigate the comparability of findings on health and behaviours using a paper-versus-electronic mode of survey with randomization design among schoolchildren.

Methods

A randomized study was conducted using a mandatory questionnaire on international Health Behaviour in School-aged Children (HBSC) study in Lithuania, enrolling 531 schoolchildren aged 11–15 years. The questionnaire included health and social topics about physical activity, risk behaviours, self-reported health and symptoms, life satisfaction, bullying, fighting, family and school environment, peer relationships, electronic media communication, sociodemographic indicators, etc. The schoolchildren within classes were randomly selected for electronic or paper mode.

Results

It was found that by study mode differences are inconsistent and in the majority of cases do not exceed 5%-point difference between the modes. The only significant difference was that in the paper survey the participants reported more exercise than in the electronic survey (OR=8.08, P<.001). Other trends were nonsignificant and did not show a consistent pattern – in certain behaviours the paper mode was related to healthier choices, while in others - the electronic.

Conclusions

The use of electronic questionnaires in surveys of schoolchildren may provide findings that are comparable with concurrent or previously conducted paper surveys.

Abstract

Introduction

Electronic survey mode has become a more common tool of research than it used to be previously. This is strongly associated with the overall digitization of modern society. However, the evidence on the possible mode effect on study results has been scarce. Therefore, the aim of this study is to investigate the comparability of findings on health and behaviours using a paper-versus-electronic mode of survey with randomization design among schoolchildren.

Methods

A randomized study was conducted using a mandatory questionnaire on international Health Behaviour in School-aged Children (HBSC) study in Lithuania, enrolling 531 schoolchildren aged 11–15 years. The questionnaire included health and social topics about physical activity, risk behaviours, self-reported health and symptoms, life satisfaction, bullying, fighting, family and school environment, peer relationships, electronic media communication, sociodemographic indicators, etc. The schoolchildren within classes were randomly selected for electronic or paper mode.

Results

It was found that by study mode differences are inconsistent and in the majority of cases do not exceed 5%-point difference between the modes. The only significant difference was that in the paper survey the participants reported more exercise than in the electronic survey (OR=8.08, P<.001). Other trends were nonsignificant and did not show a consistent pattern – in certain behaviours the paper mode was related to healthier choices, while in others - the electronic.

Conclusions

The use of electronic questionnaires in surveys of schoolchildren may provide findings that are comparable with concurrent or previously conducted paper surveys.

1 Introduction

Information and communication technology has become an ever more demanded working tool to enhance the management, efficiency, and quality of surveys on health and social phenomena. There are several kinds of electronic questionnaires – online access, mobile device administered by the researcher, or computer/device handled by respondent. The responses can be collected by participant, researcher or a proxy (if a participant is minor). Overall digitization of social life and communication suggests ever-increasing pressure to conduct digital surveys and, therefore, it is essential to assess how reliable and valid the digital methods are and, if they replace paper-and-pencil method, are the findings comparable?

The online mode reduces the study costs by saving on the costs of paper and printing as well as from transportation (1). Besides, it ensures quick data with virtually no errors and suggests fewer no-response answers (2). Another important point is that these devices permit automatic checking of responses and complex skip patterns. However, in the digital survey mode, it is essential to ensure who is filling in the questionnaire, which is not always feasible.

The literature on the effects of digital-based and computer-adaptive testing suggests that digitization of standardized tests is a precise and appropriate research mode both from a scientific and logistic point of view (3, 4). Nonetheless, some researchers propose that the reliability of data obtained by the web-based approach should be determined (5). There is also a potential for selection bias, where a particular type of participant may be more prone to a particular survey mode (e.g. preference for digital mode among younger, more affluent or educated people). Moreover, in online mode, the participants can be unknown, not meet eligibility criteria or make double entries. Therefore, due to the potential for selection bias a randomized controlled design could be regarded as the main choice in studies on potential mode effects.

Even though many studies analysing the issue of mode effect on study results use randomization, quite a lot of them address the issue of response rate foremost, while content-specific comparison receives less attention. Also, such studies rarely investigate younger groups and the majority of them do not use randomization. For example, in the international Health Behaviour in School-aged Children (HBSC) study some countries use mixed mode design for more than a decade, e.g. Belgium (6), but they usually do not randomize the schools or children, leaving the choice of mode up to the school’s or child’s preference – which may be a subject to bias.

Thus, even though research on the validity and reliability of digital versus paper mode is quite extensive, such assessment in adolescents is rarely addressed. Moreover, the randomized approach in the research of mode effect is not always applicable, leaving the findings with a potential for self-selection or school-specific bias. In addition, the health perceptions and behaviours have also been under-investigated from this perspective. Therefore, the objective of our study is to compare the findings from paper and electronic mode using a randomized controlled design among schoolchildren.

2 Methods

2.1 Study Process and Sample

The randomized controlled study was conducted in May 2017 at five secondary schools in Lithuania. All study subjects were informed about the details of the study and that the return of the filled questionnaire will be treated as the informed consent. The anonymity of study participants and confidentiality of the data was ensured. The study was conducted as a pilot project for an oncoming 2018 Health Behaviour in School-aged Children study in Lithuania. The schools were randomly selected from the national schools’ list, by choosing the first five schools who agreed to participate in the study. The schools were from the second-largest city, other cities, and one town. In every school, the questionnaire was administered to 5th, 7th, and 9th grades (predominant age of children 11, 13 and 15 years, respectively). Then, the randomization was applied for every class in the school, with one-half of students filling the questionnaire in paper mode and the other half in electronic mode. Every class was randomized to define which half of the students’ list filled the online and which the paper version of the questionnaire.

Questionnaires (both electronic and paper mode) were administered in school classrooms by trained researchers who complied with written instructions. The electronic version of the questionnaire was uploaded to Google Forms, which was available only to the researchers. During the survey, the researchers shared the web link to study participants. The online questionnaire was filled in on desktop or tablet computers. The places for survey were usually classrooms, computer rooms or libraries. In some cases, the survey of paper and online mode was conducted simultaneously in the same room. Every researcher wrote the notes about the procedure of survey.

2.2 Measurements

The tool for the study was based on the then-current version of the standardized international HBSC research protocol (7). The HBSC questionnaire covers a wide range of health and social topics about schoolchildren’s physical activity, risk behaviours, self-reported health and symptoms, life satisfaction, bullying, fighting, family, school environment, peer relationships, electronic media communication, sociodemographic indicators, etc. Only the mandatory items were included. The sequence, formulation, and overall visualization of items did not differ by mode.

Some items of the questionnaire were used from particular scales or subscales:

  • HBSC symptom checklist, 8 items (7),

  • Family Affluence Scale, 6 items (8),

  • Multidimensional Scale of Perceived Social Support: Family, 4 items (9),

  • Multidimensional Scale of Perceived Social Support: Friends, 4 items (adapted from (9)),

  • Teacher and Classmate Support Scale: Classmates, 3 items (adapted from (10)),

  • Teacher and Classmate Support Scale: Teachers, 3 items (adapted from (10)),

  • Online contact with friends and others, 4 items (11),

  • Preference for online communication, 3 items (12),

  • Social media addiction, 9 items (13).

2.3 Data Analysis

Data was processed using MS Excel 2010 and analysed using IBM SPSS Statistics, version 20. The descriptive analysis included the calculation of the prevalence of different health behaviours (%). The items were dichotomized based on the cut-offs used in the 2014 Health Behaviour in School-aged Children study report (14). The main purpose of the analysis was to estimate whether various health-related items are similarly distributed among study groups in schoolchildren that filled in the questionnaire in paper-versus-electronic mode. For this, the percentage point differences were calculated, and logistic regression was used with the calculation of certain behaviours’ risk when comparing the modes. The differences between the modes were estimated using percentage point difference and odds ratios with the reference group being electronic mode (OR=1.00). Given that despite randomization there were some imbalances between the study groups by gender, grade, and school, these indicators were adjusted for in the multivariate logistic regression model.

Due to multiple comparisons of different indicators, the Bonferroni correction was used: in total, 78 variables were compared, so the conventional significance level of P<0.05 was decreased to P<0.001 (0.05/78=0.00064). The P-values between 0.001 and 0.05 were reported as trends.

3 Results

The study sample comprised 531 schoolchildren – 261 filled the electronic questionnaire and 270 the paper version. The overall response rate was 83.0% with higher rates among girls and elder schoolchildren. A detailed comparison of study groups by gender, grade, and school are presented in Table 1. Regardless of randomization, there were some differences observed between study groups and since they were definitely random (by design of the study) their statistical significance was not calculated.

Table 1

The main characteristics of study sample.

CharacteristicElectronic modePaper modenResponse rate
Gender
Boys51.448.625577.0
Girls47.352.727589.0
Grade
5th49.750.318777.9
7th48.851.220184.1
9th49.051.014388.8
School
#1 (large city)47.352.77489.2
#2 (large city)48.751.322494.5
#3 (city)50.549.510360.6
#4 (city)50.050.04880.0
#5 (town)50.050.08291.1

In this study, the internal consistency of scales and subscales was acceptable and the difference between the modes was not more than .07 points – with no consistent superiority of either mode (Table 2).

Table 2

Internal consistency of study scales and subscales by survey mode.

ScaleNumber of itemsInternal consistency (α)
Electronic modePaper mode
HBSC symptom checklist8.78.79
Family Affluence Scale6.52.58
Multidimensional Scale of Perceived Social Support: Family4.76.69
Multidimensional Scale of Perceived Social Support: Friends4.90.85
Teacher and Classmate Support Scale: Classmates3.77.70
Teacher and Classmate Support Scale: Teachers3.75.74
Online contact with friends and others4.54.54
Preference for online communication3.84.81
Social media addiction9.75.76

3.1 Health Behaviours

In the field of health behaviours (Table 3), the largest difference depending on survey mode was observed in extensive physical activity – in paper mode, the schoolchildren more frequently reported daily exercise until getting out of breath or sweating (OR=8.08, P<.001). Other indicators had no differences except the trends that students in paper mode more frequently reported, such as having a regular breakfast on weekends (OR=1.93, P=.009). Almost all aspects of health behaviours differed between the survey modes by no more than 5% points.

Table 3

Health behaviours of schoolchildren by survey mode.

CharacteristicPrevalence, %
% differenceORP
ElectronicPaper
Eating habits
Having breakfast during the weekdaysEvery day58.862.73.91.18.366
Having breakfast during the weekendsEvery day79.687.78.11.93.009
Having breakfast with parentsEvery day41.040.7-.31.00.982
Having dinner with parentsEvery day47.145.6-1.5.96.816
Eating fruitsEvery day41.838.5-3.3.87.446
Eating vegetablesEvery day32.634.21.61.07.707
Eating sweetsEvery day16.113.8-2.3.83.453
Drinking soft drinksEvery day5.46.3.91.23.593
Drinking energy drinksEvery day2.3.4-1.9.16.097

Health and well-being
Subjective health assessmentGood88.591.83.31.59.132
Life satisfaction6–10 (10 pts scale)87.785.8-1.9.84.510
HeadacheRarely84.382.5-1.8.91.707
Stomach acheRarely93.593.7.21.04.912
BackacheRarely91.692.1.51.09.794
Feeling lowRarely80.181.31.21.14.577
Irritability or bad temperRarely72.076.54.51.39.115
Feeling nervousRarely70.569.7-.81.01.946
Difficulties in getting to sleepRarely79.783.23.51.33.214
Feeling dizzyRarely89.789.9.21.08.798
Brushing the teethMore than once a day61.362.81.51.05.809
Body imageA bit too thin11.915.03.1.71.200
A bit too fat29.130.71.6.91.654
About the right size59.054.3-4.71.00-

Physical activity
Physical activity at least 60 minutes per day (last week)7 days18.920.61.71.16.518
Exercise in free time until getting out of breath or sweatingEvery day3.119.716.68.08<.001

Risk behaviour
Cigarette smoking (lifetime)Never73.277.84.61.37.187
Cigarette smoking (last month)Never88.192.34.21.75.097
Alcohol drinking (lifetime)Never62.868.45.61.37.129
Alcohol drinking (last month)Never88.988.0-.9.89.706
Cannabis taking (lifetime)Never94.397.02.72.22.101
Cannabis taking (last month)Never98.199.21.12.13.376
Sexual intercourseNo95.093.3-1.7.81.626

3.2 Social Behaviours and School

The selected indicators of social behaviours under study showed slightly bigger differences than health behaviours, though they were inconsistent and nonsignificant (Table 4). Here the trend in paper mode was that the children were more likely to report having friends to share joys and sorrows, but also more cyber-bullying and more treatment-needed injuries (.001<P<.05).

Table 4

School behaviours of schoolchildren by survey mode.

CharacteristicPrevalence, %
% differenceORP
ElectronicPaper
Friends support
Friends helpAgree76.676.4-.2.97.870
Counting on friendsAgree75.178.33.21.18.440
Having friends to share joys and sorrowsAgree78.287.79.51.98.005
Being able to talk about problems with friendsAgree70.576.45.91.35.138

Classmate support
Enjoy being together with studentsAgree59.854.1-5.7.81.243
Students in class are kind and helpfulAgree53.352.8-.51.00.982
Students accepting one as he/she isAgree67.461.8-5.6.80.231

Teacher support
Teachers accepting one as he/she isAgree75.178.13.01.23.348
Teachers caringAgree49.852.62.81.15.426
Feeling a lot of trust in teachersAgree65.167.52.41.16.462

School perception
Feeling about schoolI like it a lot81.182.91.81.14.586
Pressure by schoolworkSome or a lot72.272.7.5.99.947

Bullying
Taking part in bullying another studentYes44.242.9-1.3.97.889
at school, last two months
Being bullied at school, last two monthsYes50.246.5-3.7.86.402
Taking part in cyber-bullying, last two monthsYes20.318.7-1.6.92.708
Being cyber-bullied, last two monthsYes14.122.38.21.82.011

Physical fighting
Having a physical fight, last yearYes31.729.4-2.3.92.688

Injuries
Being injured with treatment needed, last yearYes48.857.68.81.45.036

3.3 Family Environment

The evaluation of schoolchildren’s family environment revealed that there were almost no differences depending on survey mode (Table 5). The children in paper mode reported slightly better family communication and support, but this was nonsignificant (P=0.068). All other indicators did not reach a 5%-point difference and, regarding items on family affluence, the differences by paper mode were also minor.

Table 5

Family-related perceptions of schoolchildren by survey mode.

CharacteristicPrevalence, %
% differenceORP
ElectronicPaper
Communication
Ease to talk about things that really bother: to fatherEasy63.269.76.51.43.068
Ease to talk about things that really bother: to stepfatherEasy11.515.54.01.48.221
Ease to talk about things that really bother: to motherEasy79.782.02.31.15.526
Ease to talk about things that really bother: to stepmotherEasy9.610.1.51.08.848

Support
Family really tries helpingAgree92.093.71.71.29.461
Getting emotional help and support from familyAgree83.988.04.11.43.164
Being able to talk about problems with familyAgree69.371.21.91.13.537
Family is willing to help in making decisionsAgree86.690.33.71.45.183

Affluence
Own bedroomYes81.281.4.21.02.941
Dishwasher at homeYes62.559.7-2.8.87.496
Bathrooms at homeOne or more97.797.8.11.03.958
Family carOne or more95.093.7-1.3.78.525
Computers at homeOne or more97.397.7.41.15.806
Family travel abroad for vacation, last yearOnce or more86.581.6-4.9.68.129

3.4 Electronic Media Communication

The survey included three main aspects of electronic communication – online contact with friends, preference for online communication, and social media addiction (Table 6). Here there were two trends: in electronic mode, children reported using social media more as a way to escape from negative feelings and having conflicts with family members because of social media use (.001<P<.05). All other items were indifferent by mode and rarely exceeded a 5%-point difference.

Table 6

Electronic media communication of schoolchildren by survey mode.

CharacteristicPrevalence, %
% differenceORP
ElectronicPaper
Online contact with friends and others
Close friend(s)Every day66.763.3-3.4.81.277
Friends from a larger friend groupEvery day37.940.62.7.12.535
Friends that you got to know through the internet but didn’t know beforeEvery day11.514.73.21.32.302
Other people than friendsEvery day44.847.72.91.11.563
Preference for online communication
On the internet, I talk more easily about secrets than in a face-to-face encounterAgree27.325.7-1.6.93.725
On the internet, I talk more easily about my inner feelings than in a face-to-face encounterAgree26.822.4-4.4.78.238
On the internet, I talk more easily about my concerns than in a face-to-face encounterAgree27.620.1-7.5.67.054
Social media addiction
Regularly felt dissatisfied because you wanted to spend more time on social mediaYes16.918.01.11.03.912
Often felt bad when you could not use social mediaYes27.225.7-1.5.88.528
Tried to spend less time on social media, but failedYes28.725.3-3.4.81.284
Regularly neglected other activities (e.g. hobbies, sport) because you wanted to use social mediaYes13.812.4-1.4.90.700
Regularly had arguments with others because of your social media useYes14.912.0-2.9.76.290
Regularly lied to your parents or friends about the amount of time you spend on social mediaYes17.216.1-1.1.89.628
Often used social media to escape from negative feelingsYes30.322.5-7.8.64.030
Had serious conflict with your parents, brother(s) or sister(s) because of your social media useYes21.813.5-8.3.49.004
Regularly found that you can’t think of anything else but the moment that you will be able to use social media againYes40.639.9-.7.93.704

3.5 Procedure-Specific Findings

In this study, the environment and circumstances of the survey were also documented in order to depict the procedure-specific findings. So, during the survey and especially in online mode some participants were able to see the adjacent participants’ responses, thus infringing the privacy of other responders. In addition, the teachers sometimes refused to leave the classroom even when asked.

It was also observed that some schoolchildren were not content with the assigned mode of the survey. However, this was not mode-specific: some adolescents expressed the wish to move from paper mode to electronic, while others vice versa. The former ones were keener to choose the electronic device (computer or tablet) instead of paper, while the latter preferred more privacy. Of note, some students were concerned about the split of the class into different modes as if treated unequally.

4 Discussion

Electronic research mode is very convenient for large-scale studies. In order to address the possible effect of survey mode on its results, we conducted a randomized study to eliminate the potential for selection bias within the study sample. This is the main strength of our study since the previous research has quite frequently neglected the issue of self-selection bias that arises in non-randomized studies. This is especially relevant across different social conditions such as schools, where some of them may have better resources to prefer online mode, either through better financing or through higher quality of educational services. So, by the study design, our study avoided the possible self-selection bias or school-specific differences by randomizing the schoolchildren within classes. We also adjusted the calculations by main sociodemographic indicators that could affect the differences. Besides, the inclusion of different size schools from bigger and smaller urban areas also increased the diversity of schoolchildren.

However, when discussing our study weaknesses, we had a limited sample size, which potentially led to an underestimating of the statistical significance of differences, especially when controlling for multiplicity. Nevertheless, we found that in the majority of cases the differences between the modes were small and did not exceed a 5% point. For such five percent differences to detect as statistically significant at P<0.05 level, we would have needed the sample from 431 to 1,559 per arm – and this without the multiplicity correction that was applied in our study (for P<0.01 640 and 2,315 participants would be needed (15), respectively, and for P<0.001 even more). Compared to previous studies, our sample size was rather medium, and we had no intention to find minor differences as statistically significant. After all, the fact that absolute differences between the modes were inconsistent (i.e. not showing better health behaviours in either mode) suggests the likely absence of substantial differences.

Another limitation of our study was the lack of replicability since our study participants had an opportunity to fill in the questionnaire only in one of the modes. Therefore, the assessment of the consistency of results within individuals was not possible. This occurred because we raised no question regarding the particular subject’s replicability of responses – rather, we had an interest in comparing the population (i.e. study sample) estimates.

Overall, our study findings revealed that differences by study mode are virtually absent and in the majority of cases do not exceed five percent difference between the modes. These findings do not have many studies to compare with since the schoolchildren’s health behaviours have rarely been addressed in previous research on survey modes. The study on the HBSC sample was previously reported by Vereecken and Maes (6) in Belgium. Their findings showed some differences by mode, but our results did not support them. We did not detect those differences not only due to a smaller sample size (i.e. lower power of the study) but rather due to the absence of absolute difference.

That same study (6) noted that for several questions about feelings and affective states more socially desirable responses were found in paper format. However, in our case, this was not observed. Even though we saw some larger differences when assessing social support measures, this was also inconsistent. The fact that the adolescents provide equivalent responses in paper and computer formats was also found elsewhere (16).

It was found that adolescents were more likely to report substance use and less desirable aspects of psychological well-being using a digital format (17). However, we found that subjective health was reported as slightly better in paper mode (like Smith et al. (18) in the military sample), while higher life satisfaction was reported as better in electronic mode, which does not suggest the consistency of mode effect.

The issue of different responses by survey mode has been addressed with other samples as well. For example, patients after knee surgery reported similar levels of daily functioning, quality of life, pain intensity as well as symptoms (19). Similarly, for college students in education and psychology, the survey mode did not have strong differential effects on data quality regarding the learning environment and perceptions (20). One study on military participants found some differences in health behaviours by mode, though like our study they did not exceed five percent (18).

We also compared the internal consistency of scales. Previous studies demonstrated that electronic mode is likely to show higher internal consistency compared to the paper, with differences by up to .30 (16), however, in our study there was no superiority of either mode (differences did not exceed .07). Some other studies also showed no relevant differences in psychometric properties by mode (21).

Previous research suggests that young people are keener to choose the electronic than the paper version (22), while the studies of other samples are rather ambivalent: for instance, the study on people who take supplements and vitamins found the electronic version as more acceptable (1, 23), while HIV patients preferred the paper version (24). Interestingly, our procedure-specific findings also indicate ambiguity, since some children preferred to move to electronic, while others to paper mode. This was rather unexpected due to the hypothesized preference of digital natives toward electronic mode. It could be explained by the fact that maintaining privacy was an issue during this study, especially when filling in electronic questionnaires on desktop computers: schoolchildren were able to see the answers of adjacent classmates, which could have made them feel insecure. Additionally, in some classes, the teachers refused to leave the room, which may interfere with the confidentiality perceptions of children and the sincerity of their responses. The fact that some schoolchildren complained about having different survey modes across the class implies that, in the future, a class as an entity should preferably be investigated using the same mode.

Regarding the cost-effectiveness of the shift from paper mode to an electronic mode in our study, the main difference was related to expenses for paper and printing the questionnaire as well as typing in the responses from paper to database. In addition, the probability of data typing errors in case of the electronic mode is virtually zero. Nevertheless, the shift toward electronic mode should be approached carefully: if the survey is going to be uploaded online with a non-restricted access, the study participants cannot be controlled. This may further result in a situation where some subjects submit several questionnaires, or they do not meet the eligibility criteria for age or other relevant characteristics. It should be emphasized that in online surveys the basic concern is related to the problem of who really fills in the questionnaire and if they meet the eligibility criteria of the study. This should be controlled whenever possible.

5 Conclusions

Summarizing our study, it can be stated that the comparison of electronic and paper mode in the research of health and social behaviours among schoolchildren revealed no consistent differences between the modes. There were some items or questions that had larger differences between the survey modes, however, they did not have a trend to be healthier or more socially desirable in one particular mode. This suggests that, in the future, the use of electronic questionnaires in surveys of schoolchildren may provide findings that are comparable with concurrent or previously conducted paper surveys. However, this does not relieve the concerns related to electronic surveys where the study participants are not controlled in terms of eligibility criteria. Thus, when the electronic survey responders are unknown, this still threatens the validity of study findings.

Acknowledgments

We thank all study participants and schools who agreed to enter this study. We also would like to express gratitude to reviewers whose comments and suggestions improved the manuscript.

Conflicts of Interest

The authors declare that no conflicts of interest exist.

Funding

The study was financed by internal institutional sources.

Ethical Approval

The study protocol was approved by the Ethics Committee at the Lithuanian University of Health Sciences, reference number BEC-SP(B)-129, and it conforms to the provisions of the Declaration of Helsinki in 1995 (as revised in Edinburgh in 2000).

References

  • 1

    Touvier M, Méjean C, Kesse-Guyot E, Pollet C, Malon A, Castetbon K, et al. Comparison between web-based and paper versions of a self-administered anthropometric questionnaire. Eur J Epidemiol. 2010;25(5):287-96.

  • 2

    Bech M, Kristensen MB. Differential response rates in postal and web-based surveys among older respondents. Surv Res Methods. 2009;3:1-6.

  • 3

    Couper MP. The future of modes of data collection. Public Opin Q. 2011;75:889-908.

  • 4

    Steelman ZR, Hammer B, Limayem M. Data collection in the digital age: innovative alternatives to student samples. MIS Q. 2014;38:35578.

  • 5

    van Gelder MMHJ, Bretveld RW, Roeleveld N. Web-based questionnaires: the future in epidemiology? Am J Epidemiol. 2010;172:1292-8.

  • 6

    Vereecken CA, Maes L. Comparison of a computer-administered and paper-and-pencil administered questionnaire on health and lifestyle behaviors. J Adolesc Health. 2006;38:426-32.

  • 7

    Inchley J, Currie D, Cosma A, editors. Health Behaviour in School-aged Children (HBSC) study protocol: background, methodology and mandatory items for the 2017/18 survey. St Andrews: CAHRU, 2018.

  • 8

    Torsheim T, Cavallo F, Levin K, Schnohr C, Mazur J, Niclasen B, et al. Psychometric validation of the revised family affluence scale: a latent variable approach. Child Indic Res. 2015;9:771-84.

  • 9

    Zimet GD, Dahlem NW, Zimet SG, Farley GK. The multidimensional scale of perceived social support. J Pers Assess. 1988;52:30-41.

  • 10

    Torsheim T, Samdal O, Rasmussen M, Freeman J, Griebler R, Dür W. Cross-national measurement invariance of the teacher and classmate support scale. Soc Indic Res. 2012;105:145-60.

  • 11

    Mascheroni G, Ólafsson K. Net children go mobile: cross-national comparisons: report D3.3. Milano: Educatt, 2014.

  • 12

    Peter J, Valkenburg P. Individual differences in perceptions of internet communication. Eur J Commun. 2006;21:213-26.

  • 13

    Van den Eijnden RJ, Lemmens JS, Valkenburg PM. The social media disorder scale. Comput Human Behav. 2016;61:478-87.

  • 14

    Inchley J, Currie D, Young T, Samdal O, Torsheim T, Augustson L, et al. Growing up unequal: gender and socioeconomic differences in young people’s health and well-being. World Health Organisation, Health Policy for Children and Adolescents, 2016;7. Accessed March 19th, 2018 at: http://www.euro.who.int/__data/assets/pdf_file/0003/303438/HSBC-No.7-Growing-up-unequal-Full-Report.pdf

  • 15

    Chow S, Shao J, Wang H. Sample size calculations in clinical research. 2nd ed. Chapman & Hall/CRC Biostatistics Series, 2008:89.

  • 16

    Norman GJ, Sallis JF, Gaskins R. Comparability and reliability of paper- and computer-based measures of psychosocial constructs for adolescent physical activity and sedentary behaviors. Res Q Exerc Sport. 2005;76:315-23.

  • 17

    Wright DS, Aquilino W, Supple A. A comparison of computer-assisted and paper-and-pencil self-administered questionnaires in a survey on smoking, alcohol, and drug use. Public Opin Q. 1998;62:331-53.

  • 18

    Smith B, Smith TC, Gray GC, Ryan MAK. When epidemiology meets the internet: web-based surveys in the millennium cohort study. Am J Epidemiol. 2007;166:1345-54.

  • 19

    Bojcic JL, Sue VM, Huon TS, Maletis GB, Inacio MCS. Comparison of paper and electronic surveys for measuring patient-reported outcomes after anterior cruciate ligament reconstruction. Perm J. 2014;18:22-6.

  • 20

    Hardré PL, Crowson HM, Xie K, Ly C. Testing differential effects of computer-based, web-based and paper-based administration of questionnaire research instruments. Br J Educ Technol. 2007;38(1):5-22.

  • 21

    Riva G, Teruzzi T, Anolli L. The use of the internet in psychological research: comparison of online and offline questionnaires. Cyberpsychol Behav. 2003;6:73-80.

  • 22

    Miller ET, Neal DJ, Roberts LJ, Baer JS, Cressler SO, Metrik J, et al. Test-retest reliability of alcohol measures: is there a difference between Internet-based assessment and traditional methods? Psychol Addict Behav. 2002;16:56-63.

  • 23

    Carvalho JC, Declerck D, De Vos E, Kellen J, Van Nieuwenhuysen JP, Vanobbergen, et al. Validation of the oral survey-b system for electronic data capture in national oral health surveys. Caries Res. 2016;50:288-94.

  • 24

    Duracinsky M, Lalanne C, Goujard C, Herrmann S, Cheung-Lung C, Brosseau JP, et al. Electronic versus paper-based assessment of health-related quality of life specific to HIV disease: reliability study of the PROQOL-HIV questionnaire. J Med Internet Res. 2014;16(4):e115.

1

Touvier M, Méjean C, Kesse-Guyot E, Pollet C, Malon A, Castetbon K, et al. Comparison between web-based and paper versions of a self-administered anthropometric questionnaire. Eur J Epidemiol. 2010;25(5):287-96.

2

Bech M, Kristensen MB. Differential response rates in postal and web-based surveys among older respondents. Surv Res Methods. 2009;3:1-6.

3

Couper MP. The future of modes of data collection. Public Opin Q. 2011;75:889-908.

4

Steelman ZR, Hammer B, Limayem M. Data collection in the digital age: innovative alternatives to student samples. MIS Q. 2014;38:35578.

5

van Gelder MMHJ, Bretveld RW, Roeleveld N. Web-based questionnaires: the future in epidemiology? Am J Epidemiol. 2010;172:1292-8.

6

Vereecken CA, Maes L. Comparison of a computer-administered and paper-and-pencil administered questionnaire on health and lifestyle behaviors. J Adolesc Health. 2006;38:426-32.

7

Inchley J, Currie D, Cosma A, editors. Health Behaviour in School-aged Children (HBSC) study protocol: background, methodology and mandatory items for the 2017/18 survey. St Andrews: CAHRU, 2018.

8

Torsheim T, Cavallo F, Levin K, Schnohr C, Mazur J, Niclasen B, et al. Psychometric validation of the revised family affluence scale: a latent variable approach. Child Indic Res. 2015;9:771-84.

9

Zimet GD, Dahlem NW, Zimet SG, Farley GK. The multidimensional scale of perceived social support. J Pers Assess. 1988;52:30-41.

10

Torsheim T, Samdal O, Rasmussen M, Freeman J, Griebler R, Dür W. Cross-national measurement invariance of the teacher and classmate support scale. Soc Indic Res. 2012;105:145-60.

11

Mascheroni G, Ólafsson K. Net children go mobile: cross-national comparisons: report D3.3. Milano: Educatt, 2014.

12

Peter J, Valkenburg P. Individual differences in perceptions of internet communication. Eur J Commun. 2006;21:213-26.

13

Van den Eijnden RJ, Lemmens JS, Valkenburg PM. The social media disorder scale. Comput Human Behav. 2016;61:478-87.

14

Inchley J, Currie D, Young T, Samdal O, Torsheim T, Augustson L, et al. Growing up unequal: gender and socioeconomic differences in young people’s health and well-being. World Health Organisation, Health Policy for Children and Adolescents, 2016;7. Accessed March 19th, 2018 at: http://www.euro.who.int/__data/assets/pdf_file/0003/303438/HSBC-No.7-Growing-up-unequal-Full-Report.pdf

15

Chow S, Shao J, Wang H. Sample size calculations in clinical research. 2nd ed. Chapman & Hall/CRC Biostatistics Series, 2008:89.

16

Norman GJ, Sallis JF, Gaskins R. Comparability and reliability of paper- and computer-based measures of psychosocial constructs for adolescent physical activity and sedentary behaviors. Res Q Exerc Sport. 2005;76:315-23.

17

Wright DS, Aquilino W, Supple A. A comparison of computer-assisted and paper-and-pencil self-administered questionnaires in a survey on smoking, alcohol, and drug use. Public Opin Q. 1998;62:331-53.

18

Smith B, Smith TC, Gray GC, Ryan MAK. When epidemiology meets the internet: web-based surveys in the millennium cohort study. Am J Epidemiol. 2007;166:1345-54.

19

Bojcic JL, Sue VM, Huon TS, Maletis GB, Inacio MCS. Comparison of paper and electronic surveys for measuring patient-reported outcomes after anterior cruciate ligament reconstruction. Perm J. 2014;18:22-6.

20

Hardré PL, Crowson HM, Xie K, Ly C. Testing differential effects of computer-based, web-based and paper-based administration of questionnaire research instruments. Br J Educ Technol. 2007;38(1):5-22.

21

Riva G, Teruzzi T, Anolli L. The use of the internet in psychological research: comparison of online and offline questionnaires. Cyberpsychol Behav. 2003;6:73-80.

22

Miller ET, Neal DJ, Roberts LJ, Baer JS, Cressler SO, Metrik J, et al. Test-retest reliability of alcohol measures: is there a difference between Internet-based assessment and traditional methods? Psychol Addict Behav. 2002;16:56-63.

23

Carvalho JC, Declerck D, De Vos E, Kellen J, Van Nieuwenhuysen JP, Vanobbergen, et al. Validation of the oral survey-b system for electronic data capture in national oral health surveys. Caries Res. 2016;50:288-94.

24

Duracinsky M, Lalanne C, Goujard C, Herrmann S, Cheung-Lung C, Brosseau JP, et al. Electronic versus paper-based assessment of health-related quality of life specific to HIV disease: reliability study of the PROQOL-HIV questionnaire. J Med Internet Res. 2014;16(4):e115.

Slovenian Journal of Public Health

The Journal of National Institute of Public Health

Journal Information

IMPACT FACTOR 2017: 0.620
5-year IMPACT FACTOR: 0.488



CiteScore 2017: 0.33

SCImago Journal Rank (SJR) 2017: 0.147
Source Normalized Impact per Paper (SNIP) 2017: 0.429

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 310 310 201
PDF Downloads 72 72 37