A Three–Level Aggregation Model for Evaluating Software Usability by Fuzzy Logic

Open access


Rapid deployment of IT brings about new issues with software usability measurement. Usability is based on users’ experience and is strongly subjective, having a qualitative character. The users’ comfort is usually collected by surveys in their daily work. The present article stems from an experimental study related to the evaluation of the usability of tools by a rule-based system. The work suggests a robust computational model that will be able to avoid the main problems arising from the experimental study (a large and less-legible rule base) and to deal with the vagueness of IT user experience, different levels of skills and various numbers of filled questionnaires in different departments. The computational model is based on three hierarchical levels of aggregation supported by fuzzy logic. Choices for the most suitable aggregation functions in each level are advocated and illustrated with examples. The number of questions and granularity of answers in this approach can be adjusted to each user group, which could reduce the response burden and errors. Finally, the paper briefly describes further possibilities of the suggested approach.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • Albert W. and Tullis T. (2013). Measuring the User Experience Collecting Analyzing and Presenting Usability Metrics (Interactive Technologies) 2nd Edition Elsevier Amsterdam.

  • Allen I.E. and Seaman C. (2007). Likert scales and data analyses Technical report QP-Quality Progress http://asq.org/quality-progress/2007/07/statistics/likert-scales-and-data-analyses.html.

  • Bavdaž M. (2010). Sources of measurement errors in business surveys Journal of Official Statistics 26(1): 25–42.

  • Bavdaž M. Biffignandi S. Bolko I. Giesen D. Gravem D. and Haraldsen G. (2011). Final report integrating findings on business perspectives related to NSIS’ statistics Technical report Deliverable 3.2. FP7 Blue-Ets Project European Commission Brussels https://cordis.europa.eu/project/rcn/94081/results/en?rcn=143042.

  • Beliakov G. Pradera A. and Calvo T. (2007). Aggregation Functions: A Guide for Practitioners Springer-Verlag Berlin/Heidelberg.

  • Calinescu M. and Schouten B. (2012). Adaptive survey designs that minimize nonresponse and measurement risk Technical report Statistics Netherlands The Hague/Heerlen.

  • Calvo T. Kolesárová A. Komorníková M. and Mesiar R. (2002). Aggregation operators: Properties classes and construction methods in T. Calvo et al. (Eds) Aggregation Operators: New Trends and Applications Physica Heidelberg pp. 3–104.

  • Dubois D. and Prade H. (2004). On the use of aggregation operations in information fusion processes Fuzzy Sets and Systems 142(1): 143–161.

  • Dujmović J. (2007). Continuous preference logic for system evaluation IEEE Transactions on Fuzzy Systems 15(6): 1082–1099.

  • Dujmović J. (2018). Soft Computing Evaluation Logic: The LSP Decision Method and Its Applications Wiley/IEEE Computer Society Hoboken NJ.

  • Grabisch M. Marichal J.-L. Mesiar R. and Pap E. (2009). Aggregation Functions Encyclopedia of Mathematics and its Applications Cambridge University Press Cambridge.

  • Greiner L. and White S. (2019). What is ITIL? Your guide to the it infrastructure library in digital magazine CIO from IDG https://www.cio.com/article/2439501/infrastructure-it-infrastructure-library-itil-definition-and-solutions.html.

  • Gupta M. and Qi J. (1991). Theory of t-norms and fuzzy inference methods Fuzzy Sets and Systems 40(3): 431–450.

  • Hensher A. Rose J. and Greene W. (2015). Applied Choice Analysis Cambridge University Press Cambridge.

  • Herrera F. and Martínez L. (2001). A model based on linguistic 2-tuples for dealing with multigranular hierarchical linguistic contexts in multiexpert decision-making IEEE Transactions on Systems Man and Cybernetics B: Cybernetics 31(2): 227–234.

  • ISACA (2018). Service it governance professionals COBIT5 an ISACA framework http://www.isaca.org/cobit/pages/default.aspx.

  • ISO (2011). Systems and software engineering ISO/IEC 25010:2011: Systems and software quality requirements and evaluation (square). System and software quality models https://www.iso.org/standard/35733.html.

  • ISO (2018). ISO online browsing platform ISO 9241-11:2018: Ergonomics of human–system interaction. Part 11: Usability: Definitions and concepts https://www.iso.org/standard/63500.html.

  • Kacprzyk J. and Yager R. (2001). Linguistic summaries of data using fuzzy logic International Journal of General Systems 30(2): 133–154.

  • Kacprzyk J. Yager R.R. and Zadrożny S. (2000). A fuzzy logic based approach to linguistic summaries of databases International Journal of Applied Mathematics and Computer Science 10(4): 813–834.

  • Králiková L. (2017). Testovanie efektívnosti softvéru v podnikovej praxi z hľadiska užívateľov (Software Effectiveness Testing in Business Practice from a User Perspective) Master thesis University of Economics in Bratislava Bratislava.

  • Likert R. (1932). A technique for the measurement of attitudes Archives of Psychology 22(140): 1–55.

  • Miller G. (1956). The magical number seven plus or minus two: Some limits on our capacity for processing information Psychological Review 63(2): 81–97.

  • Morente-Molinera J. Kou G. Pérez I. Samuylov K. Selamat A. and Herrera-Viedma E. (2018). A group decision making support system for the web: How to work in environments with a high number of participants and alternatives Applied Soft Computing 68: 191–201.

  • Olson D. (1996). Decision Aids for Selection Problems Springer-Verlag London.

  • Pavlík L. (2018). Metrics for evaluating information systems Posterus Portl pre odborn publikovanie http://www.posterus.sk/?p=18957.

  • Piegat A. and Pluciński M. (2015). Computing with words with the use of inverse RDM models of membership functions International Journal of Applied Mathematics and Computer Science 25(3): 675–688 DOI:10.1515/amcs-2015-0049.

  • Rakovská E. and Hudec M. (2020). Two approaches for the computational model for soft-ware usability in practice in J. Kacprzyk et al. (Eds) Information Technology System Research and Computational Physics ITSRCP 2018 Advances in Intelligent Systems and Computing Vol. 945 Springer Cham pp. 191–202.

  • Ruspini E. (1969). A new approach to clustering Information and Control 15(1): 22–32.

  • Seffah A. Kececi N. and Donyaee M. (2001). QUIM: A framework for quantifying usability metrics in software quality models 2nd Asia-Pacific Conference on Quality Software Hong Kong pp. 311–318.

  • Snijkers G. Haraldsen G. Jones J. and Willimack D. (2013). Designing and Conducting Business Surveys Wiley Hoboken NJ.

  • Tudorie C. (2008). Qualifying objects in classical relational database querying in J. Galindo (Ed.) Handbook of Research on Fuzzy Information Processing in Databases Information Science Reference Hershey pp. 218–245.

  • Yager R. (1982). A new approach to the summarization of data Information Sciences 28(1): 69–86.

  • Yager R. and Rybalov A. (1996). Uninorm aggregation operators Fuzzy Sets and Systems 80(1): 111–120.

  • Zadeh L. (1965). Fuzzy sets Information and Control 8(3): 338–353.

  • Zimmermann H. (2001). Fuzzy Set Theory and Its Applications Kluwer Academic Publishers Dordrecht.

Journal information
Impact Factor

IMPACT FACTOR 2018: 1.504
5-year IMPACT FACTOR: 1.553

CiteScore 2018: 2.09

SCImago Journal Rank (SJR) 2018: 0.493
Source Normalized Impact per Paper (SNIP) 2018: 1.361

Mathematical Citation Quotient (MCQ) 2018: 0.08

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 78 78 8
PDF Downloads 115 115 8