Knowledge Representation in Patient Safety Reporting: An Ontological Approach
and | Sep 01, 2017
About this article
Article Category: Research Paper
Published Online: Sep 01, 2017
Page range: 75 - 91
Received: Nov 10, 2015
Accepted: May 06, 2016
DOI: https://doi.org/10.20309/jdis.201615
Keywords
© 2016 Chen Liang, Yang Gong
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.
Figure 1
Figure 2
Calculation of inter-rater reliability for the evaluation instrument.
| Item 1 | Item 2 | Item 3 | Item 4 | Item 5 | Item 6 | Item 7 | Item 8 | |
|---|---|---|---|---|---|---|---|---|
| Rater 1(WJ) | 4 | 4 | 4 | 4 | 5 | 4 | 4 | 5 |
| Rater 2 (YG) | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| Number in agreement | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 |
| Total agreement in percentile | 100% | |||||||
Workflow chart of ontology construction.
| Project | Task | Materials | Method/tool | Outcome |
|---|---|---|---|---|
| Conceptontology | Knowledge acquisition Ontology implementation | ICPS and the Common Formats Semantic knowledge organized in hierarchies | Expert analysis Expert review Ontology engineering | Semantic knowledge organized in hierarchies A concept ontology with a hierarchical structure of patient safety knowledge |
| Evaluation | Human evaluation | Hierarchical classes from the concept ontology Real-world reports from Web M&M | Survey instrument Statistics | Quality indicators of the classification by domain experts |
| Computational evaluation | Concept ontology in OWL | Statistical analysis Consistency checking | Quantitative indicators of the ontology | |
| Detailed ontology | Annotation | Concept ontology Dataset from a university hospital | Expert annotation | A detailed ontology with enriched terms, relations, and other ontological specifications |
A sample set of questions demonstrates the design of the survey instrument and the pre-assessment for validating the survey instrument.
| Dimensions | Questions in the survey instrument | Questions in the pre-assessment |
|---|---|---|
| Correctness | For the case you reviewed, the terms used in the taxonomy are well-formed and the words are well-arranged. | Does the scale purport to measure “The correctness of syntax”? |
| Meaningfulness | For the case you reviewed, the terms used in the taxonomy can represent the concepts in the real-world setting. | Does the scale purport to measure “The meaningfulness of terms”? |
| Clarity | For the case you reviewed, the terms that appear in the taxonomy are clear (no ambiguity). | Does the scale purport to measure “The clarity of terms”? |
| Comprehensiveness | For the case you reviewed, the taxonomy provides sufficient knowledge in the domain. | Does the scale purport to measure “The comprehensiveness of the taxonomy in a certain domain”? |
| Accuracy | The information the taxonomy provides is accurate. | Does the scale purport to measure “The accuracy of information”? |
| Specificity | The taxonomy satisfies your needs when you use it to categorize the case you are reviewing. | Does the scale purport to measure “Whether the taxonomy specifies agent’s specific requirements”? |
| Satisfaction | Please rate the overall satisfaction based on your experience of using the taxonomy. | Does the scale purport to measure “The overall satisfaction to the taxonomy”? |
| Educational value | Please rate the education value of the case you reviewed. | Does the scale purport to measure “The educational value of the case”? |
Statistics of ontology specific terms and imported terms.
| Ontology names | Classes | Object properties | Total |
|---|---|---|---|
| Patient Safety Ontology | 47 | 3 | 50 |
| International Classification for Patient Safety (ICPS) | 22 | 0 | 22 |
| Adverse Event Ontology (AEO) | 2 | 2 | 4 |
| Total | 71 | 5 | 76 |
Two raters rating on a 4-point scale for content validity.
| Item 1 | Item 2 | Item 3 | Item 4 | Item 5 | Item 6 | Item 7 | Item 8 | Proportion | |
|---|---|---|---|---|---|---|---|---|---|
| Rater 1 (WJ) | X | X | X | X | X | X | X | X | 1.00 |
| Rater 2 (YG) | X | X | X | X | X | X | X | X | 1.00 |
| Number in agreement | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | Mean I-CVI = 1.00 Mean rater proportion |
| Item CVI | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | = 1.00 |