1. Human-led and artificial intelligence-automated critical appraisal of systematic reviews : comparative evaluationLucija Gosak, Gregor Štiglic, Wilson Tam, Dominika Vrbnjak, 2025, original scientific article Abstract: Aim To evaluate and compare human-led and artificial intelligence-automated critical appraisal of evidence. Background Critical appraisal is essential in evidence-based practice, yet many nurses lack the skills to perform it. Large language models offer potential support, but their role in critical appraisal remains underexplored. Design We conducted a comparative study to evaluate the performance of five commonly used large language models versus two human reviewers in appraising four systematic reviews on interventions to reduce medication administration errors. Methods We compared large language models and two human reviewers in independently appraising four systematic reviews using the JBI Critical Appraisal Checklist. These models were Perplexity Sonar (Pro), Claude 3.7 Sonnet, Gemini 2.0 Flash, GPT-4.5 and Grok-2. All models received identical full texts and standardized prompts. Responses were analyzed descriptively and agreement was assessed using Cohen’s Kappa. Results Large language models showed full agreement with human reviewers on five of 11 JBI items. Most disagreements occurred in appraising search strategy, inclusion criteria and publication bias. The agreement between human reviewers and large language models ranged from slight to moderate. The highest level of agreement was observed with Claude (κ = 0.732), while the lowest level was observed with Gemini (κ = 0.394). Conclusion Large language models can support aspects of critical appraisal evidence but lack contextual reasoning and methodological insight required for complex judgments. While Claude 3.7 Sonnet aligned most closely with human reviewers, human oversight remains essential. Large language models should serve as adjuncts and not substitutes for evidence-based practice. Keywords: artificial intelligence in healthcare, multimodal large language models, nursing, evidence-based practice Published in DKUM: 12.11.2025; Views: 0; Downloads: 1
Link to file |
2. Skrb za pacienta in tehnološka kompetentnost pri uporabi elektronskega zapisa zdravstvene nege v bolnišničnem okoljuCvetka Krel, 2025, doctoral dissertation Abstract: Uvod: Za zagotavljanje kakovostne in varne obravnave je poleg skrbi za pacienta pomembno tudi dokumentiranje obravnave. Učinkovitost elektronskega zapisa zdravstvene nege (EZZN) je pomemben element pri prepoznavanju in obravnavi pacientovih potreb, vendar pa lahko pomanjkljiva tehnološka kompetentnost zaposlenih v zdravstveni negi privede do pretiranega osredotočanja na uporabo EZZN, kar lahko negativno vpliva na skrbno obravnavo pacienta. Namen doktorske disertacije je bil raziskati učinkovitost EZZN, skrbna ravnanja zaposlenih in tehnološko kompetentnost v zdravstveni negi ter ugotoviti povezave med njimi pri uporabi EZZN.
Metode: Uporabili smo zaporedni pojasnjevalni načrt raziskav mešanih metod, izveden med zaposlenimi v zdravstveni negi v štirih slovenskih bolnišnicah, ki uporabljajo enak EZZN. Raziskava je potekala od julija 2021 do julija 2023. Pred glavno raziskavo smo izvedli pilotno raziskavo. Kvantitativne podatke smo zbrali s tremi validiranimi vprašalniki ter jih analizirali z uporabo metod opisne in sklepne statistike. Analiza literature in nato kvantitativnega dela raziskave je bila osnova za oblikovanja vodila za intervju v kvalitativnem delu raziskave, ker smo podatke zbrali z delno strukturiranimi intervjuji. Za analizo podatkov smo uporabili metodo utemeljene teorije. Kvantitativni in kvalitativni del raziskave smo združili v fazi integracije, analizirani kvalitativni rezultati so nam služili kot pomoč pri razlagi ali nadgradnji kvantitativnih rezultatov.
Rezultati: Učinkovitost EZZN je statistično značilno pozitivno povezana s časom od začetka njegove implementacije v bolnišnično okolje (p < 0,05) in stopnjo zaznavanja tehnološke kompetentnosti v zdravstveni negi (p < 0,05). Zaznana tehnološka kompetentnost je pozitivno povezana z zaznavanjem skrbnih ravnanj v bolnišničnem okolju, kjer se uporablja EZZN (p < 0,05). V okviru kvalitativne raziskave smo oblikovali glavno kategorijo, sedem kategorij in petnajst podkategorij. Model je vključeval vzročne pogoje (pomanjkljiva učinkovitost EZZN), kontekstualne pogoje (vključitev EZZN v bolnišnično okolje) in posredne pogoje (obravnava pacienta), ki vplivajo na pojav pomanjkljivega dokumentiranja individualne in celostne obravnave. Oblikovali smo akcijske/integracijske strategije za obvladovanje tega pojava, ki vključujejo tehnološko kompetentnost, skrb za pacienta in dokumentiranje, z rezultati v obliki izboljšane osredotočenosti na uporabo EZZN pri skrbi za pacienta in dokumentiranju celostne obravnave. Po integraciji kvantitativnih in kvalitativnih podatkov smo identificirali pet komponent: (1) pomanjkljivosti pri implementaciji v bolnišničnem okolju, (2) prednosti in pomanjkljivosti uporabe EZZN, (3) učinkovitost EZZN, (4) tehnološko kompetentna uporaba EZZN za dokumentiranje individualne in celostne obravnave pacienta, (5) skrb za pacienta in dokumentiranje obravnave.
Razprava in sklep: Uporaba EZZN v bolnišničnem okolju je razkrila tako prednosti kot pomanjkljivosti uporabe, ki so večinoma povezane s pomanjkljivo implementacijo EZZN. Sinergija med skrbjo za pacienta, tehnološko kompetentnostjo in učinkovitostjo EZZN je ključna za ustrezno dokumentiranje individualne in celostne obravnave pacienta. Vrzeli v teh komponentah lahko vodijo do pomanjkljivosti v obravnavi pacientov in nepopolnemu dokumentiranju z uporabo EZZN.
Keywords: skrb za pacienta, učinkovitost elektronskega zapisa, raziskave mešanih metod, tehnološke kompetence, zdravstvena nega Published in DKUM: 20.10.2025; Views: 0; Downloads: 19
Full text (4,92 MB) |
3. Rethinking realities: a call for accurate terminology in eXtended reality studiesNino Fijačko, Gregor Štiglic, Christina Gsaxner, Todd P. Chang, Robert Greif, 2024, other scientific articles Keywords: resuscitation education, virtual technology, XR terminology, new terminologies, reality studies Published in DKUM: 15.09.2025; Views: 0; Downloads: 2
Full text (616,82 KB) |
4. Bibliometrična analiza raziskav o spletnih vplivnežih na področju duševnega zdravjaTamara Trajbarič, 2025, master's thesis Abstract: Uvod: Duševno zdravje predstavlja ključno področje javnega zdravja, še posebej med mladimi, ki pogosto iščejo podporo prek družbenih omrežij. Spletni vplivneži imajo pri tem vse večjo vlogo, a pogosto brez ustrezne strokovne usposobljenosti, kar odpira številna tveganja. Namen raziskave je bil z bibliometrično analizo raziskati trende raziskav o spletnih vplivnežih na področju duševnega zdravja.
Metode: Izvedena je bila bibliometrična analiza del iz različnih podatkovnih baz med leti 2020 in 2025. Podatke smo analizirali z uporabo orodij Bibliometrix, VOSviewer in RStudio. Analiza je obsegala avtorje, institucije, revije, ključne besede, citiranost teh, trende objav ter raziskovalna in tematska žarišča.
Rezultati: Bibliometrična analiza 17 del iz 16 virov je pokazala, da sta najdejavnejša avtorja Ella White in Terry Hanley. Po citiranosti izstopa revija Internet Intervention, po število prispevkov je najvplivnejša institucija University College Dublin. Prispevki prihajajo iz Združenega kraljestva, Irske, Združenih držav Amerike in Avstrije. Raziskovalna dejavnost je razpršena, brez mednarodnega povezovanja. Ključna raziskovalna in tematska žarišča so vpliv družbenih omrežij na duševno zdravje, mladostniki ter psihološka podpora in socialna pravičnost.
Razprava in zaključek: Raziskovalna dejavnost o temi spletnih vplivnežev na področju duševnega zdravja je še v razvoju. Prisotna sta omejeno mednarodno povezovanje in šibko medinstitucionalno sodelovanje. Keywords: bibliometrija, družbena omrežja, duševno blagostanje Published in DKUM: 21.08.2025; Views: 0; Downloads: 33
Full text (2,51 MB) |
5. Učenje nazalne aplikacije naloksona v navidezni resničnostiMihec Korpič Lesjak, 2025, undergraduate thesis Abstract: Leta 2021 je prepovedane substance uporabljalo približno 296 milijonov ljudi . Stopnja umrljivosti zaradi drog je znašala 18,3 smrti na milijon prebivalcev v Evropski Uniji, 44 v Sloveniji in 461,8 v Združenih državah Amerike. Številne prevelike odmerke bi lahko preprečili z izobraževanjem uporabnikov, njihovih družin in skupnosti. Če laiki prepoznajo predoziranje, lahko žrtvi aplicirajo nalokson v obliki nazalnega pršila, ki hitro odpravi učinke opioidov.
V tej eksperimentalni študiji smo raziskovali, ali je uporaba NR primerna za učenje laikov nazalne aplikacije naloksona.
Ugotovili smo, da se je z naloksonom srečalo 56 % (13) vseh udeležencev, z uporabo NR pa le 30 % (7). Vsi udeleženci raziskave so se strinjali s tem, da je NR zanimiv način za učenje aplikacije naloksona in hkrati enostaven za uporabo. Resna igra, ki smo jo uporabili, ne sledi smernicam za prvo pomoč.
Izobraževanje z uporabo NR se je izkazalo kot učinkovito in pozitivno, saj omogoča praktično učenje brez potrebe po stalnem nadzoru inštruktorja. Študenti so NR ocenili kot enostavno za uporabo, brez večjih težav s slabostjo.
Ugotovitve kažejo, da ima NR velik potencial kot učni pripomoček, vendar je za zagotavljanje varne in pravilne oskrbe nujno vsebino uskladiti z uradnimi smernicami. Keywords: nalokson, navidezna resničnost, predoziranje z opioidi Published in DKUM: 21.08.2025; Views: 0; Downloads: 46
Full text (1,20 MB) |
6. Classifying the information needs of survivors of domestic violence in online health communities using large language models : prediction model development and evaluation studyShaowei Guan, Vivian Hui, Gregor Štiglic, Rose Eva Constantino, Young Ji Lee, Arkers Kwan Ching Wong, 2025, original scientific article Abstract: Background: Domestic violence (DV) is a significant public health concern affecting the physical and mental well-being of
numerous women, imposing a substantial health care burden. However, women facing DV often encounter barriers to seeking
in-person help due to stigma, shame, and embarrassment. As a result, many survivors of DV turn to online health communities
as a safe and anonymous space to share their experiences and seek support. Understanding the information needs of survivors of
DV in online health communities through multiclass classification is crucial for providing timely and appropriate support.
Objective: The objective was to develop a fine-tuned large language model (LLM) that can provide fast and accurate predictions
of the information needs of survivors of DV from their online posts, enabling health care professionals to offer timely and
personalized assistance.
Methods: We collected 294 posts from Reddit subcommunities focused on DV shared by women aged ≥18 years who
self-identified as experiencing intimate partner violence. We identified 8 types of information needs: shelters/DV centers/agencies;
legal; childbearing; police; DV report procedure/documentation; safety planning; DV knowledge; and communication. Data
augmentation was applied using GPT-3.5 to expand our dataset to 2216 samples by generating 1922 additional posts that imitated
the existing data. We adopted a progressive training strategy to fine-tune GPT-3.5 for multiclass text classification using 2032
posts. We trained the model on 1 class at a time, monitoring performance closely. When suboptimal results were observed, we
generated additional samples of the misclassified ones to give them more attention. We reserved 184 posts for internal testing
and 74 for external validation. Model performance was evaluated using accuracy, recall, precision, and F1
-score, along with CIs
for each metric.
Results: Using 40 real posts and 144 artificial intelligence–generated posts as the test dataset, our model achieved an F1
-score
of 70.49% (95% CI 60.63%-80.35%) for real posts, outperforming the original GPT-3.5 and GPT-4, fine-tuned Llama 2-7B and
Llama 3-8B, and long short-term memory. On artificial intelligence–generated posts, our model attained an F1
-score of 84.58%
(95% CI 80.38%-88.78%), surpassing all baselines. When tested on an external validation dataset (n=74), the model achieved
an F1
-score of 59.67% (95% CI 51.86%-67.49%), outperforming other models. Statistical analysis revealed that our model significantly outperformed the others in F1
-score (P=.047 for real posts; P<.001 for external validation posts). Furthermore, our
model was faster, taking 19.108 seconds for predictions versus 1150 seconds for manual assessment.
Conclusions: Our fine-tuned LLM can accurately and efficiently extract and identify DV-related information needs through
multiclass classification from online posts. In addition, we used LLM-based data augmentation techniques to overcome the
limitations of a relatively small and imbalanced dataset. By generating timely and accurate predictions, we can empower health
care professionals to provide rapid and suitable assistance to survivors of DV. Keywords: domestic violence, online health communities, large language models, generative artificial intelligence, artificial intelligence Published in DKUM: 22.07.2025; Views: 0; Downloads: 4
Full text (780,00 KB) This document has many files! More... |
7. |
8. PICOT questions and search strategies formulation: a novel approach using artificial intelligence automationLucija Gosak, Gregor Štiglic, Lisiane Pruinelli, Dominika Vrbnjak, 2025, original scientific article Abstract: Aim
The aim of this study was to evaluate and compare artificial intelligence (AI)-based large language models (LLMs) (ChatGPT-3.5, Bing, and Bard) with human-based formulations in generating relevant clinical queries, using comprehensive methodological evaluations.
Methods
To interact with the major LLMs ChatGPT-3.5, Bing Chat, and Google Bard, scripts and prompts were designed to formulate PICOT (population, intervention, comparison, outcome, time) clinical questions and search strategies. Quality of the LLMs responses was assessed using a descriptive approach and independent assessment by two researchers. To determine the number of hits, PubMed, Web of Science, Cochrane Library, and CINAHL Ultimate search results were imported separately, without search restrictions, with the search strings generated by the three LLMs and an additional one by the expert. Hits from one of the scenarios were also exported for relevance evaluation. The use of a single scenario was chosen to provide a focused analysis. Cronbach's alpha and intraclass correlation coefficient (ICC) were also calculated.
Results
In five different scenarios, ChatGPT-3.5 generated 11,859 hits, Bing 1,376,854, Bard 16,583, and an expert 5919 hits. We then used the first scenario to assess the relevance of the obtained results. The human expert search approach resulted in 65.22% (56/105) relevant articles. Bing was the most accurate AI-based LLM with 70.79% (63/89), followed by ChatGPT-3.5 with 21.05% (12/45), and Bard with 13.29% (42/316) relevant hits. Based on the assessment of two evaluators, ChatGPT-3.5 received the highest score (M = 48.50; SD = 0.71). Results showed a high level of agreement between the two evaluators. Although ChatGPT-3.5 showed a lower percentage of relevant hits compared to Bing, this reflects the nuanced evaluation criteria, where the subjective evaluation prioritized contextual accuracy and quality over mere relevance.
Conclusion
This study provides valuable insights into the ability of LLMs to formulate PICOT clinical questions and search strategies. AI-based LLMs, such as ChatGPT-3.5, demonstrate significant potential for augmenting clinical workflows, improving clinical query development, and supporting search strategies. However, the findings also highlight limitations that necessitate further refinement and continued human oversight.
Clinical Relevance
AI could assist nurses in formulating PICOT clinical questions and search strategies. AI-based LLMs offer valuable support to healthcare professionals by improving the structure of clinical questions and enhancing search strategies, thereby significantly increasing the efficiency of information retrieval. Keywords: PICOT question, search strategies, artificial intelligence Published in DKUM: 21.07.2025; Views: 0; Downloads: 6
Full text (527,46 KB) This document has many files! More... |
9. Healthcare student engagement with health-related content through Netflix mini-series : a qualitative studyNino Fijačko, Špela Metličar, Gregor Štiglic, Lucija Gosak, Roger Watson, 2025, original scientific article Abstract: Background Conventional lecture-based teaching often struggles to keep students engaged and attentive throughout the class. Television and film, when used as educational tools, can promote a compassionate, person-centered approach to healthcare. Film-based education enhances critical thinking and allows students safely to explore and express their professional values and beliefs. This qualitative research aimed to explore how a Netflix mini-series could engage healthcare students and enhance their interest in learning, rather than its direct instructional effectiveness. Method In September 2023, the “Simulation and Games in Nursing Education” international summer school featured daily episodes from “The Nurse” Netflix mini-series, each lasting 40 to 60 min. Healthcare students attended this event as part of their summer school program. We employed a six-step thematic analysis framework to analyze the collected data and Sankey diagrams were used for visualization. Results Our research involved 28 healthcare students from six European countries, predominantly women (82%, 23/28), with a mean age of 22 years (SD 2.35). None had seen the series before the summer school. The analysis revealed ‘Cinenurducation’ as the main theme, supported by three subthemes: (1) Observational learning and Critical thinking; (2) Cinenurducation feelings; and (3) Relationships and Personal behaviors. Conclusion While Netflix mini-series can provide engagement opportunities, this research underscores the importance of distinguishing between engagement and educational effectiveness in using media as a teaching tool. Clinical trial number None. Keywords: Netflix, drama, serial killer, learning, healthcare, nursing Published in DKUM: 30.06.2025; Views: 0; Downloads: 4
Full text (1,85 MB) |
10. Učinkovitost uporabe mobilne aplikacije za samooskrbo v vodenju pacienta s sladkorno boleznijo tipa 2Lucija Gosak, 2025, doctoral dissertation Abstract: Uvod: Sladkorna bolezen tipa 2 postaja vse večji javnozdravstveni izziv in zahteva upoštevanje kompleksnega režima zdravljenja. V sodobni zdravstveni oskrbi se poudarja aktivna odgovornost pacientov za samooskrbo bolezni, kar vključuje sprejemanje zdravih življenjskih navad. Z rastjo digitalne tehnologije se povečuje tudi število mobilnih aplikacij, namenjenih samooskrbi. Namen doktorske disertacije je oceniti učinkovitost uporabe mobilne aplikacije za izboljšanje samooskrbe pacientov s sladkorno boleznijo tipa 2.
Metode: Izvedli smo sistematično iskanje mobilnih aplikacij v spletnih trgovinah Google Play Store in iPadian. Funkcije vedenja samooskrbe so bile ocenjene v skladu z okvirjem AADE7, kakovost pa je bila ocenjena z uporabo lestvice uMARS. Uporabili smo dva anketna vprašalnika, "Self-Care of Diabetes Inventory" za oceno samooskrbe in "Brief Illness Perception Questionnaire" za oceno percepcije bolezni, ki sta bila validirana v
pilotni raziskavi. V randomizirani klinični raziskavi smo ocenili vpliv uporabe mobilne aplikacije na izboljšanje samooskrbe, percepcije bolezni in zdravstvenega stanja pacienta. Pred začetkom raziskave in po štirih tednih so pacienti izpolnili oba anketna vprašalnika, ter izvedli klinične meritve. V končnem delu so sledili delno strukturirani intervjuji z medicinskimi sestrami in pacienti.
Rezultati: Mobilna aplikacija forDiabetes: diabetes self-management app je bila izbrana kot najprimernejša za nadaljnjo raziskavo. V pilotni raziskavi je sodelovalo 141 pacientov z diagnosticirano sladkorno boleznijo tipa 2. Svojo samozavest za izvajanje samooskrbe so ocenili z najvišjo povprečno oceno, medtem ko so najnižje ocenili upravljanje samooskrbe. Anketni vprašalnik Self-Care of Diabetes Inventory je bil na podlagi rezultatov pilotne raziskave slabo usklajen z metrikami uporabniškega modela za spremljanje samooskrbe, samozavest in upravljanje samooskrbe, vendar je CFA za vzdrževanje samooskrbe pokazal dobro ujemanje. Cronbachova alfa je pokazala odlične rezultate za samozavest, dobre za spremljanje, sprejemljive za vzdrževanje in šibke za upravljanje samooskrbe. Izračuni za vprašalnik Brief Illness Perception Questionnaire so pokazali visoko stopnjo strinjanja med ocenjevalci in odlično veljavnost vprašanj.
Cronbachova alfa za zanesljivost je bila 0,652, kar kaže na zadovoljivo notranjo skladnost. V glavni raziskavi štiri tedne po uporabi mobilne aplikacije ni bilo mogoče zaznati statistično pomembnega izboljšanja v samooskrbi, percepciji bolezni in kliničnih meritvah. Na podlagi kvalitativnega dela raziskave smo ugotovili, da imajo zdravstveni delavci in pacienti pozitiven pogled na uporabo mobilne aplikacije ter da jim je uporaba pripomogla k večji doslednosti pri spremljanju svoje bolezni in lažji vpogled v meritve. Na podlagi pridobljenih rezultatov smo adaptirali teorijo srednjega obsega za samooskrbo kroničnih bolezni v povezavi z mobilnim zdravjem in dodatno vključili naslednje komponente: znanje, doslednost, karakteristike mobilnega zdravja in mobilne aplikacije, varovanje in integriteta pacientovih podatkov, izidi pri pacientu, priporočene značilnosti in funkcionalnosti mobilne aplikacije ter usmeritve za implementacijo mobilnega zdravja v zdravstveno oskrbo.
Razprava in sklep: V doktorski raziskavi smo ugotovili, da pri pacientih s sladkorno boleznijo tipa 2 prihaja do odstopanj v samooskrbi. Čeprav vključitev mobilnih aplikacij v samooskrbo ni prinesla klinično pomembnih rezultatov, so tako zdravstveni delavci kot pacienti izrazili pozitiven pogled na njihovo uporabo. Ugotovitve raziskave nakazujejo potrebo po uvedbi enotnih smernic, ki bi opredeljevale standarde za uporabo mobilnega
zdravja v vseh zdravstvenih institucijah v državi. Za natančnejšo oceno dolgoročnega učinka uporabe mobilne aplikacije na samooskrbo, percepcijo bolezni in zdravstveno stanje pacientov bi bilo smiselno izvesti raziskave na večjem vzorcu in v daljšem časovnem obdobju. Keywords: sladkorna bolezen tipa 2, mobilno zdravje, zdravstvene mobilne aplikacije, samooskrba, samoupravljanje bolezni Published in DKUM: 17.06.2025; Views: 0; Downloads: 82
Full text (8,77 MB) |