Opinion of Healthcare Organization Employees on the Usefulness, Ease of Use, and Effectiveness of Electronic-Learning for Continuous Professional Development

INTRODUCTION

Continuous professional development (CPD) in healthcare is essential to empower practitioners to gain and apply the skills necessary for providing effective care. This involves practitioners regularly refreshing their skills to align with evolving healthcare needs, emerging care delivery models, and changing public expectations1,2.

Modern educational technologies, such as electronic learning, provide resources that could aid practicing healthcare professionals in their ongoing CPD3. Electronic learning (e-learning) refers to training conducted via digital means, encompassing any instructional delivery to trainees through e-learning systems utilizing modern Information and Communication Technologies4,5. It is referred to by different names such as web-based learning, online education, computer-assisted instruction, and virtual training4,6. The use of e-learning for CPD has significantly increased lately. For instance, e-learning activities accounted for 43% of physician CPD learning in 2017, and for 45% by 2019 in the United States of America6. During the era of COVID-19, there was a rapid surge in demand for e-learning in the form of virtual and distance training in response to lockdown measures5,6.

Saudi Arabia embraced e-learning in the early 1990s and expanded its use in the subsequent decades with the advancement of computer technology7. During the COVID-19 pandemic in Saudi Arabia, there was a rise in the adoption of e-learning, with various online learning management systems utilized to help achieve educational goals8.

The General Directorate of Health Affairs (GDHA) in the Riyadh Region is responsible for overseeing, monitoring, and evaluating the delivery of health services in the region. Its workforce comprises both healthcare professionals, as well as non-health professional staff. The responsibilities of healthcare professionals include supervising healthcare provision across different levels of care at health facilities and developing and delivering training programs for healthcare providers. Therefore, it is crucial for them to stay updated on rapidly produced medical knowledge and advancements in healthcare to effectively fulfill their roles. The Training and Institutional Development Department (TIDD) at the GDHA is responsible for the CPD of its employees. In response to the COVID-19 pandemic and beyond, TIDD introduced e-learning in addition to the traditional class-based training activities.

Well-designed e-learning medical education programs not only yield knowledge comparable to traditional classroom instruction but also result in significant self-reported changes in professional practice9. Moreover, E-learning reduces training time, lowers delivery costs, boosts accessibility, and provides study flexibility4,10,11. However, to realize its potential as an effective mode of training delivery, e-learning must be embraced and utilized by its intended users, which depends on their opinions regarding its usefulness, ease of use, and effectiveness12.

Perceived usefulness refers to the subjective probability of a prospective user that utilizing a specific application system will enhance their job performance within an organizational setting. Perceived ease of use pertains to the prospective user's expectation of the target system being effortless to use13. According to Kirkpatrick (1996), training effectiveness is the extent to which training objectives are achieved and beneficial for the organization and trainees14.

Numerous studies conducted in Saudi Arabia exploring the attitudes and satisfaction of various healthcare professionals, as well as medical students, regarding e-learning, consistently report favorable attitudes towards it8,15,16. However, these studies were conducted among health professionals delivering care at health facilities, with no studies conducted among healthcare professionals or individuals from other professions tasked with supervising and monitoring healthcare delivery, as well as delivering training to practicing healthcare providers.

Transitioning to e-learning for the delivery of CPD activities, the TIDD must assess the opinions of GDHA employees regarding e-learning, which will provide valuable insights into employee experiences, and potential challenges, thereby guiding decisions regarding the potential expansion of e-learning initiatives. This study aimed to evaluate the opinions of healthcare and other professionals at GDHA in the Riyadh Region regarding the usefulness, ease of use, and effectiveness of e-learning for CPD training.

METHODS

Design and setting

This study was an institution-based, descriptive, cross-sectional study conducted in the GDHA in the Riyadh Region, which comprises seven directorates and employs around 2000 individuals. The TIDD organizes two types of e-learning CPD activities: 1) Live synchronous online CPD training activities delivered virtually at a specific pre-scheduled time, using widely used electronic platforms such as Zoom or Microsoft Teams. 2) Non-synchronous e-learning or Internet enduring materials offered via three government-accredited electronic training platforms. In this latter type, participants can choose when to attend the training, as there is no fixed time or location requirement for participation6.

Participants

The study population consisted of employees who have taken part in the e-learning CPD training programs organized by the TIDD. The employees are categorized into two groups: 1) healthcare professionals, including physicians, dentists, pharmacists, nurses, and other allied healthcare professionals; 2) non-healthcare professionals such as human resources management, information technology, law, engineering, and clerical roles. The sample size of 217 participants was calculated using the online calculator of the open Epi Program (version 3.01-2013), assuming a total study population of 1000, a 95% confidence level, a 5% confidence limit, and a 50% frequency of outcome. The participants were selected using the stratified sampling technique to cover the seven general directorates in the RHD, where the study participants were selected using the systematic random sampling technique.

Procedure and ethical considerations

The study protocol was approved by the Institutional Review Board of King Fahad Medical City (IRB Log Number: 24-L26E), and Princess Nourah Bint Abdulrahman University (RB Log Number: 24-0571). Informed consent was obtained from the study participants.

Measures

The data were collected using a self-administered, structured, pre-coded questionnaire designed with the Google Forms application. The questionnaire was adapted from validated instruments used in previous studies to evaluate the effectiveness, ease of use, and usefulness of e-learning, information technology, and training in general17-20. The questionnaire consisted of four sections: Section 1 covered background information, Section 2 included variables assessing the effectiveness of received e-learning CPD training (seven items). Responses for the first two items were rated on a 4-point Likert scale (great, somewhat, small, no), while the remaining five variables used a 3-point Likert scale (yes to a great extent, yes to some extent, no). Section 3 encompassed variables related to the usefulness of e-learning (ten items) and the ease of use of e-learning (five items), measured on a five-point Likert scale (very strongly agree, strongly agree, neutral, disagree, very strongly disagree). The questionnaire was pilot-tested with a sample of 30 respondents, yielding Cronbach's Alpha values of 0.971, 0.941, and 0.865, and for items measuring the effectiveness, usefulness, and ease of use of e-learning, respectively

Statistical analysis

The data was exported from the Excel sheet into the Statistical Package for the Social Sciences (SPSS) version 21 file. The analysis involved descriptive statistics, calculating proportions for categorical variables and averages [mean (standard deviation)] for continuous variables. The opinions about the effectiveness, usefulness, and ease of use of e-learning for CPD training were computed by obtaining the average scores of the items measuring them and the overall average. Comparisons between subgroups were conducted using one-way analysis of variance and t-tests for means of independent samples. A p-value less than 0.05 at a 95% confidence interval was considered significant.

RESULTS

Two hundred and ten individuals from the target population completed the questionnaire, resulting in a response rate of 96.8%. The background information of the study participants is shown in Table 1. The majority of the respondents were male (70%), with over half of them (58.6%) under 45 years of age. Approximately two-thirds (68.6%) were health professionals, while the remaining third (31.4%) were from non-health professions. Additionally, 65.2% of respondents reported their participation in the training needs assessment within the past year.

Usefulness of e-learning

Table 2 presents the respondents' opinion on the usefulness of e-learning, yielding an overall score of 29 (8.5) out of 40 points. Specifically, favorable opinions were expressed towards the aspects of (e-learning requires no movement), (e-learning improve access to education), (e-learning saves time), and (e-learning is flexible and comfortable) with scores of 3.4(8), 3.4(9), 3.4(9), and 3.2(9) respectively. However, less favorable opinions were observed regarding items (e-learning enhances interaction with trainer), (e-learning enhances relations with trainer), and (e-learning enhances relations between trainees), which garnered scores of 2.6(1.1), 2.5(1.2), and 2.4(1.2) respectively. The lowest rating was associated with the opinion that (e-learning is effective for subjects requiring hands-on learning experiences) receiving a score of 2.3(1.3).

Ease of use of e-learning

Table 2 also shows respondents' opinions on the ease of use of e-learning, with an overall average score of 15.9(4.2) out of 20 points. Specifically, favorable views were expressed regarding (having the necessary equipment), (having adequate internet connection), and (having sufficient personal technical skills for online learning), scoring 3.3(.9), 3.3(1), and 3.2(.9) respectively. Conversely, a less positive opinion was noted regarding the availability of a suitable private location for e-learning, achieving a score of 2.3(1.2).

Effectiveness of e-learning

Table 3 presents respondents' opinions on the effectiveness of e-learning they received in the previous year, yielding an overall effectiveness score of 14.6(5.6) out of 16 points. Notably, all items achieved nearly similar scores.

Factors associated with usefulness, ease of use, and effectiveness of e-learning

Table 4 displays the factors influencing opinions on the usefulness, ease of use, and effectiveness of e-learning. Males held a more favorable opinion regarding the usefulness of e-learning compared to females (males: 30(8.3) versus females: 27(8.6) – p-value = 0.010). On the other hand, age group and type of profession were not found to be associated with opinions regarding usefulness. Likewise, males expressed a more positive opinion regarding the ease of use of e-learning compared to females (males: 16.7(3.8) versus females: 13.9(4.6) – p-value = 0.000). However, similar to opinions on usefulness, neither age group nor the type of profession were associated with perceptions of ease.

Individuals in the age group 45 – 49 years expressed a more favorable opinion about the effectiveness of e-learning compared to those under 45 years of age (45-49 years: 15.8(5.1) versus <45 years: 13.8(5.9) – p-value = 0.014). However, neither sex nor type of profession were associated with opinions regarding effectiveness.

DISCUSSION

This research was conducted to examine the viewpoints of healthcare workers on the implementation of e-learning for continuous professional development (CPD) training.This study revealed that employees at GDHA, including health professionals and other professionals, generally held favorable opinions regarding the usefulness, ease of use, and effectiveness of online e-learning activities for CPD. This finding aligns with similar reports from studies conducted in Saudi Arabia and other countries, indicating that various healthcare workers find e-learning to be a useful and feasible learning method3,8,11,15,16,21. The learner's opinion holds significance as they are central to the learning process, and their opinion influences learning efficiency, motivation, and knowledge application21. The continuous evaluation of learners' opinions becomes particularly relevant amidst the rapid evolution of technology and information resources and emerging learning technologies11,15.

In recent decades, e-learning has proven its capacity to deliver consistent education, shorten instruction times, boost learning effectiveness and efficiency, and enhance learner motivation and satisfaction6,11,22. These benefits were particularly notable for CPD, especially for individuals like healthcare workers who have various competing interests demanding their time6,8. For instance, flexibility in terms of time and space was acknowledged by our respondents as a strength of e-learning. Convenience, encompassing easy access, reduced travel, and scheduling flexibility, was identified as the main advantage of e-learning in most studies evaluating this learning method6. This makes it more suitable than the class-based model as it can better accommodate the constraints of learners' work, serving as an appropriate learning approach for CPD, especially for those with limited time11,21,23,24. The respondents also perceived e-learning as efficient in enabling more trainees to attend specific training, a benefit seen in both CPD for healthcare professionals and undergraduate studies. This increase in attendance numbers is an advantage of e-learning that is not attainable with class-based training6,11,25.

The opinion of the respondents was also favorable regarding the ease of use of e-learning, particularly concerning the availability of internet connection and equipment. This is facilitated by the well-maintained and continuously expanding technological infrastructure needed for e-learning in Saudi Arabia12. All employees at GDHA in Riyadh are provided with their own computers equipped with reliable, continuous internet connections. This can significantly influence an individual's willingness to engage in and complete e-learning activities21. The respondents also perceived that they possess a sufficient level of personal information technology (IT) skills, enabling them to participate in e-learning without encountering technological barriers. This perception is consistent with reports from Saudi Arabia and other countries, where technology has not hindered e-learning3,12. Having IT skills is crucial for participating in web-based learning programs6,12,21, as the absence of such skills often leads health professionals to perceive e-learning as challenging and contributes to fostering a negative attitude towards e-learning4,17,21.

As crucial as assessing perceptions of usefulness and ease of use is, evaluating the outcomes and benefits of online CPD is essential, including determining the extent to which trainees' performance is enhanced12. The respondents perceived e-learning as effective in equipping them with knowledge and skills, thereby enhancing their performance. Likewise, other studies in Saudi Arabia have also reported that online CPD effectively enhances the clinical performance of various healthcare professionals8,26. Literature from other countries also suggests that virtual CPD activities are considered comparable in quality to face-to-face learning11and have the potential to improve physicians' knowledge, clinical care, and health outcomes for patients6,27

While the respondents generally held a positive opinion, their attitude was less favorable towards specific aspects of e-learning, that should be considered when planning the expansion of e-learning CPD activities..

The respondents showed concerns about the effectiveness of e-learning for subjects that require hands-on experiences, a concern shared by respondents in other studies, especially undergraduate medical students who believe that face-to-face training is essential to provide practical experience to students25,28. However, within the context of CPD, professionals may need to enhance or update their existing skills, and the e-learning approach may suffice for achieving this. This is particularly noteworthy given that technological advancements enable learners to utilize interactive features and multimedia tools on the internet, allowing them to simulate skills in authentic work settings6. Nevertheless, this necessitates careful planning by the TIDD for CPD training activities, deciding which ones can be delivered more effectively online, which ones are better suited for onsite delivery, or a combination of both.

Similarly, the opinion of our respondents was less favorable regarding the impact of e-learning on interaction with trainers and other trainees. Interestingly the literature reported conflicting views regarding this aspect of e-learning. Several studies have cited the absence of interactions, active involvement, and prompt feedback as significant disadvantages of e-learning for CPD6,11,21,29. On the other hand, other studies have reported that e-learning is interactive, enables active participation and immediate feedback, and enhances communication with learners from diverse locations6. Moreover, according to trainers' perspectives, technology promotes participation from learners who may have been hesitant in traditional learning. E-learning offers equal discussion opportunities, unlike traditional settings where ensuring equal participation is challenging due to the 'safer' environment22. It appears that it is not the mode of delivery of training (face-to-face or virtual) that affects interaction; rather, it is the mode of teaching adopted by the trainer, whether didactic or interactive, considering that computer-based programs offer the potential for interactivity27. To boost e-learning interaction, recommended strategies include emphasizing "two-way interaction" in course design, integrating interactive question and answer sessions, setting up online discussion areas, and forming web-based learning communities21,30.

The predominant concern expressed by respondents in this study regarding the usability of e-learning was the availability of an appropriate, private space at their workplace to engage in e-learning activities without interruptions. A conducive learning environment is crucial as it can impact an individual's motivation to engage in and complete distance learning courses21. A specific office space could be designated for individuals who wish to attend e-learning, where they can participate in training without distractions.

Multiple studies have indicated that the adoption of online learning is significantly influenced by the characteristics of learners4. In the current study, males held a more favorable opinion regarding the usefulness and ease of e-learning compared to females. There is a discrepancy in the study results concerning the perception and use of e-learning by males and females. Some studies report that females use online CPD more than males27, while others agree with our findings that females are less inclined to use e-learning compared to males11. This is attributed to better computer literacy among males compared to females11,31 in addition to females having personal commitments that compete with their pursuit of e-learning for CPD6. This study also found that older employees perceive e-learning as more effective in providing and enhancing knowledge and improving performance compared to their younger counterparts. This finding contrasts with reports from other studies that indicate a better perception and use of e-learning among younger healthcare professionals8,27. The reason could be that older healthcare professionals, who may have experience with traditional forms of continuing education such as in-person conferences and seminars, may perceive e-learning as adequate for their CPD needs8.

No difference was observed in the opinions of health and non-health professionals regarding the usefulness, ease of use, or effectiveness of e-learning. This suggests that the benefits and limitations of e-learning are universally perceived, regardless of professional background. It also implies that e-learning programs are designed and implemented effectively to meet the needs and preferences of both groups, resulting in consistent perceptions among them.

A limitation of this study is that the findings may have limited generalizability beyond GDHA in the Riyadh Region. However, the context and characteristics are similar to other healthcare organizations and institutions within Saudi Arabia.

CONCLUSION

The respondents in this study expressed a favorable view of e-learning for CPD, highlighting its usefulness, ease of use, and effectiveness. They found it convenient and efficient, and appreciated the availability of necessary IT skills, internet access, and equipment. Moreover, they acknowledged its effectiveness in knowledge acquisition and performance improvement. However, they showed concerns about the level of interaction, suitability for hands-on training, and availability of distraction-free locations. Therefore, e-learning stands out as a viable option for implementing CPD training; however, this choice requires careful planning and considerations to effectively address and overcome any shortcomings or limitations associated with this mode of learning.

REFERENCES

1. Mlambo M, Silén C, McGrath C. Lifelong learning and nurses\' continuing professional development, a metasynthesis of the literature. BMC Nurs. 2021;20(1):62. doi:10.1186/s12912-021-00579-2

1. Manley K, Martin A, Jackson C, Wright T. Nurse Education Today A realist synthesis of effective continuing professional development ( CPD ): A case study of healthcare practitioners ' CPD. Nurse Educ Today. Nurse Educ Today. 2018; 69:134-141. doi: 10. 1016/j. nedt.2018.07.010.Epub2018 Jul24.

1. Cook DA, Blachman MJ, Price DW, West CP, Thomas BLB, Berger RA, et al. Educational Technologies for Physician Continuous Professional Development : A National Survey. Acad Med. 2018;93(1):104–112. doi: 101097/ACM.0000000000001817

1. Ullah O, Khan W, Khan A. Students’ Attitude towards Online Learning at Tertiary Level. Humanities and Social Sciences. 2017; 25(1-2):63-82

1. Hashem M, Sfeir E, Hejase HJ, Hejase AJ. Effect of Online Training on Employee Engagement During the COVID-19 Era. Asian Business Research. 2022; 7(5):10-40.

1. Cheng C, Papadakos J, Umakanthan B, Fazelzad R, Martimianakis MAT, Ugas M, et al.. On the advantages and disadvantages of virtual continuing medical education: a scoping review. Can Med Educ J. 2023;14(3):41-74.

1. Al Shamari D. Challenges and barriers to e-learning experienced by trainers and training coordinators in the Ministry of Health in Saudi Arabia during the COVID-19 crisis. PLoS ONE. 2022; 17(10). doi.org/10.1371/journal.pone.0274816

1. Shalabi KM, Almurdi MM. Satisfaction and attitudes towards online continuous medical education and its impact on clinical practice among physiotherapists. BMC Medical Education. 2024;24:70. doi.org/10.1186/s12909-024-05049-2

1. Aggarwal R, Gupte N, Kass N, Taylor H, Ali J, Bhan A, et al. A Comparison of Online versus On-site Training in Health Research Methodology : A Randomized Study. BMC Medical Education. 2011;11:37. doi: 10.1186/1472-6920-11-37

1. Berndt A, Murray CM, Kennedy K, Stanley MJ, Gilbert-hunt S. Effectiveness of distance learning strategies for continuing professional development ( CPD ) for rural allied health practitioners : a systematic review. BMC Medical Education. 2017; 17:117. DOI 10.1186/s12909-017-0949-5

1. Radovic-Markovic M. Advantages and Disadvantages of E-learning in comparison to Traditional Forms of Learning. Annals of the University of Petroşani, Economics. 2010; 10(2): 289- 298.

1. Alwadie AD. Online Continuing Medical Education in Saudi Arabia. Performance Improvement. 2013; 52(1): 24–28.

1. Davis FD, Bagozzi RP, Warshaw PR. User Acceptance of Computer Technology: A Comparison of Two Theoretical Models. Manage Sci. 1989;35(8):982–1003.

1. Kirkpatrick D. Great ideas revisited: revisiting Kirkpatrick's four-level model. Training and Development. 1996; 50: 54-57

1. Al-Sughayr AM, Al-Abdulwahhab BM, Al-Yemeni MR. Primary health care physicians\' knowledge, use, and attitude towards online continuous medical education in Saudi Arabia. Saudi Med J. 2010;31(9):1049-1053.

1. Khalil R, Mansour AE, Fadda WA, Almisnid K, Aldamegh M, Al-nafeesah A, et al. The sudden transition to synchronized online learning during the COVID-19 pandemic in Saudi Arabia : a qualitative study exploring medical students ’ perspectives. BMC Medical Education. 2020; 20:285. doi.org/10.1186/s12909-020-02208-z.

1. Mulhem A Al. Exploring the Key Factors in the Use of an E-Learning System Among Students at King Faisal University, Saudi Arabia. iJIM. 2020;14(3):19–37.

1. Davis FD, Davis F. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly.1989; 13(3):319-340

1. Pettersson D. E-learning acceptance in higher education A quantitative study comparing pre- and post-COVID-19 students' views on e-learning using the UTAUT model. Master [thesis]. Uppsala: Uppsala University; 2023.

1. Abbad GS, Borges-andrade JE, Sallorenzo LH. Self-Assessment of Training Impact at Work : Validation of a Measurement Scale. Revista Interamericana de Psicología/Interamerican Journal of Psychology. 2004; 38( 2):277-284

1. Yu S, Yang KF. Attitudes toward web-based distance learning among public health nurses in Taiwan: A questionnaire survey. Int J Nurs Stud. 2006;43(6):767–774.

1. Kenny A. Online learning: Enhancing nurse education? J Adv Nurs. 2002;38(2):127–35.

1. Kirkwood A. Understanding Independent Learners’ Use of Media Technologies. Open Learn. 2003;18(2):155–175.

1. Lyall R, McNamara S. Influences on the orientations to learning of distance education students in Australia. Open Learn. 2000;15(2):107–121.

1. Harvey LA, Glinsky J V., Lowe R, Lowe T. A Massive Open Online Course for teaching physiotherapy students and physiotherapists about spinal cord injuries. Spinal Cord. 2014;52(12):911–918..doi.org/10.1038/sc.2014.174

1. Nambi G, Abdelbasset W.K, Abodonya A, Saleh, A K, Khanam H. Effects of webinar based teaching program on therapists’ attitude and satisfaction of LBP patients. Arch PharmaPract. 2020;11(1):122-128..

1. Harris J. Women Physicians Are Early Adopters of on-Line. Journal of Continuing Education in the Health Professions. 2003;23:221-228.

1. Rajab MH, Gazal AM, Alkattan K. Challenges to Online Medical Education During the COVID-19 Pandemic. Cureus. 2020; 12(7): e8966. doi:10.7759/cureus.8966

1. Murphy D, Yum JCK. Understanding Hong Kong distance learners. Distance Educ. 1998;19(1):64–80.

1. Ismail II, Abdelkarim A, Al-Hashel JY. Physicians’ attitude towards webinars and online education amid COVID-19 pandemic: When less is more. PLoS ONE. 2021; 16(4): e0250241. https:// doi.org/10.1371/journal.pone.0250241

1. Washer P. Barriers to the use of web-based learning in nurse education. Nurse Educ Today. 2001;21(6):455–60.