Online Learning cum Evaluation Test Exercise
Crisis Communication Management
An Initial Report on What, How, and Cost-effectiveness
(I still have to include a feedback-survey from participants. – ROJ)
Reynaldo O. Joson, MD, MHA, MHPEd, MSc Surg
August 24, 2012
On July 23, 2012, I facilitated a seminar on crisis communication management to 46 senior and middle managers of Manila Doctors Hospital (MDH).
On that day, I tried out my online learning cum evaluation test exercise (OLETE) on crisis communication management.
The OLETE consisted of my making an online test through a freeware in the NET. In this case, I used the QuizStar (http://quizstar.4teachers.org). I made 70 objective questions (true or false and multiple choices) to facilitate instant correction and analysis by the software.
Here were my instructions on how to take the OLETE.
- Get into the Internet.
- URL: http://quizstar.4teachers.org/
- Under Student Login Page, click “Students, get started”
- Click “Sign-up”
- Place your first name; last name – middle name; username; passwords.
- Then, click “register.”
- Then login your username and password.
- Then click “Search”
- In the box, under quiz title, enter the following: “Crisis Communication Management for MDH Staff”
- Click “exact match” then “register.”
- Then, “go to test” then “take.”
- Then, take your test.
Before the seminar on July 23, 2012, I asked the participants to bring a laptop. I also asked Information Technology of MDH to ensure accessibility of WIFI in the conference room.
Out of the 46 participants who attended, only 34 were able to take the OLETE as some did not bring a laptop as instructed; some were still computer illiterate; and some had technical difficulties with their laptop.
After the July 23, 2012 session, I gave instructions for the 34 participants who took the test to continue taking the online test within 2 weeks until they get a perfect score based on the recommended answers. Follow-ups and reminders were done with the help of the staff of Quality Management Office, specifically, Francis Arcilla and Abigail Geralde. Technical computer assistance was provided by the Mr. Arcilla and Ms. Geralde. Mr. Roberto Alvaro, Jr. of the MDH-Pfizer Library also provided technical computer assistance when participants used the computers in the library.
Below shows the results as of August 17, 2012, the deadline set for the participants to get a perfect score based on recommended answers. The deadline was extended to 4 weeks.
Out of 34 participants who took the OLETE, 24 already got the perfect scores. Those who have not gotten a perfect score numbered 10. (They will be reminded to take again.)
Above shows the average attempts made, which is 3 out of 10 attempts allowed. The lowest score is 47 or 67%.
There were 10 who have not reached yet the perfect score (70). Once a participant reached a 100% score, he could opt not to repeat taking the test. In this initial offering, there were those who already reached 100% and when they took the tests, their score went down to below 100%. This is not taken against them. Main reasons why they retook the tests were they wanted to test the technical functionality of the software (see Joson) and they just did it just for fun (see Benito).
In this software, there is another useful information for the facilitator. This is the question analysis. This will help me, the facilitator, review the test questions especially in correcting any ambiguity that may be present.
Above shows how I designed the OLETE on Crisis Communication Management for MDH Staff.
- Ten (10) attempts were given to give allowance for technical online hitches. In the future, I will reduce the number to 5 attempts.
- Correct answers were given right after the test.
- Time allotted for the test was 60 minutes for the 70 objective questions (all were able to finish the test within this allotted time).
I gave the following instructions at the start of the OLETE last July 23, 2012:
1. Answer all questions.
2. Each question has only one answer.
3. Reference for correct answers can be seen in http://hospitalcrisiscommunicationmanagement.wordpress.com
4. The ultimate target is to get a 100% score.
5. You will be given 10 attempts to get a perfect score within 2 weeks. Don’t worry if you do not get 100% perfect score in the first and initial attempts.
6. After you have attained 100% score, you will be issued a certificate of proficiency. This certificate is valid for 3 years only. A re-certification is needed after 3 years.
Important Information: 1. There will be a “random testing” among those who have attained 100% score within the three year validity period of the certificate of proficiency. “Random testing” means administering the test (not necessarily the complete original test) on a randomly taken sample of those who have attained a certificate of proficiency. This random testing will be for administrative use only and will not affect the certificates previously issued.
2.This test will be accompanied by exercises (table-top, drill, and /or full-scale).
3. This test will also be accompanied by internal and external audits of practices and actual performances of staff on crisis communication management.
My Personal Reflections and Recommendations:
Over the years, I have been trying to find an efficient way of education for all staff of MDH. In the balanced scorecard of MDH, for the past three or four years or even earlier, we have been batting for a 100% attendance or participation in learning sessions in topics like disaster preparedness, quality management, basic life support, infection control, etc. We have difficulty achieving our targets. If we did, the strategies are not sustainable. Besides, there had been no measurement on how much the participants have learned.
I vividly remember that it was in July 27, 2011 that I started to have a more concrete conceptualization of this OLETE strategy. I was attending the DOH celebration of Disaster Consciousness Month in the Heritage Hotel with a launching of the Department of Health Health Emergency Management Staff’s publication and website. It was also on this day that HEMS announced the publication of Beyond Data which contained a write-up of MDH Disaster Preparedness Program. While listening, I was on my seat scribbling an OLETE strategy. (I will try to locate my notes on this in the near future.)
One year after, July 23, 2012, I concretized my OLETE through this seminar on Crisis Communication Management for MDH Staff. I repeated this OLETE on the seminar on Medical Sharps Waste Management in the afternoon of July 23, 2012, also in MDH.
I personally think the OLETE offers an efficient educational strategy in organizations and hospitals like MDH in which one can be confident, if not assured, of learning of all staff with minimal cost and loss of time from services. As deduced from the “Quiz Instructions” above, participants will learn more in terms of knowledge at least compared to attending face-to-face lectures. There is active learning just by answering the questions. Simultaneously, there is evaluation of baseline knowledge and subsequent ones. This is reason why I dub the exercise as “Online Learning cum Evaluation Test Exercise” or “OLETE.”
Consider a face-to-face lecture session. Unless the facilitator gives out a pretest and a posttest, it will be difficult for him to measure how much the participants learned. Second, hardly does the facilitator expect all the participants to achieve a perfect score in the posttest, if there is one conducted. After the lecture, the facilitator does not know what will happen to the items which were not answered correctly. Were they corrected or not?
In OLETE, the target is a perfect score based on the recommended answers, no matter how many times the participants take the test within a prescribed period of time (from the experience, 2 weeks will be reasonable and practical.) The testing can be continued after a face-to-face session if there is one. Note: in the future, with the presence of an accompanying website, with online education, most of the time one can do away with a face-to-face learning session or this will only be done on per need basis.
In OLETE, there is measurement and documentation of learning that took place. This is hardly seen in the current set-up. The OLETE will be efficient in covering all the staff of an organization. In MDH, there is about 1000 staff. The OLETE will be very cost-effective for these 1000 staff.
Lastly, a re-certification can easily be done after a certain period of time. The OLETE can be used in the re-certification.
I recommend MDH and other hospitals to adopt my OLETE. By the way, the software, QuizStar is free.
Other action plans and recommendations:
- I will recommend to MDH Administration the adoption of my OLETE.
- I will request all the 10 participants in the first offering who have not yet reached a perfect score to take the test again.
- I will issue certificates of proficiency for those who already achieved a perfect score.
- I will fix the glitches in the technical aspect of online hosting of the tests and online registration for the test.
- I will do a feedback-survey among the 34 participants who took the test.
Questions? Email: email@example.com