Formative Evaluation
Evaluation plays a key role in the design process. For those of you that developed online classes, what types of formative evaluations have you used?
Nori:
This is an excellent question. Most institutions with which I have worked donot conduct a formative evaluation either because of time constraints or monetary constraints or availability of students representative of the population for whom the course is being developed. For that reason, we recommend and work with them to build adequate evaluation strategies for each phase of the design and development process so that we keep revisions to a minimum in the final phase before the course is released.
Satrohan
I have completed teaching a basic fundlementals accounting course where the book was not reviewed or the course structure was not written correctly. The first hour into the course I knew there would be problems.
Through this 80 hour course I had to make several changes to and adjust my teaching style due to the book and the way the outline was written for the course.
The individual who picked the book does not have any concept of ISD. Therefore the course was unpleasant for the students, because we had to make several modifications to adjust the course for the level of training they were supposed to recieve. When evaluations are done all materials and media needed to be worked through and tried out prior to doing so on a full classroom.
I believe that Instructors need to be a part of the process not excluded when developing any course materials.
Rhonda:
I fully agree with you rproposition about Instructor involvement in the development of the course. Further to that, i also recommend that the dvelopment schedule includes time for the Instructor to review the course before it is launchd.
Satrohan
Well said Satrohan! Instructors need to review the material prior to the launching of it. This is the only way a course can be taught confidently by an Instructor presenting material for the first time.
Thank you for the positive feedback, Rhonda. My recommendation is for the courses to be in a ready-to-launch state 30 days before the actual launch date. This gives the Instructors two weeks to review; one week to submit final changes; and the institution, one week to implement those changes.
Satrohan
I have been involved in on-line annual training for a large healthcare system.
When the employee/student completed the training they were required to evaluate the material and training. If the employee did not evaluate the program they would not get credit for the class. The evaluation was built into the training as a final module.
The evaluation was a set of questions with multiple answers; a. agree b. disagree. There was a phone number set up for any additional comments from the employee about the training.
The employees did contact us and the training center did make changes to the program based on their suggestions
Lois:
These are all valid rpocedures to follow to evaluate training. Some corporations engage in what is called training validation. At an appropriate interval after employees participate in their training, they are asked to complete another evaluation geared to assessing the degree to which the training has enabled them to perform their duties. This is particularly suitable for perfromance-based training.
Satrohan
Oh boy, I'm beginning to develop a course that nobody has tried here before and learning as I go along. What is helping is having used projected images for the last couple of quarters. I picked the hundreds I use to illustrate particular points. Now I can see building a textal course around the flow of ideas that the selected images enhance.
As I mentioned in another forum, there is a danger in using too many "seebites", but careful use of illustrations (animation could be overkill) can enhance, but not replace, the verbal content.
Richard:
I support your approach------text first, media after. For example, we can define a bimodal distribution. A diagram to support the definition will go a far way to enable students to grasp the concept. Same for using a normal curve to illustrate mean, median, mode, Standard Deviation, and Z score. The diagram leave, in the student's mind, an imprint of the relationship among these measures derived from a set of raw scores.
Satrohan
Evaluation plays a key role throughout the design, development, and implementation of a program. An analogy would be similar to completing a needs analysis.
Feedback throughout the process would eliminate unnecessary work and it would be more cost-effective (people, time, resources, etc.).
Rhonda Cameron mentioned including the instructor(s) in the design/ development process. That's a great idea; however, it doesn't occur as often as we like. I enjoy participating in textbook and curriculum evaluations so that I am familiar with the changes and can provide insight into the programs.
Charlotte:
Adopting the right approach to designing, developing, evaluating, and implementing a learning program, as well as involving the right decision makers is highly recommended. It is the way to go. Far too often I find that both time and money prevent this from happening.
Satrohan
Our school recently went online, and we lacked in design, developing, evaluating, and implimentation areas. Instructors were shown examples and trained on the capibilites and limitations of the online portal, but left on their own to develop content. Design of the course was basically a result of the way the portal is set up to deliever information.
We are now almost finished with the first round of online classes, and are survying students regarding their experiences in these online classes. The feedback from these surveys will be used to revise the online courses.
Design, developing, evaluating, and implimenting online classes should be given the same attention as is given to the developing of on ground classes. Schools need to provide time for instuctors to create all courses, whether online or onground.
Kathleen:
The successful implementation of online programs requires a significant amount of planning, specialists, and funding---to mention the minimum. The use of exisitng Instructors as resources requires careful scrutiny to ensure their availability.
Satrohan
I agree that instructors must be involved in course design as well as textbook selection. Administrators tend to forget quickly what it's like to be an instructor or a student. So, they tend to have pie in the sky ideas which is ok to an extent. I think we need both instructors and curriculum specialists to help with course design as well input from current and former students. Former students are especially important since they are the ones who are competing for the jobs and know first hand what they were lacking or what their strengths were when they graduated.
Good point on the involvement with the textbook. There are books out there that simply do not work online as they do at the campus.
We use Sakai platform for educational interface and interaction, and for grading Forums, there is a statistics link that provides data to responses, and how many Forum entries have been read. This is a helpful tool for evaluating effectiveness of the Forum question and response by online learner. It will show how many responses have been authored by student and how many entries they have read. The educator then evaluates content of responses by reading and entering grade.
Margaret,
Interesting! I like the statistics link. Students like to know exactly where they stand on a topic!
Shelly Crider
