:

Key Dates

Sample Learning Objective - Due Today! Send to Dr. Haines

TLC Seminars

September 5th - Learning Objectives and Presentations

Sept 19th - Small Group Facilitation

Oct 3rd - Multiple Choice Exams

Discussion Board - Traditional vs Active Learning Strategies - Aug 22 to Sept 5

Creating Exams and Writing Test Questions

Creating exams and developing questions to be used on exams is an important skill. Exams are intended to measure whether (or not) learners have mastered the material. Exams should be directly related to the learning objectives for the instructional activity. Therefore, teachers should strive for instructional alignment, thereby making the learning objectives, the learning methods, the learning materials, and the evaluation (or exam) congruent with one another and forming a cohesive whole.

Students typically think exams are intended to determine their grades. Indeed, exams are often used by the teacher to assign a grade to an individual student. But exams and grades are actually two different constructs. Exams, if well constructed, should measure the degree of mastery of a "sample" of factual, conceptual, or procedural knowledge ...or skills. We don't test everything a student knows or is able to do. We test a sample. Grades are pretty arbitrary. There is nothing magical about >90 = A or <60 = F. Someone (and frankly, we don't know who!) made that up, and we've come to accept these cutpoints as the norm. However, they are not based on an analysis of data. So, whether most of the students in a course receive A's ... or F's ... really depends on the difficulty of the exam (and the material) ... or how poorly the exam is constructed. Unfortunately, many exams aren't particularly valid (e.g. they are not accurately measuring what is intended). And some exams don't measure an adequate sample.

Exams can take many forms, and the items that appear on exams can be in different forms. An exam can be a written test where the student writes open-ended responses to a series of question prompts. An exam can be an oral challenge test where the student provides verbal responses. An exam can be a simulation that is observed, like an objective structured clinical exam (OSCE). However, because they can be quickly scored and simultaneously delivered to large groups of learners, the multiple-choice exam is (by far) the most frequent type of exam used in higher education today. While there are several pros to multiple-choice exams, they have certain limitations. One of the biggest issues with multiple-choice exams is they must be carefully constructed. And they aren't very good at measuring certain kinds of knowledge or skills.

Unfortunately, most faculty don't receive formal instruction on how to develop valid test questions. So, during our TLC Seminar in October, we will focus on how to construct an exam, how to write good multiple-choice items, and how to perform quality assurance following an exam to determine whether items performed as intended.

Watch This Video before October 1

Before our TLC Seminar on October 3rd, you'll need to complete some homework! Please watch this video (60 minutes) about the BEST PRACTICES - MULTIPLE-CHOICE EXAMS (click on the hyperlink) which I recorded some years ago (when I was on faculty with the University of Maryland). I think you'll be surprised about all the thought and care that goes into writing valid test items.

Watch this video

Psychometrics - Using Data to Construct and Validate Exams!

Once an exam is constructed, the fun doesn't stop! You'll want to use data to help determine if the exam was "fair" — that items measured what you wanted them to measure and weren't misleading. That's were psychometrics comes in handy. Psychometrics is the science of constructing, validating, and interpreting psychological tests.

When exams are scored by computers, we can quickly aggregate data and perform all sorts of statistical tests. These statistical tests can help uncover items that might be problematic and require additional analysis. If you want to learn more about these tests and what they can (and can't reveal), check out this video by the folks at ExamSoft (the leading testing software vendor to pharmacy schools:

Psychometrics 101: Know What Your Assessment Data is Telling You 

(click on the hyperlink)

Education: the path from

cocky ignorance

to miserable uncertainty.

— Mark Twain

Blog Essay Assignment

One of the required assignments in the TLC program is to write a blog essay about teaching or learning as it relates to health professions education. Your essay will be posted on the Educational Theory and Practice Blog (subscribe today!). One purpose of this activity is for you to use educational literature to inform your educational philosophy and instructional practices. While you are encouraged to share your personal experiences in your essay, your primary thesis should be evidence-based.

You have considerable latitude regarding the topic, but we want to avoid duplicating what others will be writing about this year. Moreover, I'd like to avoid significant overlap with other essays that have already been posted on the Educational Theory and Practice blog. Thus, before you start writing, please send me a quick email with your proposed topic so that I can "approve" it.

I've developed a BLOG ESSAY SCHEDULE and everyone will be assigned a due date for the initial draft of the essay. The schedule is primarily to reduce my workload (I can't review and provide feedback to everyone all at once) and help keep you on track (without a deadline you'll probably wait until June 29th to write your essay ... it's human nature!). Please review the schedule to be certain there are no major conflicts in your life that would preclude you from submitting your draft essay on that date. If, for some reason, you are unable to submit your draft by the due date, please email me and ask for an extension. I can be flexible, but you need to alert me in advance.

I've developed a rubric to help guide you (you can download it using this link). This is what I'm going to look for in your essay. It's important to use the education literature and include at least one primary research article. Your essay should include practical tips—things you believe educators should do based on the information in your essay. Feel free to include some of your own observations and experiences as a learner or teacher.

Lastly, as we'll discuss in the first TLC Seminar, several journals are dedicated to publishing scholarly works related to teaching and learning. Here are a few journals and newsletters to get you started:

• Academic Medicine (subscription required but some content is free each month)
• American Journal of Pharmaceutical Education (AJPE) (free)
• Currents in Pharmacy Teaching and Learning (CPTL) (subscription required but you may access through your health science library)
• Faculty Focus (free blog essays released every 2 weeks; focuses on teaching in higher education - not just pharmacy or medical education)
• Medical Teacher (subscription required; lots of great review articles)

University of Mississippi School of Pharmacy

Contact: Stuart T Haines

Director, Pharmacy Professional Development

University of Mississippi School of Pharmacy

Department of Pharmacy Practice

Jackson, MS 39216

Email: sthaines@olemiss.edu

Office: (601) 815-1845