One of the most difficult attributes of applying reliability and validity to assessment is recognizing that over time minor changes to content, expectations, or interpretations of content have a cumulative effect that can undermine assessments. In other words, it is very difficult to overcome a static perspective of reliability and validity. Over time, it's very likely that many small changes occur in content, interpretation of assignments, and emphasis on specific content elements. There seems to be much more focus on updating material and less focus on the reliability and validity of assessments. It's possible that very small changes in course content can significantly affect the validity and reliability of assessments. For example, the recently released APA formatting (6th edition), may represent a change that in the instructors perception should not affect reliability and validity. Add that to the mix of small changes to refine the course. Yet this is a perception, unsubstantiated with evidence. On one hand I understand it is not necessary to review validity and reliability of assessments every time a minor change is implemented, but on the other hand I am a bit worried about the cumulative effect of small increments over time.
This module has prompted me to go back and review impact of these small changes over time and assess their cumulative impact on the validity and reliability of the assessments.
Does anyone have some guidelines or experiences that would prompt an instructor as to when to go back and review validity and reliability of assessments?