Self-Assessment on Outcomes Assessment

Why Assessment is Necessary

Why do faculty give grades to students instead of trusting that each one learned what was taught? One might presume grades are assigned to let students know they have learned (or not). Grading then becomes a tool for learning and growing because it helps students learn what they did and did not understand in the material.

In this same way, assessment is a tool. Just as the students can use grades to improve, assessment is about getting the same benefit for ourselves. Specifically, assessment affords an opportunity to gather feedback we can use for our own benefit. The primary purpose of assessment is to improve learning (Angelo & Cross, 1993; Maki, 2004; Suskie, 2009)²⁶. Stiehl and Null (2018)²⁷ suggest we assess for three reasons: to assist students with learning, to advance students through a program of study, and to adjust our teaching or curriculum.

Assessment, while never perfect, is a tool to help faculty and an institution know and articulate how well students are achieving course and program learning outcomes. Prior to the learning outcomes model currently in place throughout colleges and university systems, assessment efforts were more focused on indirect measures and assessments of achievement, such as GPA and transfer rates. Although these measures are still monitored, the current model provides greater benefits for faculty and students by incorporating direct measures such as assignments and assessments that are embedded at the course level.

²⁶Angelo, T.A. & Cross, K.P. Cross (1993). Classroom assessment techniques: A handbook for college teachers (2^nd Ed). San Francisco, CA: Jossey-Bass.; Maki, P.L. (2004). Assessing for learning: Building a sustainable commitment across the institution. Sterling, VA: Stylus Publishing, LLC.; Suskie, L. (2009). Assessing student learning: A common sense guide (2nd Ed). ). San Francisco, CA: Jossey-Bass.

²⁷Stiehl, R. & Null, L. (2017). The assessment primer: Assessing and tracking evidence of learning outcomes. Corvalis, Oregon: The Learning Organization.

Getting Started

If you have not already, you are strongly encouraged to create a curriculum map (see Part 2 of this handbook) so you know where in the curriculum the outcomes and related content are introduced, reinforced, and emphasized. A curriculum map can also be utilized to “map” assessment to the outcomes. After you know what you expect students to do or know and when, you can select the right tools and develop an assessment plan. An outcomes assessment plan helps an instructor, program faculty and staff, and even an institution know what will be assessed, when it will be assessed, and how it will be assessed.

Developing assessment plans for an outcomes-based curriculum is far from unique to Palm Beach State College. It is good practice and commonly accepted in higher education, and many good resources are available, such as those found on websites such as the National Institute for Learning Outcomes Assessment, the American Associate of Colleges and Universities Assessment and VALUE, James Madison University Assessment Resources, and the University of Hawai’i Manoa.

Selecting Assessments

Assessment should be both formative (along the way) and summative (near or at the end), and multiple measures should be built into an assessment plan. Assessment should measure student performance and capture specifically the skills required to achieve a learning outcome. Assignment grades should not be used to assess a learning outcome unless the grade equates to a score that specifically measures the skills associated with achievement of the outcome.

When choosing an assessment, be certain you know why you are using it, who will assess and be assessed, what outcome it will measure, and what the scores will help you know or understand about student learning. Include direct measures, that is, assessments that require student performance directly related to a learning outcome.

Direct Measures of Learning Outcomes

Each assumes a clear relationship to a given outcome.

• Ratings of student performance
• Rubric scores on projects, presentations, research, written work, or other assignments
• Portfolios
• Quiz or test scores
• Results of student performance observations
• Results of skills-based or lab-based performance
• Results from student response systems

Indirect Measures of Learning Outcomes

• Course grades (will not give any information related to achievement of individual outcomes)
• Grades on projects, presentations, research, written work, or other assignments unless a rubric or scoring criteria clearly relate the assignment to a specific outcome
• Scores on external exams for entry into other programs unless the exam has isolated content scores that can be directly related to a specific outcome
• Satisfaction surveys
• Course evaluation surveys

Scoring Guides, Rubrics, and Checklists

Scoring guides, rubrics, and checklists can be effective assessments for formative assessment because such instruments typically include multiple criteria that students must master before taking a summative assessment to show they have achieved an outcome. Scoring guides and rubrics can be effective as summative assessment instruments. These tools most often clearly state criteria and require an instructor to either “rate” the degree to which the students meet the criteria or simply indicate the student does or does not meet each expectation. Scoring guides and rubrics can be applied to (used to score) a variety of assignments, including projects, presentations, research, written essays or papers, case studies, skill demonstrations, and more, and – assuming multiple faculty use the same scoring instrument and calibrate their use of the instrument – trend analysis becomes possible. Trend analysis allows faculty to address the degree to which their students are learning in the classroom, and trend analysis allows an institution to discuss the quality education it provides.

Selecting Performance Standards

Standards are often referred to as achievement targets, benchmarks, or expectations. Standards help faculty and the institution know the degree to which students are learning and easily provide a way to articulate student learning to each other and to external stakeholders. Setting standards and recording whether students meet those standards (or do not) is an important part of closing the assessment loop each cycle. There are multiple considerations when selecting standards.

Minimum Scores

A standard might be a minimum score, say a score of “4” on a 5-point rubric. It might also be a comparison to an external score such as a national benchmark, for example equal to or greater than the national average for students at a community college. A standard might also be set against previous performance, for example, with an expectation of annual improvement or maybe a demonstrated gain by the end of a semester over a pretest taken during the first week.

Achievement Targets

After a standard is established, it is also important to know the expected percentage of students who should meet the standard. In some cases, for example a licensure exam, meeting a standard may be a requirement to complete the program. In other cases, faculty may expect that a minimum percentage of students, say 80%, will achieve a minimum score. In still other cases, faculty may be looking for improvement over an initial point in time or an initial score.

Sample achievement targets include the following:

• 95% of students will demonstrate the skills safely and accurately in a lab setting.
• 75% of student scores will score at least 20 points higher on the post-test than on the pre-test.
• 80% of students will achieve a B or higher on the exam.
• 80% of students who take the licensure exam will achieve a score equal to or greater than the national average.

Developing Assessment Plans

Assessment plans at the institutional, program, or unit level should be developed with content experts to ensure careful alignment to the outcomes. Assessment plans should include very specific components, and the plans should be documented. Faculty developing assessment plans for a course should consider documenting the plan on a template similar to one provided in this handbook (Tables 12 and 13, Appendix M and Appendix N).

Assessing the Assessment Plan

Assessment plans should be evaluated for clarity and connection to the curriculum. The following criteria should be met when choosing assessment instruments and benchmarks.

• Selected assessments purposefully measure an intended outcome.
• Selected assessments are affirmed by content experts (faculty, staff, or literature).
• Selected assessments provide information that is as accurate and valid as possible.
• The assessment plan includes multiple measures with at least one direct authentic measure of student learning for each learning outcome.
• Achievement targets are clearly stated and justified by faculty who teach the related content.
• Data collection processes are explained and appropriate.
• The plan includes ways to share, discuss, and use the results to improve student or institutional learning (see Part 4 of this handbook).

Two options for “assessing assessment plans” are included as Appendix O and Appendix P.