Part 3: Assessment of Learning Outcomes

The Need to Know Information

Main Function

Assessment should inform teaching and improve student and institutional learning.

Key Points to Know

• Assessment should provide useful information to faculty and the institution.
• Assessment should measure the intended learning outcomes.
• Assessment should never be used punitively.
• Assessment is not perfect.

Benefits

• Good assessment is a tool for learning: it lets students demonstrate outcomes achievement with feedback to tell them where they are missing the mark and allows faculty adjust teaching when necessary.
• Good assessment lets faculty know what progress is being made toward an outcome.
• Good assessment lets faculty test what they teach instead of teaching to a test.
• Good assessment lets faculty stay ahead of legislation.
• Good assessments provide a way for faculty and the institution to articulate student learning to internal and external constituents.

Self-Assessment on Outcomes Assessment

Take the self-assessment below related to assessment in one or more courses you teach. Answers will vary and are discussed throughout Part 3.

Why Assessment is Necessary

Why do faculty give grades to students instead of trusting that each one learned what was taught? One might presume grades are assigned to let students know they have learned (or not). Grading then becomes a tool for learning and growing because it helps students learn what they did and did not understand in the material.

In this same way, assessment is a tool. Just as the students can use grades to improve, assessment is about getting the same benefit for ourselves. Specifically, assessment affords an opportunity to gather feedback we can use for our own benefit. The primary purpose of assessment is to improve learning (Angelo & Cross, 1993; Maki, 2004; Suskie, 2009)²⁶. Stiehl and Null (2018)²⁷ suggest we assess for three reasons: to assist students with learning, to advance students through a program of study, and to adjust our teaching or curriculum.

Assessment, while never perfect, is a tool to help faculty and an institution know and articulate how well students are achieving course and program learning outcomes. Prior to the learning outcomes model currently in place throughout colleges and university systems, assessment efforts were more focused on indirect measures and assessments of achievement, such as GPA and transfer rates. Although these measures are still monitored, the current model provides greater benefits for faculty and students by incorporating direct measures such as assignments and assessments that are embedded at the course level.

²⁶Angelo, T.A. & Cross, K.P. Cross (1993). Classroom assessment techniques: A handbook for college teachers (2^nd Ed). San Francisco, CA: Jossey-Bass.; Maki, P.L. (2004). Assessing for learning: Building a sustainable commitment across the institution. Sterling, VA: Stylus Publishing, LLC.; Suskie, L. (2009). Assessing student learning: A common sense guide (2nd Ed). ). San Francisco, CA: Jossey-Bass.

²⁷Stiehl, R. & Null, L. (2017). The assessment primer: Assessing and tracking evidence of learning outcomes. Corvalis, Oregon: The Learning Organization.

Getting Started

If you have not already, you are strongly encouraged to create a curriculum map (see Part 2 of this handbook) so you know where in the curriculum the outcomes and related content are introduced, reinforced, and emphasized. A curriculum map can also be utilized to “map” assessment to the outcomes. After you know what you expect students to do or know and when, you can select the right tools and develop an assessment plan. An outcomes assessment plan helps an instructor, program faculty and staff, and even an institution know what will be assessed, when it will be assessed, and how it will be assessed.

Developing assessment plans for an outcomes-based curriculum is far from unique to Palm Beach State College. It is good practice and commonly accepted in higher education, and many good resources are available, such as those found on websites such as the National Institute for Learning Outcomes Assessment, the American Associate of Colleges and Universities Assessment and VALUE, James Madison University Assessment Resources, and the University of Hawai’i Manoa.

Selecting Assessments

Assessment should be both formative (along the way) and summative (near or at the end), and multiple measures should be built into an assessment plan. Assessment should measure student performance and capture specifically the skills required to achieve a learning outcome. Assignment grades should not be used to assess a learning outcome unless the grade equates to a score that specifically measures the skills associated with achievement of the outcome.

When choosing an assessment, be certain you know why you are using it, who will assess and be assessed, what outcome it will measure, and what the scores will help you know or understand about student learning. Include direct measures, that is, assessments that require student performance directly related to a learning outcome.

Direct Measures of Learning Outcomes

Each assumes a clear relationship to a given outcome.

• Ratings of student performance
• Rubric scores on projects, presentations, research, written work, or other assignments
• Portfolios
• Quiz or test scores
• Results of student performance observations
• Results of skills-based or lab-based performance
• Results from student response systems

Indirect Measures of Learning Outcomes

• Course grades (will not give any information related to achievement of individual outcomes)
• Grades on projects, presentations, research, written work, or other assignments unless a rubric or scoring criteria clearly relate the assignment to a specific outcome
• Scores on external exams for entry into other programs unless the exam has isolated content scores that can be directly related to a specific outcome
• Satisfaction surveys
• Course evaluation surveys

Scoring Guides, Rubrics, and Checklists

Scoring guides, rubrics, and checklists can be effective assessments for formative assessment because such instruments typically include multiple criteria that students must master before taking a summative assessment to show they have achieved an outcome. Scoring guides and rubrics can be effective as summative assessment instruments. These tools most often clearly state criteria and require an instructor to either “rate” the degree to which the students meet the criteria or simply indicate the student does or does not meet each expectation. Scoring guides and rubrics can be applied to (used to score) a variety of assignments, including projects, presentations, research, written essays or papers, case studies, skill demonstrations, and more, and – assuming multiple faculty use the same scoring instrument and calibrate their use of the instrument – trend analysis becomes possible. Trend analysis allows faculty to address the degree to which their students are learning in the classroom, and trend analysis allows an institution to discuss the quality education it provides.

Selecting Performance Standards

Standards are often referred to as achievement targets, benchmarks, or expectations. Standards help faculty and the institution know the degree to which students are learning and easily provide a way to articulate student learning to each other and to external stakeholders. Setting standards and recording whether students meet those standards (or do not) is an important part of closing the assessment loop each cycle. There are multiple considerations when selecting standards.

Minimum Scores

A standard might be a minimum score, say a score of “4” on a 5-point rubric. It might also be a comparison to an external score such as a national benchmark, for example equal to or greater than the national average for students at a community college. A standard might also be set against previous performance, for example, with an expectation of annual improvement or maybe a demonstrated gain by the end of a semester over a pretest taken during the first week.

Achievement Targets

After a standard is established, it is also important to know the expected percentage of students who should meet the standard. In some cases, for example a licensure exam, meeting a standard may be a requirement to complete the program. In other cases, faculty may expect that a minimum percentage of students, say 80%, will achieve a minimum score. In still other cases, faculty may be looking for improvement over an initial point in time or an initial score.

Sample achievement targets include the following:

• 95% of students will demonstrate the skills safely and accurately in a lab setting.
• 75% of student scores will score at least 20 points higher on the post-test than on the pre-test.
• 80% of students will achieve a B or higher on the exam.
• 80% of students who take the licensure exam will achieve a score equal to or greater than the national average.

Developing Assessment Plans

Assessment plans at the institutional, program, or unit level should be developed with content experts to ensure careful alignment to the outcomes. Assessment plans should include very specific components, and the plans should be documented. Faculty developing assessment plans for a course should consider documenting the plan on a template similar to one provided in this handbook (Tables 12 and 13, Appendix M and Appendix N).

Assessing the Assessment Plan

Assessment plans should be evaluated for clarity and connection to the curriculum. The following criteria should be met when choosing assessment instruments and benchmarks.

• Selected assessments purposefully measure an intended outcome.
• Selected assessments are affirmed by content experts (faculty, staff, or literature).
• Selected assessments provide information that is as accurate and valid as possible.
• The assessment plan includes multiple measures with at least one direct authentic measure of student learning for each learning outcome.
• Achievement targets are clearly stated and justified by faculty who teach the related content.
• Data collection processes are explained and appropriate.
• The plan includes ways to share, discuss, and use the results to improve student or institutional learning (see Part 4 of this handbook).

Two options for “assessing assessment plans” are included as Appendix O and Appendix P.

Components of an Assessment Plan

Plans at all levels (institutional, program, course) should include at a minimum, the measures and achievement targets for each learning outcome. Every outcome should have at least one measure and a corresponding standard should be established for each measure. The standard may be called something else such as benchmark or achievement target. Currently at Palm Beach State, both terms are used in assessment plans.

Implementation and Data Collection

Good assessment plans also include details regarding implementation and data collection. For example, if a course-level assessment is selected as a program measure, the assessment plan should clarify for all instructors in that course what the assessment items are and exactly when in the course the assessment is to be administered. “When” may be inherent because it is stated as a specific unit or lab within the course. However, if an assessment is not specific to a unit test or assignment, it should be clear what elements of the course should be covered before students take the assessment or complete the assignment.

An assessment plan can succinctly clarify other important details. Consider including:

• Semesters or date ranges for the assessment cycle
• Who will be assessed (students in what class?)
• When in that cycle the data are to be collected (state any specific unit(s) that must be covered before the assessment is administered, or provide a time frame such as ‘during the last regular week of class’)
• Directions for reporting results

All academic programs at Palm Beach State College have developed assessment plans, and faculty in many courses have done the same. Following are sample plans for both a program (Table 15) and a course (Table 16). In each case, only a portion of the plan is provided. A template to write an assessment plan is provided as Appendix N.

Table 15. Sample Program Assessment Plan (adapted from Electrical Power Technology)

Table 16. Sample Course Assessment Plan (Adapted from ETS2530C)

Authentic Assessment

The assessment plan should address the issue of the type and amount of credit students can earn for participating in the assessment. Specifically, faculty should discuss how much credit and whether credit earned is “extra” or “authentic,” meaning it is applied to the course grade. Authentic assessment is, of course, preferred. When students participate in assessment with results attached to their course grade, they are more likely to give their best effort, and faculty are more likely to find the results useful to improve learning.

In some programs, faculty will not have options regarding how to award credit, but in those cases where it is optional, ideally, faculty will try to agree on a percentage of the grade a student can earn on the assessment. For example, faculty may agree that the assessment will be worth between two and three percent of the total grade.