Developing Learning Outcomes

There are many issues to consider when developing outcomes for any curriculum. To borrow a concept from Stephen Covey, it is important to begin this process with the end in mind.¹⁷

Begin by asking, “What will the students be able to do upon successful completion (of the degree, the program, or the course)?” Consider in the answer the knowledge, skills, and abilities you might expect a student to have years after successfully completing your course or program. There may be several outcomes, and there is no magic number of outcomes for an institution, a program, or a course. What is important is that the essential components of learning are represented. Once a desired outcome is identified, it must be developed and written.

Author Ruth Stiehl (2017)¹⁸ provides an excellent four-part backward-design model for developing learning outcomes. Her model requires faculty to collaboratively consider and find consensus regarding the concepts and issues that students must understand and address, the skills students must master, the assessments that students must take to demonstrate mastery in class, and finally, what students can do after they finish the course or program. The result of this process is an outcome for the course, program, or institution (see Table 1).

Table 1. Ruth Stiehl Model for Writing Learning Outcomes (See Appendix J for template)

Concepts & Issues	Skills	Assessment Tasks	Intended Outcomes
What must the learners understand or resolve to demonstrate the intended outcome?	What skills must the learners master to demonstrate the intended outcome?	What will learners do “in here” to demonstrate evidence of the outcome?	What do learners need to be able to do “out there” in the rest of life?

Once the intended outcome is identified, it must be carefully written. Good outcomes clarify expectations of the faculty member to the student and ultimately the public, so ensuring the outcome is in fact “good” becomes important.

Several qualities characterize a good learning outcome, including these.

Good learning outcomes include action verbs.
Good learning outcomes clearly state who is to do the action.
Good learning outcomes clearly state what action is to be done.
Good learning outcomes are achievable.
Good learning outcomes are observable.
Good learning outcomes are measurable.
Good learning outcomes are aligned to the curriculum.

Selecting Verbs for the Outcome

Verbs to avoid

When writing learning outcomes, stay away from verbs or phrases that are not easily observed or measured. Verbs such as understand, appreciate, comprehend, and learn, are very difficult to measure or observe. When faculty lean toward using one of these words, Stiehl and Sours (2017)¹⁹ suggest a way out. Begin with the phrase, “Use their understanding of _________ to…” They provide as an example an outcome for a geriatric program: “(Use their understanding of the aging process to) observe and respond to subtle living patterns and behaviors,” and they add that one can drop the first part to begin the outcome with the action verbs. The outcome immediately becomes observable and measurable: “Observe and respond to subtle living patterns and behaviors” (Stiehl & Sours, 2017. p 49).

Tempted to write, “Students will understand?” Stop! Instead, follow the advice of Stiehl and Sours (2017).

Begin with the phrase, “Use their understanding of _________ to…” They provide as an example an outcome for a geriatric program: “(Use their understanding of the aging process to) observe and respond to subtle living patterns and behaviors,” and they add that one can drop the first part to begin the outcome with the action verbs. The outcome immediately becomes observable and measurable: “Observe and respond to subtle living patterns and behaviors” (p 49).

Verbs to use

Use verbs that represent observable and measurable behaviors. These are verbs such as apply, demonstrate, describe, compute, explain, analyze, evaluate, compare, conclude, defend, and construct. Many educational institutions use Bloom’s taxonomy (1956) or some variation, such as Anderson and Krathwohl (2001), as a resource because it enables faculty to rely on precise language for expressing the learning outcomes of programs and courses.

The six categories of Bloom’s taxonomy allow a faculty member to assess a different type of skill or behavior in the course, starting from the lowest level of learning, the knowledge level, to the highest level, evaluation. The taxonomy is often presented visually as a pyramid (Figures 4, 5). In this representation, the most basic methods of cognition occur at the lowest levels; learning outcomes at the college level should focus on higher levels as often as possible.

Figure 4: Bloom’s Taxonomy Pyramid - Original

Figure 5. Bloom’s Taxonomy Pyramid – Revised

While many faculty are familiar with the levels in Bloom’s taxonomy, it is usually helpful to examine the variety of verbs that may be used to construct outcomes, assignments, and assessments when the goal is to articulate the type of behaviors that are desired. Three resource tables follow to provide examples of verbs that may be selected.

Table 2 (original Bloom’s) and Table 3 (revised version) define what is meant at each level of the taxonomy, and it provides related behaviors as verbs that can be selected when developing outcomes and related activities.

Table 2. Bloom’s Taxonomy: Definition and Related Behaviors

Category	Definition	Related Behaviors
Knowledge	Recalling or remembering something without necessarily understanding, using, or changing it	define, describe, identify, label, list, match, memorize, point to, recall, select, state
Comprehension	Understanding something that has been communicated without necessarily relating it to anything else	alter, account for, annotate, calculate, change, convert, group, explain, generalize, give examples, infer, interpret, paraphrase, predict, review, summarize, translate
Application	Using a general concept to solve problems in a specific situation, using learned material in new and concrete situations	apply, adopt, collect, construct, demonstrate, discover, illustrate, interview, make use of, manipulate, relate, show, solve, use
Analysis	Breaking something down into its parts; may focus on identification of parts or analysis of relationships between parts, or recognition of organizational principles	analyze, compare, contrast, diagram, differentiate, dissect, distinguish, identify, illustrate, infer, outline, point out, select, separate, sort, subdivide
Synthesis	Relating something new by putting parts of different ideas together to make a whole	blend, build, change, combine, compile, compose, conceive, create, design, formulate, generate, hypothesize, plan, predict, produce, reorder, revise, tell, write
Evaluation	Judging the value of material or methods as they might be applied in a specific situation, judging with the use of definite criteria	accept, appraise, assess, arbitrate, award, choose, conclude, criticize, defend, evaluate, grade, judge, prioritize, recommend, referee, reject, select, support

Table 3. Bloom’s (Revised²⁰) Taxonomy: Definition and Related Behaviors

Category	Definition	Related Behaviors
Remember	Recalling something without necessarily understanding, using, or changing it	define, describe, identify, label, list, match, memorize, point to, recall, select, state
Understand	Understanding something that has been communicated without necessarily relating it to anything else	alter, account for, annotate, calculate, change, convert, group, explain, generalize, give examples, infer, interpret, paraphrase, predict, review, summarize, translate
Application	Using a general concept to solve problems in a specific situation, using learned material in new and concrete situations	apply, adopt, collect, construct, demonstrate, discover, illustrate, interview, make use of, manipulate, relate, show, solve, use
Analysis	Breaking something down into its parts; may focus on identification of parts or analysis of relationships between parts, or recognition of organizational principles	analyze, compare, contrast, diagram, differentiate, dissect, distinguish, identify, illustrate, infer, outline, point out, select, separate, sort, subdivide
Evaluation	Judging the value of material or methods as they might be applied in a specific situation, judging with the use of definite criteria	accept, appraise, assess, arbitrate, award, choose, conclude, criticize, defend, evaluate, grade, judge, prioritize, recommend, referee, reject, select, support
Creating	Putting pieces of information together in a new way; reorganizing parts into a different form	blend, build, change, combine, compile, compose, conceive, design, formulate, generate, hypothesize, plan, predict, produce, reorder, revise, tell, write

When reviewing options for verb use, the same verb may be used in more than one level. This is true because the outcome or activity to learn the associated skills influences how the verb is applied. Tables 4 and 5 (original and revised taxonomy) provide several verbs to help construct outcomes (or assignments, assessments, or learning activities) at each level of the taxonomy.

Table 4. Suggested Verbs for Each Level of Bloom’s Taxonomy

Knowledge	Comprehension	Application	Analysis	Synthesis	Evaluation
Change	Account for	Apply	Analyze	Arrange	Appraise
Choose	Change	Assess	Appraise	Assemble	Assess
Define	Cite examples of	Change	Calculate	Collect	Choose
Identify	Demonstrate use of	Compute	Categorize	Compose	Compare
Label	Describe	Demonstrate	Compare	Construct	Confirm
List	Determine	Dramatize	Conclude	Create	Consider
Match	Differentiate between	Employ	Contrast	Design	Critique
Name	Discriminate	Generalize	Correlate	Develop	Estimate
Organize	Discuss	Illustrate	Criticize	Devise	Evaluate
Outline	Estimate	Initiate	Debate	Enlarge	Judge
Recall	Explain	Interpret	Deduce	Explain	Measure
Recognize	Express	Modify	Detect	Formulate	Qualify
Record	Identify	Operate	Determine	Manage	Rate
Recount	Interpret	Practice	Develop	Modify	Review
Relate	Justify	Predict	Diagnose	Organize	Revise
Repeat	Locate	Produce	Diagram	Plan	Score
Reproduce	Modify	Quantify	Differentiate	Predict	Select
Select	Pick	Quantify	Distinguish	Prepare	Test
Underline	Practice	Relate	Draw conclusions	Produce	Validate
	Recognize	Schedule	Estimate	Propose	Value
	Report	Shop	Evaluate	Reconstruct
	Respond	Solve	Examine	Re-organize
	Restate	Suggest	Experiment	Set-up
	Review	Use	Identify	Synthesize
	Select	Utilize	Infer	Systematize
	Show	Verify	Inspect
	Simulate		Inventory
	Summarize		Predict
	Tell		Question
	Translate		Relate
	Use your own words		Separate
			Test

Table 5. Suggested Verbs for Each Level of Bloom’s (Revised²¹) Taxonomy

Remember	Understand	Apply	Analyze	Evaluate	Create
Choose	Account for	Apply	Analyze	Appraise	Arrange
Define	Change	Assess	Appraise	Assess	Assemble
Find	Cite examples of	Compute	Calculate	Choose	Change
Identify	Demonstrate use of	Demonstrate	Categorize	Compare	Compose
Label	Describe	Dramatize	Compare	Confirm	Construct
List	Determine	Employ	Conclude	Consider	Create
Match	Differentiate between	Generalize	Contrast	Critique	Design
Name	Discriminate	Illustrate	Correlate	Estimate	Develop
Outline	Discuss	Initiate	Criticize	Evaluate	Devise
Recall	Estimate	Interpret	Debate	Explain	Enlarge
Recognize	Explain	Modify	Deduce	Judge	Formulate
Record	Express	Operate	Detect	Measure	Invent
Recount	Identify	Organize	Determine	Rate	Modify
Relate	Interpret	Predict	Diagnose	Review	Plan
Repeat	Justify	Produce	Diagram	Revise	Predict
Reproduce	Locate	Quantify	Differentiate	Score	Prepare
Select	Modify	Quantify	Distinguish	Select	Produce
Underline	Pick	Relate	Draw conclusions	Test	Propose
	Practice	Schedule	Estimate	Validate	Reconstruct
	Recognize	Solve	Examine	Value	Re-organize
	Report	Suggest	Experiment		Set-up
	Respond	Use	Identify		Synthesize
	Restate	Utilize	Infer		Systematize
	Review	Verify	Inspect
	Select		Inventory
	Show		Predict
	Simulate		Question
	Summarize		Relate
	Tell		Separate
	Translate		Test
	Use your own words

Finally, Stiehl and Sours (2017)²² recommend outcomes be written to represent what students should be able to do after leaving the current academic experience. In other words, an outcome should indicate what students will be able to do in another course, in a subsequent degree program, on a job, or even in life, after successfully completing the academic course or program. Table 6 provides ideas to promote such transferability.

Table 6. Using Verbs to Construct Outcomes that Translate Outside the Classroom

Transferable Skill(s)	Verbs that Lead to Evidence of the Skills
Creativity	originate, imagine, begin, design, invent, initiate, state, create, pattern, elaborate, develop, devise, generate, engender
Psycho-motor skills	assemble, build, construct, perform, execute, operate, manipulate, calibrate, install, troubleshoot, measure, transcribe
Self-appraisal or reflection on practice	reflect, identify, recognize, evaluate, assess, criticize, judge, critique, appraise, discern, judge, consider, review, contemplate
Planning or management of learning	plan, prioritize, access, use, select, explore, identify, decide, strategize, organize, delegate, order, manage, propose, design
Problem-solving	identify, choose, select, recognize, implement, define, apply, assess, resolve, propose, formulate, plan, solve
Communication or presentation skills	communicate, express, articulate, question, examine, argue, debate, explain, formalize, respond, rebut, justify, defend, listen, illustrate, demonstrate, organize, pace, model, summarize, inform, persuade
Interactive, interpersonal, or group skills	accommodate, interact, collaborate, participate, cooperate, coordinate, structure, arbitrate, initiate, lead, direct, guide, support, decide, set goals, motivate, reflect, evaluate, recognize, enable, redirect, mediate

In developing learning outcomes, faculty may need to consult existing course outlines as these list current outcomes. The State Course Numbering System Web page provides a search tool to find all approved and existing course numbers. Program faculty should ensure alignment to curriculum frameworks at the state level and specialized accreditors (if applicable). Links are below.

PBSC Course Outlines: www.palmbeachstate.edu/utilities/CourseOutlines
State Course Numbering System website: http://scns.fldoe.org
State Curriculum Frameworks: www.fldoe.org/workforce/dwdframe

Table 7 provides sample outcomes at the institution, program, and course levels. Some examples are taken from actual outcomes at Palm Beach State College, while others are modified versions or written explicitly as an example for the table.

Table 7. Sample Outcomes (may or may not exist at PBSC)

INSTITUTION LEVEL SAMPLES

1. Critical thinking: Engage in purposeful reasoning to reach sound conclusions.
2. Ethics: Make informed decisions based on ethical principles and reasoning.
3. Information literacy: Find, evaluate, organize, and appropriately use information from diverse sources.
4. Communication: Employ writing, speaking, presenting, and reading skills to communicate appropriately and professionally to a variety of audiences.
5.Civility: Respectfully collaborate with diverse persons.
6. Mathematics: Use mathematical concepts to solve problems.
7. Humanities: Analyze creative arts and cultural perspectives.

PROGRAM LEVEL SAMPLES

1. Accounting: Prepare basic financial statements.
2. Computer Programming: Develop application software that can access files and databases.
3. Environmental Science Technology: Explain the importance of ethics and data integrity in scientific studies.
4. Human Services: Apply knowledge of mental health and human service trends, issues, and regulations to inpatient, outpatient, and other programs within the human services delivery system.
5. Interior Design Technology: Plan interior spaces that efficiently address client needs, furniture and equipment requirements, budgets, and environmental issues.
6. Law Enforcement Officer: Demonstrate proficiency in all high liability skills (firearms, defensive tactics, vehicle operations, first aid, and dart firing stun gun).
7. Massage Therapy: Competently communicate with massage therapists, clients, patients, and health care providers.
8. Nursing: Appraise patient or client health status through analysis and synthesis of relevant data.
9. Surgical Technology: Demonstrate the skills required for surgical procedures.

COURSE LEVEL SAMPLES

1. BSC 2420: Describe the major applications of modern molecular biotechnology and the implications of those applications.
2. ENC 1102: Compose non-fictional prose with a degree of formality appropriate to its subject, audience, and purpose.
3. LIT 2110: Identify significant ideas contributed to the world by international authors.
4. MAC 2233: Use integration to solve applications for business and economics.
5. MUL 1010: Analyze the stylistic characteristics of musical compositions.
6. POS 1041: Analyze national and domestic interests and foreign policy-making practices in the United States.
7. PSY 2012: Compare the theoretical principles that formed the field of psychology.

Outcomes may also include the expected result of the action, or the result may be stated separately as a target or benchmark. Examples of a health course learning outcome are shown both ways in Table 8.

Table 8. Options for Writing Learning Outcomes*

Outcome and target written separately	Target integrated into outcome
Outcome: Students will identify consumer, political, and economic issues influencing health disparities in diverse populations. Target: At least 80% of students will score 75 points or more on the 100-point unit test that requires demonstration of the skill stated in the outcome.	At least 80% of students will identify consumer, political, and economic issues influencing health disparities in diverse populations by scoring 75 points or more on the related 100-point unit test.

In each case, the instructor and students in this health class know what students are expected to do after completing the related unit in the class, and they know what students must do to prove they have accomplished the outcome. Faculty also know what to look for in the assessment results to communication the degree to which students are achieving the outcome. (Assessment is covered in Part 3 of this handbook).

¹⁷Covey, S.R. (2003). The seven habits of highly effective people: Restoring the character ethic. New York: Free Press. Covey says that to “begin with the end in mind” is Habit 2.

¹⁸The New Outcome Primers Series 2.0 (2017 includes six “primers” on outcomes and related topics, published by The Learning Organization, Corvalis, Oregon. Visit www.outcomeprimer.com

¹⁹Stiehl, R. & Sours, L. (2017). The outcome primer: Envisioning learning outcomes. Corvallis, Oregon: The Learning Organization.

²⁰Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing, Abridged Edition. Boston, MA: Allyn and Bacon.

²¹Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing, Abridged Edition. Boston, MA: Allyn and Bacon

²²Stiehl, R. & Sours, L. (2017). The outcome primer: Envisioning learning outcomes. Corvallis, Oregon: The Learning Organization.