Through the institutional effectiveness assessment cycle, every area of Palm Beach State College uses data, information, and analysis to assess and improve all aspects of the College’s operations. For Academic Affairs, this means developing, implementing, and refining a process to review academic programs to ensure that the curriculum (courses and programs) supports student success.
Program review is the means by which each program and discipline is reviewed by the College from both a quantitative and qualitative standpoint. This approach allows the College to look at both the “numbers” a program is producing and at the quality of learning that is occurring in each program or discipline.
Program Review at the State Level
Program review is an integral part of the Florida College System. Florida Statutes require the review of every major program in the State College System “every five years or whenever effectiveness or efficiency of a program is jeopardized.” Statutes also require the Division of Colleges to “conduct periodic reviews of existing programs.” Rule 6A-10.039(1), Florida Administrative Code, requires the Division of Colleges to annually submit to the Department of Education “a five-year review plan or annual update of the current five-year plan.”
The state program review plan consists of three levels.
- Level I includes the annual production of a data display for each certificate and degree program. The data displays are used by each college and the Division of Colleges to review programs to determine which programs and elements within the programs need additional review. The data display for the Associate in Arts degree program shows the performance of students in the State University System (SUS) upper division programs and compares SUS native students with transfer students from Florida colleges and with other transfer students. The performance measures are grade point averages, suspensions, graduations, average course loads, and credits earned for a degree.
- Level II is the review of certificate and degree programs by the individual colleges, either independently or in cooperation with independent and outside groups. It is this level of program review that the College designs to meet its needs for program review and assessment.
- Level III program review is the college system-wide review of selected programs by the Division of Colleges to address any concerns regarding these programs. The programs and issues for such reviews are determined by the Department of Education based on Level I and Level II information, Department of Education priorities, legislative and other interested parties or emerging issues.
Additionally, as of May 2020, a statewide program audit is being conducted by a multi-phase and systemic review of all programs. The College is participating in that audit and internal revisions through the strategic planning process, and changes to the program review process are expected. Necessary updates will be added to this section of the Academic Management Manual and communicated with program faculty and staff as soon as anticipated revisions are finalized.
The focus of this section is on how Palm Beach State College fulfills the requirements of Level II program review and for the College’s commitment to institutional effectiveness.
The three major ways in which curriculum can be divided at Palm Beach State is in terms of disciplines, programs and courses. Disciplines are defined as major areas of emphasis in the Associate in Arts degree and in general education; disciplines include areas such as mathematics, humanities and science. General education and the Associate in Arts degree are reviewed by the General Education Council with input from faculty College-wide. Programs are defined as career credit and noncredit programs of study that result in an award that does not necessarily transfer to another college or university.
Components of the Palm Beach State Program Review Process
Program review at Palm Beach State is a process in which every career-oriented program is reviewed annually. Six different types of information are reviewed:
- Program Health Indicators (PHI);
- Perkins Performance Measures;
- Student Learning Outcomes;
- Occupational Projections;
- Equity of Enrollment; and
- “Other” relevant information when applicable.
Program Health Indicators and the Perkins Performance Measures are primarily quantitative measures of program performance such as enrollment and completion data. Student Learning Outcomes, Occupational Projections (employment outlook) and the “Other” category of information tend to assess the quality of the program in that they address student learning and whether the program’s curriculum is current and in demand by local business and industry. Following a review of all available data, a recommendation is made regarding program viability and action plans for improvement are designed and implemented.
Figure 1 – Components of Program Review at Palm Beach State
Program Health Indicators
The Program Health Indicators (PHI) report constitutes the major quantitative portion of program review at Palm Beach State. For Associate in Science degree, College Credit Certificate, vocational certificates, Applied Technical Diploma, Advanced Technical Certificate, and Bachelor’s degree program, the PHI report presents a comprehensive data review. The report contains five-year trend data on program completers (broken out by degrees and certificates nested within the program, if applicable), FTE, headcount, number of sections offered, success and withdrawal rates in core curriculum, average student ratings of the instructors in the core curriculum and other program measures, if applicable. See “Program Review Resources” on the College Effectiveness Web page for a link to the most recent PHI report.
Perkins Performance Measures
For the career and technical programs, the state of Florida provides the College with a set of performance measures used by the federal Perkins grant for accountability purposes. Every program that received funding from the Perkins grant reviews three of these measures: Technical Skill Attainment, Completion, and Student Retention and Transfer. Any program that falls below the state benchmark on more than one of the three measures is required to write an improvement plan or provide an explanation. See “Program Review Resources” on the College Effectiveness Web page for a link to the most recent Perkins Performance report.
Student Learning Outcomes
Learning outcomes represent the knowledge, skills, and abilities students have attained as a result of their involvement in a particular set of educational experiences. By developing educational experiences based on what students should be able to do with their knowledge, the learning outcomes approach helps faculty, staff and students understand the point of the educational activity, be it a program or course.
Learning outcomes are developed at the institutional, program, and course levels. Program outcomes are posted on the College website (see Program Learning Outcomes) and assessed in an ongoing cycle with results being reported annually. See the Assessment Plans and Processes Web page for the most recent assessment reports.
Occupational Projections
The IRE office publishes the projected number of annual openings over the next five years for every career and workforce degree and certificate The data are based on a list of Standard Occupational Classification (SOC) codes relevant to the training provided by each degree and certificate within the programs; the codes are reviewed every three years by program personnel to ensure that the list of occupations for which the programs are providing training is accurate and current. Based on the program-specific SOC codes, a software package, Economic Modeling Specialist, Inc. (EMSI) derives the employment outlook in Palm Beach, Broward, and Dade counties for graduates with each degree or certificate. See “Program Review Resources” on the College Effectiveness Web page for a link to the most recent Occupational Projections report.
Other Relevant Information
The other types of information available to programs for purposes of program review vary by program and by assessment cycle. Many programs have a business partnership council. The Councils provide qualitative feedback to the programs regarding the performance of program graduates in the workplace and the currency or relevancy of the program curriculum. They may also provide informal feedback as to the current local job market for graduates of the program.
Many programs also have a program-specific accreditation agency. When a program has just completed a reaffirmation or new accreditation process, the feedback from the accreditor is typically included in the program review.
Action Plans for Improvement
In every cycle, the program is required to select at least one learning outcome that is targeted for improvement. Exceptions are made only when every program learning outcome is met, and the program has documented an action plan to improve other outcomes. For programs receiving Perkins funding, falling below more than one benchmark may prompt an additional improvement plan. If any Performance Health Indicator (PHI) measures are deemed unacceptable, the programs also write appropriate improvement plans for these measures.
The review of the occupational projections may have implications for the future viability of the program as a whole or specific certificates embedded within the program. Program review may result in recommendations to revise or eliminate programs based on changing market needs.
The strategies written into improvement plans vary. Strategies might involve changes in a course such as requiring students to practice more on a specific skill or revised instruction of that skill. Other strategies might involve the need for additional equipment, staffing, recruitment, or marketing. Still other plans might include revisions in the program curriculum. If the improvements involve financial resources, the improvements must be approved by the supervising dean and campus provost or vice president who submits the financial request during the College budget process. If the change involves program curriculum, it must undergo the Curriculum Review Process described below.
Curriculum Review Process
When faculty members, with input from program managers, administrators, or business partnership councils, create or update a course or a program, the first step of a continuous flow of information and quality control is initiated. Any curriculum action can potentially affect more than just the discipline proposing the change, so a carefully constructed process ensures that any revisions will benefit the institution and avoid unintended negative consequences.
Any curriculum action to courses or programs must be voted upon first by the faculty cluster. Additionally, if the program works with a Business Partnership Council (BPC), the BPC must also approve. Proposed actions must also be sent to the cluster liaison (associate dean) and academic dean. With all approvals, the action moves forward to the Curriculum Committee for review. If approved at this committee level, the action must then be evaluated by the Academic Deans’ Council and in some cases, the District Board of Trustees (DBOT). This level of review evaluates the impact the curriculum action will have on the College as a whole, such as budget, facilities, need for additional faculty, or accreditation as well as any impact it may have on other programs.
Program additions, deletions, and extensive revisions must be approved by the DBOT, and new programs not currently offered in the State of Florida must be reviewed and approved by the Division of Workforce Education and adhere to the substantive change process with the Southern Association of Colleges and Schools (SACSCOC).
For new career programs and when needed for existing programs, an additional process called “Developing a Curriculum” (DACUM) may be required. This process utilizes a focus group of experts in a storyboarding workshop to produce a comprehensive job analysis. The analysis identifies the duties, tasks, knowledge, skills, and traits related to a specific job or occupation in a chart format which provides input for needed curriculum revision or development.
Program Review Process: What, When, Who and How
Bachelor’s Degree, Associates Degree, and College Career Programs
The table below shows the annual program review process and timeline for the degree programs and CCPs. Updates to this timeline are published as needed in an expanded table under the “Program Review Resources” heading on the College Effectiveness Web page.
Palm Beach State College Program Review: Actions, Participants and Completion Deadlines
| PROGRAM REVIEW ACTION |
RESPONSIBLE PERSON(S) |
DEADLINE |
| Data collection period for Program Learning Outcome (PLO) data |
Program Faculty (FT/PT)
Program Directors |
Ongoing (Summer-Fall- Spring) |
| Programs enter all PLO assessment results in Planning (formerly Compliance Assist). Associate Deans ensure that all PLO data are entered into Planning (except action plans that may need to be finalized in early fall with faculty). |
Associate Deans
Program Directors |
August 31 |
| IRE staff update PHI, Perkins, FETPIP, Occupational Projections, and Equity reports on the Assessment Resources Web page. |
IRE Staff |
August 15 |
Program Meeting: Associate Deans meet with all available program staff to:
1. Review all assessment data (PLO, PHI, Perkins, FETPIP, and Occupational Projections, other if available/applicable).
2. Review previous action plans and progress reports.
3. Determine which outcomes or measures need improvement and develop improvement strategies for new cycle, if not already done.
4. Update action plans for PLOs in Planning, if needed.
5. Draft Program Review Summary Form in Planning. |
Associate Deans
Program Staff (Directors, Faculty, Adjuncts, Staff) |
September 30 |
| Potentially Immediate Action Required: Academic Deans request new faculty positions (or reductions) if needed. |
Academic Deans |
September 309 |
First Campus Meeting(s): Academic Deans meet with Associate Deans to review summaries and recommendations.
1. Associate Deans present summary for each program
2. Academic Deans work with Associate Deans to finalize overall recommendations to include needs for additional resources, faculty, equipment, funds, etc.
3. Determine who will complete follow-up actions |
Academic Deans
Associate Deans |
October 15 |
Second Campus Review Meeting(s): VPAA/Provosts meet with Academic Deans to review program review summaries and recommendations to forward to President’s Cabinet as needed.
1. Academic Deans make summary presentations of overall program health and make requests for additional resources, faculty, equipment, funds, etc.
2. VP/Provosts make determinations for final recommendations regarding program termination to forward to President’s Cabinet as needed. |
VPAA/Provosts
Academic Deans |
November 15 |
9Estimated deadline is for subsequent academic year and subject to change annually; check the Faculty Hiring Process and Timelines”
schedule for updates
Associate in Arts Degree
The College has five ILOs to encompass communication, critical thinking, information literacy, S.T.E.M. (science, technology, engineering, and mathematics), and socio-cultural understanding. Each ILO has an accompanying rubric for assessing the competencies associated with the outcome and with the related general education area (Communication, Humanities, Mathematics, Natural Sciences, or Social Sciences.). The outcomes and rubrics may be viewed on the Institutional Outcomes and Assessment Web page.
Primary assessment of the ILOs is conducted at the course level with Signature Assignments. These are assignments that are required for the course grade and that can be scored by at least one dimension of the rubric for a given outcome. Faculty are asked to voluntarily submit assignments each fall which are reviewed by the College-wide Assessment Committee. If sampled, faculty are asked to submit student work on the selected assignment in either the fall or spring semester. That work is then scored in the summer by a trained faculty scoring team.
Review of the A.A. degree and general education begins annually in the fall semester by the General Education Council. A multi-measure approach is taken as the Council reviews results of student performance on the Signature Assignments, performance of PBSC students in the State University System, and graduation rates of PBSC students. This faculty group makes recommendations for improvement strategies which are then shared with the Strategy Council and College-wide faculty groups for additional feedback and revisions. A final report is shared with the Academic Deans’ Council to close the loop and begin implementation of improvement strategies.
Assessment Management System
Assessment data, improvement plans, and program review summaries are entered into Campus Labs Planning, an online application that systematically documents all of the College’s assessment and program review activities. Program faculty who are responsible for entering results will find the link to Planning on the Institutional Research and Effectiveness Web page. If access is needed, faculty and staff should contact IRE.