Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports

1. We’re Accredited!
Now What?
Preparing the First QA
Report
Presented by:
Aaron Glassman, D.Mgt.
Kelly Whealan George, PhD
Dixie Button, DBA

2. University Overview
• 3 campuses:
– Daytona Beach (~ 4500 UG & 500 grad)
– Prescott (~ 1500 students)
– World Wide (>150 locations, ~ 40 states, S. America,
Europe, Asia and Middle East; ~26,000 FTE students)
• Emphasis is aviation & aerospace
• BS, MS and PhD programs
• ~ $17M in research

3. Personnel Changes
Accreditation granted in December 2013. Since then . . .
• Third Reorganization
• Third Dean
• Third CAO
• Chancellor, Combined CAO/Chancellor, back to Chancellor
• New Accreditation campus liaison to work w/University liaison
• New faculty with faculty roles
• ACBSP co-champion leaves. New co-champion appointed 5
months before QA report due

4. Personnel Changes
In the midst of personnel changes . . . .
• What is this thing called Notes and Conditions attached to the
accreditation?
• What is this thing called a Quality Assurance (QA) report?

5. Design Teamwork for Success
ACBSP Committee to address upcoming QA report issues
• Monthly teleconferences
• Knowledge management (Sharepoint)
• Developed a road map
• Attempt to intersect QA requirements with all other program
requirements (SACS, QEP….)
• Spreads knowledge base around the College

6. Conditions and Notes
Notes & Conditions attached to ACBSP accreditation
• Okay . . . But what does this mean?
• The initial accreditation letter says “Conditions and notes
indicate that either the Standard or Criteria is not fully met,
and should be viewed as an opportunity to move your
program to a higher level of excellence.”
• First thought . . . Is satisfaction of Conditions & Notes due by
the first QA report?
• No, just need to show that we are addressing them.

7. Conditions and Notes
ACBSP Process Book - Conditions
Most standards are met, but there are some deficiencies in fully
meeting all of the ACBSP standards.
ACBSP Process Book - Notes
In compliance with ACBSP standards, but not some of the criteria
. . . Notes represent concerns of the Board of Commissioners and
until those concerns are dealt with by the institution and accepted by
ACBSP, the progress on improving the process to remove the notes
should be included in the quality assurance report submitted to
ACBSP every two years.

8. Conditions and Notes
Conditions must be removed by 9/15/2017 and progress on
notes must be reported in future Quality Assurance Reports the
first of which is due September 15, 2015
• Note on Standard 2
• Condition on Standard 3
• Condition on Standard 4, Criterion 4.2, 4.3, and 4.4
• Note on Standard 5, Criterion 5.3.2.b
• Note on Standard 6, Criterion 6.3.6 and 6.3.7

9. Conditions and Notes
Note on Standard 2
• Referenced input from faculty in the development of both
long-term and short-term strategic goals and objectives with
measures, action plans, and timelines.
• Result: Removed
Condition on Standard 3
• Need feedback from stakeholders of the business programs
other than students.
• Result: Condition reduced to a Note

10. Conditions and Notes
Condition on Standard 4, Criterion 4.2, 4.3, and 4.4
• Show evidence of changes/improvements resulting from
internal assessments
• Need to better document and analyze trend data as well as
use comparative data to continuously improve student
learning outcomes.
• Result: Conditions reduced to a Note

11. Conditions and Notes
Note on Standard 5, Criterion 5.3.2.b
• Show progress on plan to ensure 40% of undergraduate credit
hours taught by academically qualified faculty within two years.
• Result: Ongoing, Note maintained
Note on Standard 6, Criterion 6.3.6 and 6.3.7
• Document results & improvements resulting from newly
implemented dashboard
• Result: Notes removed

12. Shifting Priorities
• QEP (Undergraduate Research)
• Research
• Externally Funded Research
• Promotion and Tenure
• Cross-campus collaboration

13. Publicly Available Information
ACBSP Requirement
• For Demonstrating Excellence in Baccalaureate/Graduate degree
Business Programs the business unit must routinely provide
reliable information to the public on its performance, including
student achievement, such as assessment results.

14. Publicly Available Information
What to include in publicly available information?
• Formed a committee, developed policy, execution ongoing.
(http://worldwide.erau.edu/degrees-
programs/colleges/business/index.html)

15. Publicly Available Information
Biggest Challenges
Sources of Information
• ACBSP requirements from acbsp.org
• Review of other ACBSP accredited colleges, exp. North
Carolina Central University and Gardner-Webb University

16. Publicly Available Information
Biggest Challenges (Cont.)
What will be displayed
• Student Satisfaction Survey
• Worldwide Alumni Survey
• Graduation and Retention Rates for the College of Business
• External Assessment Data (Peregrine), to fulfill the
requirement of publishing student learning outcome
assessment results

17. How we assess programs…..
• Courses have activities
• Activities are mapped to Learning Outcomes
• Learning Outcomes are mapped to Program Outcomes
• Program Outcomes Rubrics Embedded in LMS
• Learning Analytics Data Extracted from LMS

18. Tips, Tricks, and Traps
Tip #1 – Do not map all activities even though it can be done.
• Only assess activities that have grade significance (students
are less likely to skip the activity and grades are more normally
distributed).
• Avoid mapping discussion board posts and other “route”
weekly assignments as interrater reliability is usually poor
even with a rubric.

19. Tips, Tricks, and Traps
Tip #2 – Provide clear directions to faculty on using rubrics with LO
assessments embedded.
• Faculty may unintentionally override the rubric to give the student a “0” or
“100” score which means LO data is not collected.
• Faculty need to understand that attaining a LO does not reflect the students
activity score. (e.g., student can earn a 75 in a paper for improper APA
formatting and other non-content deductions but still achieve the mapped
LO’s at the level of mastery)

20. Tips, Tricks, and Traps
Trap #1 – We only think we are assessing programs.
• We are actually assessing course activities and the sum of the
activities across multiple courses is how we gauge program
health.
• Students from different programs may take the same course.

21. Tips, Tricks, and Traps
Trap #2 – Binary/Administrative grades pollute the data.
• Students who receive zero scores for not submitting
assignments skew learning outcome attainment data
(averages) to the negative.
• Activities that are pass/fail (100pts) skew the outcome
attainment data to the positive.
• These errors are hard to account for.

22. Tips, Tricks, and Traps
Trick #1 – Use multiple data streams to assess programs.
• Direct Measure – Student Scores on Mapped Activity,
Capstone, External Assessment, Artifact Grade/Regrade, etc.
across multiple sections of multiple classes.
• Indirect Measure – Post-Course Feedback (Student), Post-
Course Feedback (Faculty), Grade Distributions, Admissions vs.
Graduation Rates
• Configure all systems with assessment in mind!

23. Lessons Learned
• Fight for continuity
• Prepare yearly report similar to the 5th year QEP summary
• Incentivize key people
• Development by attending ACBSP conference

24. Contacts
Dr. Kelly Whealan
George
Director of Accreditation,
Assessment, and
Research Advancement of Ignite
Discipline Chair, Social Sciences
and Economics
Dr. Dixie Button
MBA-A Program Chair
ACBSP Co-Champion
Dr. Aaron
Glassman
Department Chair,
Technology Management
Department