Systematic Program Evaluation

Department of Educational Studies Counselor Education

The Ohio State University Counselor Education Program engages in systematic program evaluation to make sure that our students are prepared at the highest level to become ethical and culturally competent professional counselors and counselor educators. 

Our program evaluation includes information and data from the following sources:

  1. Practicum and internship evaluations by site supervisors
    • At least twice each semester, our student interns and practicum students are comprehensively evaluated by their site supervisors. Although this process is primarily intended to evaluate the professional development of the individual student, the faculty uses these evaluations to look for themes about the program as well. For example, if several students appear to be struggling with similar skills or knowledge, we recognize that these could reflect programmatic shortcomings. The overall results of the evaluation are discussed at the annual summer faculty retreat. 
  2. Passing of licensure exam 
    • Since the information has been tracked by the program (1996), OSU has had 100% passing of the Clinical Counseling (NCE) and the School Counseling Licensure exam, all on the first try. Since the implementation of the NCMHCE, OSU has had 100% passing of this exam as well. We believe that this is a strong indication that our graduates have received the appropriate academic training in the areas covered by the licensure exams. For the clinical counseling graduates who complete the exam, the program receives information on the mean scores of our graduates for each of the core CACREP areas. Faculty use this information when it is distributed by the state (every 5 years) to look for areas where our program may need improvement. 
  3. Input from our advisory board
    • Our advisory board meets at least annually to give us input and ideas about the program and our students. Our advisory board has alumni from both school and clinical M.A. programs, Ph.D. alumni, supervisors, current students, and faculty. We meet to share ideas and to continue to shape the future of our program. 
    • Advisory board meetings focus on the direction of the program and planning for improvements. In addition, advisory board meetings allow for future planning and for incorporation of best practices into the program. 
    • Data that has been collected regarding program evaluation by alumni and
      employers is shared at the advisory board meetings (as allowed re: confidentiality and FERPA). Input is solicited and action plans developed based on input from advisory board members concerning quality indicators about the program. 
  4. Surveys of employers and alumni 
    • Alumni are solicited every 2-3 years regarding their experiences in the counselor education program and as employers of the graduates of the program. Standardized in 2013, the alumni survey is designed to solicit input from both clinical and school alumni, who are asked to complete an online survey that contains a total of 39 Likert-like items and short answer items. 
    • Also standardized in 2013, the employer survey is distributed via the counselor education listserv to alumni and alumni’s employers, and participants are granted total anonymity for their responses. The responses to the surveys are shared among the faculty members and the advisory board; future planning occurs based on those results. 
  5. Evaluation of Faculty and Supervisors by students
    • In each class, the university conducts an electronic Student Evaluation of
      Instruction (SEI). In this process, students evaluate their class experience and the quality of the instruction and instructor. Each instructor can add open-ended questions to allow the students to voice their experience and provide suggestions for improvement. When these reports are provided to instructors, the scores, comparison scores, participation rate by students, and comments are saved by the instructor for the annual review process, when the department chair and the advisory board for the department uses the information for determinations about annual reviews and rehiring decisions. 
    • The evaluation process of site supervisors is ongoing in both the school and clinical tracks. Because each MA student has a doctoral supervisor, a site supervisor, and a faculty instructor of the clinical class, the student has multiple opportunities to evaluate all these supervisors. Problems are handled as learning moments: for the doctoral supervisor, the faculty supervisor provides ongoing feedback. Similarly, if there are concerns about site supervisors, the faculty supervisor who monitors placement will process the feedback with the supervisor after the student has graduated, or will decline placement with that supervisor if problems continue. 
  6. Placement of our graduates into counseling/counselor education positions
    • We track the placement of our M.A. graduates into their first positions. For over a decade, all of our M.A. graduates who have sought a counseling position have been hired in the field. We recognize that careers transition over time, and we use our alumni listserv and alumni website to try to keep track of the career trajectories of our graduates. The information that they share with us consistently demonstrates strong professional trajectories, with many of them moving into more advanced and/or supervisory positions over the course of their careers. Our M.A. students are highly sought after by agencies and schools. We consistently receive feedback from potential employers that our graduates are highly prepared, competent, and ready to take on their professional counseling roles. We receive many requests from potential employers to have our students and graduates apply for their open positions. 
    • We track the placement of our Ph.D. graduates into their first positions. Most of our Ph.D. graduates seek academic positions, and for those who are seeking tenure-track counselor education positions, we have a 100% placement record. Other Ph.D. graduates pursue careers as leaders or directors of counseling (at universities, counseling agencies, or medical centers). 
  7. State and national reputation of our students and graduates 
    • Each year, we track the OSU alumni who present at the All Ohio Counselor’s Conference, as one way to understand the career trajectories of our graduates who remain within the state. The presentations by OSU alumni, students, and faculty are posted on the website each year and shared with our OSU Counselor Education community via listservs. 
    • We host an annual OSU Counselor Education reception at the All Ohio
      Counselor’s Conference each year, and typically have more than 100 students and alumni come together to share fellowship and networking. We use this opportunity to stay in touch with our Ohio-based alumni. 

Program Improvements Made From Evaluation Feedback

Curricular Quality

Defined as: Course Content

Input Modification
Licensure Exam Pass Rate Current curricular quality is affirmed with 100% pass rate on first exam
Alumni Survey Content was adjusted in several courses to reflect feedback
Student Input

Semester assignment calendar was created to provide students with comprehensive calendar for their first semester.

Discussions are ongoing about M.A. vs. Ed.D. 
Student Performance 

Individual instructors use student performance to judge the effectiveness of course content and assignments.

Adjustments are made continuously ton insure student
learning.

Recent example: Multicultural counseling was amended to include more flexibility in the immersion experience
Student SEI Feedback 

Individual instructors use student SEI feedback to evaluate the effectiveness of course content and assignments.

Adjustments are made continuously to insure student
learning. 

Program Quality

Defined as: The Sequence of Courses in the Program and Structure of Program Experiences 

Input Modification
Student Input

Students and faculty have open dialogues about the student experience.

Recent example:
When students informed faculty that their 3rd semester was too much, faculty began exploring changes to the schedule and other options. Faculty discussed the individual assignments and course load for all 3rd semester classes at a faculty meeting.
Alumni Survey

Input from alumni surveys are discussed at faculty meetings.

Recent example: Several alumni suggested a change to the comprehensive exam format. However, when this was discussed with the broader alumni group, there was resounding support for the current format.

Program Quality

Defined as: Selection and Retention of Faculty, Supervisors, Adjunct Instructors, and Other Personnel 

Input Modification
Student Input Alumni Survey 

If students inform faculty that there is a problem with a certain instructor or supervisor, faculty determine whether there should be a conversation with the individual and/or whether the person should not be invited to return.

Recent examples: An adjunct instructor was not invited back to teach after poor evaluations and input from students. An adjunct instructor with extremely positive feedback has been rehired on an on-going contract. 
Student SEIs SEI reports on adjunct instructors are sent to program coordinator (as well as department chair) who informs faculty about results so a decision can be made about rehire status. 

Student Quality

Defined as: The Academic and Dispositional Potential of Students at the M.A. and Ph.D. Levels 

Input Modification
Licensure Exam Passing rate Current curricular quality and student selection criteria is affirmed with 100% pass rate on first exam. 
Student Performance in Class We have decided to maintain the college’s GRE scores for admission to maximize the potential for success for our students. 
Student Input If a potential student’s behavior is inappropriate during interview day, we take that information very seriously. We maintain the admissions process to ensure that current student input is sought on admissions applicants during the interview process.