Evaluation

AA.     Program faculty members engage in continuous systematic program evaluation indicating how the mission, objectives, and student learning outcomes are measured and met. The plan includes the following:

1.      A review by program faculty of programs, curricular offerings, and characteristics of program applicants.

Program objectives for the Department of Counselor Education are regularly reviewed and, when necessary, revised as part of the self-evaluation studies required by the department, the School of Education, and major external program accreditation reviews. 

At the departmental level, the self-study process begins in the fall term (hyperlink to Fall 2011 faculty meeting agenda) when Counselor Education faculty review and discuss the analyzed results and narrative program evaluation feedback obtained from the following: students who are about to graduate; field site supervisor’s evaluation of student interns and the program; the prior year’s program graduates; and employers of the prior year’s graduates (discussed in Sections AA.2. and 3. below).  Based on this input, their own assessment of student learning outcomes, and any new state and national certification/licensure and/or counselor education training standards’ changes and trends, faculty review the department’s program objectives, student performance expectations, curriculum offerings, and operational policies and procedures to determine if there is a need to discuss making any revisions to the same throughout the academic year.

During this annual department level review of program evaluations, faculty also review the current status of the Department’s program specializations, long-range planning ventures, and any program objectives, curriculum offerings, operational policies and procedures, student performance expectations that would be related to these longer-term planning processes.  Additionally, issues pertaining to the types of students seeking admission during the recent and prior year admissions’ decision process are reviewed. Some of the most important student admissions’ issues that are examined include: the characteristics of students that are appropriate for each program, the cultural diversity of current and future students, and program enrollment balance and management.

If revisions are necessary, the department discusses, drafts, reviews, and approves these during monthly focus groups established for departmental program planning and development purposes and scheduled over the course of the current academic year (hyperlink to Spring 2012 faculty meeting agenda). In this way, changes are ready for implementation and incorporation into departmental and college publication and planning documents for the next academic year. 

In addition to this self-evaluation and review process, the department participates in program review and planning initiatives at both the School of Education and the College levels.  On a regular basis the College requires the department to examine its program goals, objectives, curriculum, student learning activities, and instructional and operational practices in terms of how these continue to support and enhance the mission of the School of Education.  Typical areas of concern for these reviews include, but are not limited to: short- and long-term strategic planning, how the program fulfills outside accreditation standards and trends, and what instructional, programmatic, staffing, facilities and fiscal resources are needed to appropriately service this program.  The department faculty meet in focus groups to address these issues and make any necessary program revisions, and the Counselor Education Chairperson submits reports regarding these reviews to the administration, faculty and students (hyperlink to reports: College Steering Committe correspondence; Correspondence with Registrar about program changes; Memorandum to students about program changes; Memorandum to students about Practicum/Internship changes).

Beyond these internally directed self-evaluation studies, the department of Counselor Education also reviews and, when necessary, revises its program objectives during the course of major external program accreditation reviews. The following chart represents the present reaccreditation cycles for these reviews:

Accreditation Group                 Last Accreditation Year            Reaccreditation Year

   Middle States                                            2005                                    2015

     NCATE                                                    2009                                    2014

     CACREP                                                 2005                                    2013

    CACREP Midcycle Report                       2009                                    2017

During these external accreditations, input pertaining to local and national counselor preparation needs and trends as well as current Counselor Education program objectives, curriculum offerings, operational policies and procedures, and clinical practices is obtained through individual, small group, and/or advisory committee meetings, and through solicitation of feedback from advisory committee members through online surveys, such as the one conducted in April-May of 2012 (hyperlink to Qualtrics survey results).  These feedback surveys and meetings invite input from the following: regular, adjunct and affiliate program faculty; current and former students; field site supervisors; cooperating agencies; and college administrative and related services staff. At least one year of internal self-study preparations precedes each of these external accreditation reviews to afford the department sufficient time to design, approve, and implement any necessary revisions.

2.      Formal follow-up studies of program graduates to assess graduate perceptions and evaluations of major aspects of the program.

The Department of Counselor Education secures program evaluation input from its graduates through annual dissemination of its Graduate Follow-Up Questionnaire.  Specifically, one year following the graduation date, a letter is mailed to program graduates with the following request (please click on hyperlink to preview the questionnaire):

“Dear ____________ (Graduate’s Name):

As a means of assessing our graduate programs in Counselor Education, I would like to ask your help in completing and returning the following information requests.

The enclosed forms are an Alumni Information Request (Graduate Follow-Up Questionnaire) and Alumni Employer Information Request form. Please complete and return the Graduate Follow-Up Questionnaire as soon as possible in the stamped return envelope.  If you are currently employed in the field of counseling, would you please ask your employer to complete the Alumni Employer Information Request form and also return it to me? Yours and your employer’s feedback regarding your preparation and training in counseling will be helpful to us in our program planning and development efforts.

Thank you very much for your continued support and assistance. Best wishes for continued success.

Sincerely,

Atsuko Seto, Ph.D.

Clinical Placement Coordinator”

Over the past 3 years, the return rate of Graduate Follow-Up Questionnaires has dropped, e.g., 2008 graduates (13 completed), 2009 graduates (6 completed), and 2010 graduates (3 completed).  Therefore, in the summer of 2012, our process for gathering feedback from 2011 graduates has changed from mailings to e-mail surveys.  Our hope is that this generation of students will be more likely respond to an email request with a link to an online Graduate Follow-Up Survey (and a 2nd email to graduates to be forwarded to the graduate’s employer for program feedback).

3.      Formal studies of site supervisors and program graduate employers that assess their perceptions and evaluations of major aspects of the program.

Assessment perceptions about the program are obtained from employers of Counselor Education program graduates through two different means. In the first, the Department forwards an Employer Evaluation form directly to Counselor Education graduates and asks the latter, if they are willing, to give these forms to their current employers to complete and return to the department. The second means involves procurement of input from Counselor Education program alumni who constitute a viable employment and/or field placement supervisory network that services current program students. As such, these alumni employers provide regular program evaluation input to the department when they are asked to serve as program review self-study advisory committee members, and when, as field placement supervisors, they evaluate the Counselor Education clinical program using the Evaluation of Clinical Program by Site Supervisor form, which can be found on Counselor Education website under Clinical Forms.  Assessments of the Counselor Education clinical program are obtained from field placement supervisors and personnel in cooperating and associated agencies through the same evaluation instrument.  All of this information is collected annually, analyzed at the end of the Spring term, and used for subsequent program reviews that begin the following Fall.

4.      Assessment of student learning and performance on professional identity, professional practice, and program area standards.

In addition to the systematic, developmental student assessment processes described in Standard P. (in Section I, “The Academic Unit”), a student learning outcomes assessment system was developed that provides systematic and continuous data collection related to the students’ acquisition of program-specific knowledge and skills (i.e., related to professional identity, professional practice, and program area standards).  Specifically, an online e-Assessment system was created through the use of course management software that was developed locally by instructional technology personnel at The College of New Jersey (TCNJ).  This e-Assessment system is housed within and linked to the local TCNJ course software called “SOCS” (which is an acronym for “Simple Online Course Software”).  The e-Assessment system is also used for the NCATE-accreditation student assessment process. (NCATE is the national accrediting body for teacher preparation programs within the School of Education at TCNJ.)

At the end of each semester, department faculty enter data related to students’ proficiency in specific-program standards that are covered in their courses.  During this process, faculty log-on to their course “SOCS” e-Assessment webpage (hyperlink to example) and see listed there under each of their courses a link to their e-Assessments of program standards.  Faculty then click on the link that corresponds to the e-Assessment Rubrics for their current semester course.  Their task is to complete the rubrics that are provided for each of student in the course.  For each student, they will click on “Input Score” and the scoring rubric(s) appears (hyperlink to CMHC example).  The Rubric Name includes the course assignments that provide the evidence for which choice to make related to the students’ performance in the class.  That is, faculty will consider the student’s performance on the assignments in the course (by looking back at specific rubrics created for each assignment) and complete the e-Assessment rubric(s) following the instructions listed for each standard (there may be only one rubric or several for each standard) (hyperlink to SC example). When the faculty make their choice(s), they then click “Submit,” and move on to assess the next student.  Through this process, they complete all of the e-Assessment rubrics for each student in their course. 

The following guidelines are provided for faculty when selecting their ratings:

4 – “Exceeds Standard” – This student showed an exceptional, creative, and insightful integration and application of material related to the standard.

3- “Meets Standard” – This student demonstrated an expected level of integration and application of the material related to the standard; less creative and insightful than a “4” rating.

2- “Approaching Standard” – This student demonstrated some integration and application of the material related to the standard and shows promise for fully meeting this standard during their program of study.

1- “Does Not Meet Standard” – This student’s assignments had weaknesses and/or showed insufficient development of the material related to this standard; further guidance from faculty is needed in order to meet this standard.

Please note several important points related to the use of Assignment Rubrics and e-Assessment Rubrics:

1.) Where relevant, Assignment Rubrics may be adapted by the faculty in consultation with the Department Curriculum Committee (which is composed of core full-time faculty). This allows for a flexible and adaptable annual evaluation process of how the assignments are meeting the program objectives;

2.) The e-Assessment Rubric ratings may correlate with student grades; however, they are not linked directly to a student’s grade in the course;

3.) If a student receives a rating of “1” on any e-Assessment Rubric, then faculty are asked to notify the Department Chair through email. The Chair is the faculty member who is responsible for following up with the student and their advisor in the program to help the student to meet program standards; and

4.) For transparency purposes, students will have access to their own e-Assessment ratings through their SOCS course webpage.

In closing, the data collected through the online e-Assessment system provides valuable information on two (2) levels (e.g., at the individual and program-specific levels) as follows:

1.)    As indicated above, when an individual student receives a “Does Not Meet Standard” rating in a course, then that student is afforded the opportunity to work with program faculty to develop a plan to satisfactorily meet the standard.  Therefore, the e-Assessment system allows for individual student evaluation; and

2.)    This e-Assessment system collects data in the aggregate that supports the necessary process of engaging in continuous systematic program evaluation that indicates how the mission, objectives, and student learning outcomes are measured and met, and ultimately helps faculty to ensure student learning and performance and to maintain program excellence over time.

5.      Evidence of the use of findings to inform program modifications.

As part of the annual evaluation of how, where and the extent to which program objectives are addressed in course syllabi (see AA.1. above and Section II.B., “Foundations”) the department faculty discuss ways in which to incorporate findings from this evaluation process as well as input from curriculum reviews and graduate follow-up questionnaires, alumni employer evaluations, and clinical site supervisors into program offerings and course syllabi.  On the basis of these annual reviews the department faculty authorized a number of changes in course content throughout the program. 

The following are recent changes resulting from this review process:

  • In anticipation of the new CACREP-accreditation standards that were released in July 2009, the Department of Counselor Education developed two (2) new courses that were approved through the College governance system in the spring 2009.  They are: EPSY 661: “Counseling At-Risk and Exceptional Children and Adolescents” and COUN 555: “Differential Diagnosis and Treatment Planning”.  We began offering these courses as electives in the summer of 2010 for both the School and Community Counseling programs.  More importantly, these two courses have been designed to improve our curriculum based on the changes in the 2009 CACREP standards.  For example, the “Counseling At-Risk and Exceptional Children and Adolescents” course provides breadth and depth to our curriculum in School Counseling, while the “Differential Diagnosis” course has  become a core piece in our new 60-credit Clinical Mental Health Counseling (CMHC) program that was developed in response to the July 2009 standards (described below).  

 

  • As described in Standard P. (in Section I, “The Academic Unit”), an evaluation form was created to collect data on students’ personal and professional development at key points during their program of study.  This form was implemented in the Fall of 2011 to be completed by course instructors at the conclusion of COUN 501 (Introduction to Counseling), COUN 670 (Counseling Theories and Techniques), and COUN 675 (Group Counseling).  These evaluation forms, along with a student completed self-evaluation form covering the same performance areas, are reviewed by the faculty when considering a student’s readiness to initiate the Practicum (COUN 690).  These evaluation forms are then completed one last time at the conclusion of the Practicum as students are evaluated for readiness to complete their culminating clinical sequence in the Internship Seminar (COUN 693 or COUN 694/695).

 

  • As described in AA.4. above, a student learning outcomes assessment system was developed by program faculty in response to the 2009 CACREP standards that provides systematic and continuous data collection related to the students’ acquisition of program-specific knowledge and skills (i.e., related to professional identity, professional practice, and program area standards). 

 

  • The department faculty made changes in the marriage and family counseling and therapy curriculum in anticipation of the College’s approval of a new 60-credit program in Marriage, Couple and Family Counseling and Therapy (MCFCT) (described in detail below).  Because we currently have a 24-credit Post-Masters Educational Specialist degree program in Marriage and Family Therapy offered through our department, we were able to create this new 60-credit MCFCT program without creating additional courses in marriage, couple and family counseling and therapy.

 

  • Since our accreditation in 2005, we have made adjustments to our clinical instruction.  For example, Dr. Atsuko Seto, who took over the role of Clinical Coordinator in the fall of 2008, has made adjustments to the “Clinical Site Approval Request” forms for Clinical Mental Health, Marriage, Couple and Family Counseling and Therapy and School Counseling.  In response to the July 2009 Standards (specifically, Section III, E. concerning Supervision Contracts), we have also created an “On Site Supervision Agreement” that addresses in more detail the supervision relationship, and includes spaces for signatures of agreement to the contractual expectations for both the student and the clinical site supervisor.

 

  • The 2009 CACREP standards substantially changed the accreditation process for our 48-credit masters-degree program in Community Counseling.  Specifically, the capacity to re-accredit our Community Counseling programs at the 48-credit level has been eliminated.  Students currently enrolled in the 48-credit Community Counseling programs have been advised to complete their degree requirements by May of 2013 in order to graduate within the timeframe remaining for our current Community Counseling accreditation status.

 

  • As a faculty, we developed a plan to meet the 60-credit standard for re-accreditation in 2013 based on the 2009 CACREP-standards.  In brief, we expanded of our accredited Community Counseling program options from 48-credits to 60-credit programs, which continue to lead to a Master of Arts in Counseling.  The two (2) 60-credit accredited-program options in Marriage, Couple and Family Counseling and Therapy (MCFCT) and the Clinical Mental Health Counseling (CMHC) help not only to maintain our CACREP-accreditation status, but are a relevant response to the  growing needs of our profession.  For example, a 60-credit masters-degree will match the educational requirements for licensure as Professional Counselors and Marriage and Family Therapists in New Jersey as follows: 
  • The old “Community Counseling: Human Services” program was expanded from 48-credits to a 60-credit program in Marriage, Couples and Family Counseling and Therapy. We are seeking accreditation for this program under the 2009 CACREP-standards for Marriage and Family Counseling programs.  We submitted documents to the state licensure board that described how graduates from this 60-credit program will meet the educational requirements for licensure as both a Licensed Marriage and Family Therapist (LMFT) and as a Licensed Professional Counselor (LPC) in New Jersey. For example, the 21-credits of curriculum for the “Required Courses in Marriage, Couple and Family Counseling” portion of this new program contains the same courses that are currently being offered through our Post-Masters Educational Specialist (Ed.S.) degree program in Marriage and Family Therapy that meets the educational requirements for the LMFT in New Jersey.  Practicum and internship sites are required to be family focused in their treatment orientation.  The two additional electives offer students the opportunity to increase their knowledge in such practice areas as addictions counseling and/or counseling adolescents, women and/or men.  Additionally, the core coursework (33 credits) for this degree continues to match the nine (9) core content areas outlined in the educational requirements for licensure as an LPC. 
  • The old “Community Counseling: Substance Abuse and Addiction” program was expanded from 48-credits to a 60-credit program in Clinical Mental Health Counseling. We are seeking accreditation for this program under the 2009 CACREP-standards for Clinical Mental Health Counseling programs.  We submitted documents to the state licensure board that described how students within this program will be able to choose to take electives in addiction counseling that would meet the educational requirements for licensure as both a Licensed Clinical Alcohol and Drug Counselor (LCADC) and a Licensed Professional Counselor (LPC) in New Jersey.  If students are interested in general practice as a mental health counselor, then they may choose to take a variety of electives with their core mental health counseling coursework.  This general practice option would meet the educational requirements for licensure as a Licensed Professional Counselor (LPC) in New Jersey, as all coursework will be specific to counseling practice, and the core coursework (33 credits) for this degree continues to match the nine (9) core content areas outlined in the educational requirements for licensure as an LPC. Practicum and internship sites for the LCADC elective option will be required to be substance abuse and addiction focused in terms of their treatment population. 

Documentation regarding course changes are in the respective course syllabi while programmatic revisions are reflected in The College of New Jersey’s Graduate Bulletin and the Department of Counselor Education’s Student Manual and Department Website.

The mission of every program in the department is to create exemplary counseling professionals prepared to enrich the lives of the people of New Jersey and the nation through counseling and service and to provide a national model of the preparation of counselors.  It is our hope that these program improvements will both serve this mission and the standards established by CACREP.

6.      Distribution of an official report that documents outcomes of the systematic program evaluation, with descriptions of any program modifications, to students currently in the program, program faculty, institutional administrators, personnel in cooperating agencies (e.g., employers, site supervisors), and the public.

Each year the Chairperson of the Department of Counselor Education prepares an Annual Report (Click here for an example of the most current Annual Report) which summarizes the program faculty’s review of the program objectives, curriculum and characteristics of program applicants; findings from follow-up studies of program graduates, program graduate employers and clinical site supervisors; and/or addresses curricular, programmatic and policy changes that have been implemented as a result of the review process.  Program faculty and institutional administrators are provided email notification that the Annual Report is posted on the Department of Counselor Education’s web site.  Students are informed through class announcements and e-mail of the availability of the Annual Report while personnel in cooperating schools and agencies such as site supervisors are informed via letter of its internet-based access. 

Currently enrolled students are informed of the results of program reviews through large group presentations, scheduled small group discussions, advisement and registration interviews, e-mail announcements, and/or small group discussions with their classroom professors. In addition, all entering Counselor Education students are apprised of any curriculum revisions, new offerings, and/or policy and procedural changes occurring as a result of recent program reviews during the new student orientation meeting at the beginning of each semester.  All students who are approaching the end of their studies and about to enroll in the Practicum/Internship clinical sequence receive additional program evaluation information specific to acceptable field placement sites, policies, and procedures. This information is provided during the Fall semester clinical orientation meeting (and a second Spring clinical refresher meeting) as well as through the Department’s Clinical Resources, the Clinical Manual for Students, and the Clinical Field Site Supervisor Manual. All of the latter are annually updated to assist students in identifying, obtaining, and engaging in well supervised, appropriately structured clinical experiences, and both clinical manuals are disseminated to students during the fall clinical orientation meeting.

BB.     Students have regular and systematic opportunities to formally evaluate faculty who provide curricular experiences and supervisors of clinical experiences.

Throughout the duration of their Department of Counselor Education degree program, students regularly provide both quantitative and qualitative evaluations of their instructional faculty and the curricular experiences provided by these individuals through the following assessment instruments.

On an ongoing basis students are asked to complete an evaluation of the instructional quality and content of each course they take (click here to review an example of The College of New Jersey’s Anonymous Student Feedback on Teaching Form and administrative instructions for the same). These evaluations are distributed, completed anonymously and collected in one classroom session within the final two weeks of the semester. Evaluations are then forwarded to Academic Affairs unopened where the scores are tabulated in summary form.  The evaluations are then returned to the department for review by the Chairperson who reviews and distributes the evaluations to the course instructors and discusses the same as needed to improve course/curriculum delivery.  These evaluations are also helpful to determine future hiring patterns of adjunct faculty in the department.

Clinical experiences are evaluated by students during the last semester of their Practicum/Internship clinical sequence (COUN 693 or COUN 695). The students’ clinical evaluations include the evaluation of the clinical site and evaluations of their individual and site supervisors. The information obtained is both quantitative and qualitative in nature and provides input on the effectiveness and appropriateness of clinical training experiences and clinical supervision as each relates to course objectives for Practicum (COUN 690) and Internship (COUN 693 or COUN 694/695).

Finally, during the last three weeks of their Internship (COUN 693 or COUN 695), students are asked to anonymously complete a comprehensive evaluation of their entire degree program (the following respective links are examples from the 2011 Department of Counselor Education Program Evaluations: Community Counseling and School Counseling). This questionnaire seeks student feedback on the quality of core and elective courses, faculty instruction, academic support and administrative services in the program, and program objectives).  Graduating students from the Clinical Mental Health Counseling (CMHC) program and the Marriage, Couple and Family Counseling and Therapy (MCFCT) program will be completing program evaluations at the conclusion of their May 2013 programs of study, i.e., these students will be our first graduating classes from the CMHC and MCFCT programs).

CC.     Annual results of student course evaluations are provided to faculty.

As indicated in response to Standard BB., faculty receive their completed TCNJ Student Feedback on Teaching Forms each semester after final course grades have been submitted to the Registrar.  Due to the confidential nature of student evaluations, the completed TCNJ Student Feedback on Teaching Forms are considered the property of instructional staff.  Faculty members who intend to submit these student evaluations as part of their performance review (reappointment, tenure, and/or promotion) must have these forms tabulated in summary forms which can be done by the TCNJ College Computer Center.

DD.    Written faculty evaluation procedures are presented to program faculty and supervisors at the beginning of each evaluation period and whenever changes are made in the procedures.

At The College of New Jersey program faculty are presented with detailed written descriptions of all faculty evaluation procedures at the onset of their employment, the beginning of each evaluation period, and/or whenever changes are made in these procedures. Proposed changes in or revisions to any of these evaluation procedures are also drafted and distributed to all The College of New Jersey faculty, appropriate college governance bodies, and the American Federation of Teachers’ Union (which represents the College faculty) for review and input prior to finalization and adoption.

In addition to the standardized The College of New Jersey Anonymous Student Feedback on Teaching Form and administrative instructions for the same that were discussed and documented in Standard BB., current TCNJ faculty evaluation procedures include those required for: Reappointment (3rd, 4th and 5th years), Tenure (6th year), Promotion, and Five Year Tenured Faculty Reviews. Comprehensive written guidelines that describe the criteria, standards, procedures, and timetables for The College of New Jersey Reappointment, Tenure, and Promotions review processes can be viewed by clicking here.  Faculty performance criteria and standards for Five Year Tenured Faculty Reviews are consistent with those cited in these Reappointment, Tenure, and Promotions materials. The three areas of performance that form the criteria for each of these The College of New Jersey faculty performance evaluations are: teaching ability, scholarly/creative/professional activity, and service.