Assessment

Assessment of the Senior Class and Alumni

Saint Michael’s is proud of our strong ties to our graduates. We rely on our alums to inform us about how we are doing and in what ways the Saint Michael’s experience has influenced their lives. This assessment begins with our Senior Survey but also includes a post-graduation “destination survey” (Career Development Office) for recent graduates and other alumni surveys.

Senior Exit Survey: Alumni/ae assessment

Each year we undertake a survey of all graduating seniors (conducted in the senior capstone course) to determine student satisfaction with their overall educational experience and future plans. Selected survey questions may also correspond to queries on the CIRP, NSSE, Alumni and Mission surveys, as well as other questions related to academic advising and satisfaction with the curriculum in the students' majors. Academic departments are encouraged to utilize the occasion of the senior survey to customize their own departmental surveys for their graduating seniors. Results of the senior survey are shared with academic departments who can determine the satisfaction of their majors compared with graduating students in general. We track these indicators to establish trends and adjust our academic and student programming to better meet the challenges and opportunities presented by our students. Longitudinal results from survey questions allow the college to better understand these factors over time.

Post-College Destinations: Alumni/ae assessment

The Institutional Research Office, the Career Development Office and other areas (e.g. departmental faculty) annually gather data about graduating seniors' intended post-college destinations: whether that be graduate school, some type of service corps, or job and career opportunity. Responses in the Senior Exit Survey are augmented with a post card questionnaire by the Career Development Office six months later to determine specific outcomes.

Alumni/ae Survey: Part of our alumni/ae assessment

The College has regularly asked alumni/ae who graduated five years previously to complete a survey indicating their level of satisfaction with their Saint Michael's experience. The questions range from academic to co-curricular experiences, and include general satisfaction queries regarding their overall experience at the College and how it prepared them for their future careers.

Senior Surveys

Class of 2010
Class of 2009
Class of 2008
Class of 2007
Class of 2006
Class of 2005
Class of 2004
Class of 2003
Class of 2002

Longitudinal 2002-2010

Employment Outcomes Reports

Class of 2005
Class of 2007

Class of 1995-2002 Longitudinal Alumni Survey

College Wide Assessment Efforts

In addition to the specific assessment efforts in the four areas that focus on major points in our students' educational journey (Entering Students, Students in Residence, Retention and Alumni), we also utilize other periodic surveys and available data sets to analyze and better understand the overall educational environment within which Saint Michael's College operates. We characterize these areas as “college-wide” for they refer to the many other facets of a college education.

The Factbook and Dashboard Indicators

The College also maintains reports on a wide array of internal/external data that are required for annual submissions to national organizations, or helps with internal analysis and institutional effectiveness. This information is provided in the Fact Book which has over 150 pages of reports and comparative studies about the college and the educational environment in which it operates. Many of these Factbook reports can be found on this website. The College administration also keeps the Board of Trustees apprised of our progress using the Dashboard Indicators reports that the Board mandated a number of years ago. The dashboards represent more than 20 indicators that show snapshots of the college’s performance during the academic year. Many of the Dashboard indicator reports can be found on this website as part of the Factbook or other data link.

Mission and Religious Values Surveys

With the leadership and assistance of the Sociology Department, Saint Michael's has conducted a Mission survey and a Religious Values survey on a few occasions over the past several years. In addition, Saint Michael's has recently worked with a group of other Catholic colleges in New England to compare data on the Religious Values survey. Saint Michael’s also belongs to the Catholic Higher Education Research Cooperative (CHERC), and organization that promotes important research among Catholic Colleges.

Thanks to the efforts of Dr. Vincent Bolduc (Sociology Department) and students from his Research Methods class, a survey was conducted to ascertain the level of understanding and engagement that students have regarding the Mission of the College. In addition, the survey seeks to determine the importance that the religious character of the institution has for students, as well as the students' estimation of how well the College is fulfilling its mission.

Results of Mission Survey 2009

Spirituality and Ethics (NSSE data)

The NSSE asks several important mission related questions regarding spirituality, ethics, and community service, that we have collected and utilize to better understand how these factors contribute to the realization of our mission.

Spirituality and Ethics NSSE 2008

The Outcomes Assessment effort begins even before the young men and women arrive on campus and matriculate as First Year students. In recent years, the Admission Office has utilized a Student Profile developed in conjunction with the Campus Culture Committee to inform its progress in achieving enrollment related goals. With each new class (cohort) certain academic and other indicators are monitored to assess how they may relate to retention and other enrollment goals. Each year we also participate in the Cooperative Institutional Research Program (CIRP), and track a variety of measures over time to better understand the many qualitative, attitudinal and behavioral aspects of each new class and to determine whether any institutional responses are needed in light of the survey results.

Student Profiles

Nearly a decade ago, the Campus Culture Committee developed a 'Saint Michael's Student Profile' to help the College identify qualities and characteristics in students that would help promote a successful college career. The qualities identified by the Committee were placed in three broad categories: Academic, Social, and Personal. Incoming students arrive at Saint Michael's with demonstrated achievement in each of these areas. It is hoped that the profile will help us to better understand and describe the considerable growth and development that takes place during a student's college career. The working statement identifying the ideal Saint Michael's student follows:

Saint Michael's College seeks students who wish to become active members of a learning community whose principles embody excellence in the search for truth combined with a commitment to community service within the context of moral and spiritual growth. Students should be prepared for a rigorous intellectual experience through previous study and have developed a curiosity which draws them to active study in the liberal arts and sciences. In addition, students should value community life of the College and beyond. This commitment can be manifested by actions to care for other members of the community as well as an attitude which is other-centered, open to change, honors diverse human experience, and engages in the broader world. Both academic and social qualities are anchored in personal and spiritual values which assert the respect for human dignity and engage the serious ethical questions and moral implications of knowledge both within the classroom and in society at large.

The College collects a number of indicators (profile) that help describe various academic and other aspects of the incoming class in light of our enrollment goals and desire to enroll the best possible class within the means of the College. The profile can be viewed as part of a longitudinal report allowing the College to regularly assess these indicators and any related trends. Certain indicators within the profile may be used to track retention for a specific cohort of students.

Incoming Class Profiles, Fall 1999-2003
Incoming Class Profiles, Fall 2004-2008
Incoming Class Profiles, Fall 2010

Cooperative Institutional Research Program (CIRP)

Over the past decade, Saint Michael's College has participated regularly in the Cooperative Institutional Research Program, a national longitudinal study administered by the Higher Education Research Institute at UCLA. The survey provides information on students' self-reported pre-College attitudes and behaviors over a variety of areas.

Saint Michael's utilizes results from the CIRP survey to better understand each class of newly entering students, and to make comparisons over time. The College has identified a number of indicators from CIRP related to our Student Profile, and we track these indicators to establish trends and adjust our academic and student programming to better meet the challenges and opportunities presented by our students. Certain findings may be compared with results from the NSEE survey, the Senior Survey, and other survey instruments.

Fall 2008-2010 Saint Michael's College Selected CIRP Results

Like all institutions of higher education, we recognize the importance of retaining current students and supporting them in their pursuit of a degree from Saint Michael’s. While maintaining an accurate record is important in this effort, how we respond to the results of the surveys and instruments concerning our students lives at the heart of our success at retention. The Class Cohort Study follows certain demographic factors, major, projected and actual GPA, SAT, class rank, financial aid status, selected co-curricular activities and other characteristics thought to have significant explanatory value with regard to persistence to graduation.  We use this information to inform the development of policy and good practices.

We also participate in a national consortium CSRDE (Consortium for Student Retention Data Exchange), and compare our efforts and retention results with peer and aspirant schools. Finally, the College has instituted a systematic and comprehensive Exit Interview for students who decide to withdraw or take a leave of absence from the College. We explore the outcomes of these interviews for any patterns, trends or issues that ought to be addressed.

Saint Michael's College also annually reports graduation rate information in compliance with the Higher Education Act of 1965, as amended: Student Right To Know Graduation Rates

Retention Class Cohort Study

Most Saint Michael's students are categorized as "first year" students and are tracked throughout by their projected year of graduation (i.e. 2007 matriculates are the "Class of 2011"). Each year the College gathers data on these class cohorts including the following areas: demographics, concentration, financial aid status, projected versus actual GPA, selected co-curricular activities, exploratory students, and other factors thought to have significant explanatory value with regard to persistence to graduation. This information provides valuable insight for addressing challenges related to admissions, advising and retention.
Consortium for Student Retention Data Exchange (CSRDE)

The purpose of Consortium for Student Retention Data Exchange, is two-fold: to make available a national retention database and to provide a forum for discussing retention-related issues. The consortium is currently represented by more than 400 colleges and universities from all 50 states, Washington D.C., Puerto Rico, Virgin Islands, and Canada. Saint Michael's uses data on first and second-year students from 20 comparable colleges to track retention trends at the College and similar institutions. We also study comparable data on 4, 5, and 6-year graduation rates. Per agreement of consortium members comparative retention reports are for internal use only, so we have only included Saint Michael's College data here.

Retention at Peer and Aspirant Schools

The College collects data about retention at other schools that are like Saint Michael's. This data comes from standard sources such as the Integrated Postsecondary Education Data System (IPEDS), the US News magazine rankings, and University and College Accountability Network (UCAN). Comparison with other colleges and universities tells us whether our efforts are more or less effective than other schools. Retention data at peer and aspirant schools is for internal use only. This Department of Education website is provided to allow visitors to research specific colleges and to make comparisons among schools.

Exit Interview for Students Leaving Without a Degree

All students who withdraw or take a leave of absence from Saint Michael's College are requested to meet with the Associate Dean of the College for a one-on-one exit interview. Most of these students also meet with a representative of Student Life, such as a Resident Director, or even the Dean of Students. Students also complete a Withdrawing Student/LOA Survey which serves as the focal point for the exit interview. Data, and sometimes anonymous anecdotal information, is shared with various academic and student life groups as well as the Admission and Enrollment Management teams to ascertain any trends that would indicate changes in policies and approaches we use in attracting and retaining students. Because interviews are confidential no results of interviews are available online.

As students progress in their various academic programs at the College, we collect data in a variety of ways to assess our efforts and develop policies and practices in response to what we learn. On a regular basis we conduct surveys of students regarding their experience of Academic Advising and compare their responses with the experience of the advisors themselves. We also have participated in the National Survey of Student Engagement (NSSE) in nine of the last eleven years. In fact, Saint Michael’s was one of twelve institutions invited to participate in the original NSSE pilot survey.

Recently, we have begun to utilize selected questions from the NSSE to assess our students in terms of their learning outcomes in the Liberal Studies Area (LRE) of our curriculum. This follows closely on the heels of the successful development of assessment plans for each of our concentrations.

We realize that at various points some students may struggle with their academic programs, and in that light Saint Michael’s works at keeping a balance between intervention and allowing students to make decisions that sometimes have serious consequences. Several years ago the Provost and Dean of Students established an ad hoc group of academic and student life administrators to keep all parties alert to any students who may need additional attention and support in both Academic and Residential settings. The spirit of this initiative continues in the close working relationship between these offices.

Academic Advising

Academic Advising at Saint Michael’s College is both learning centered and student focused. This very important exchange between faculty and students is primarily an academic activity which focuses on facilitating the student’s capacity to make wise choices at the College in preparation for wise choices in life. Such a process is information rich, multi-layered, and developmental. With access to relevant information on courses and careers, drawing upon the advice of not only formal academic advisors but also other members of the College community and friends and family, students “learn” how to make wise choices over the course of four years.

Advising Survey

On a regular basis, we survey all undergraduates concerning their experience with their academic advisors. In addition to exploring student opinions about the effectiveness of advisors and student satisfaction with the advising process, we also ask students to evaluate their own roles as advisees. Finally, we share departmental specific findings with individual Department Chairs.

Academic Advising Survey 2009
Academic Advising Survey 2003
Academic Advising Survey 2003 vs. 2005
Academic Advising from Senior Survey 2002-2009

Assessment of the Learning Experience of Our Students

National Survey of Student Engagement (NSSE)

Saint Michael's College was one of twelve institutions selected in 1998 to participate in the original development and testing of the National Survey of Student Engagement. We have participated in the NSSE in nine of the last eleven years. The survey queries students about the amount of time and effort spent on studies and other activities which promote learning. It also seeks to demonstrate how institutions organize and structure other learning opportunities through the curriculum and activities that research has shown are linked to student learning.

We track these NSSE indicators to establish trends and adjust our academic and student programming to better meet the challenges and opportunities presented by our students.

Using NSSE in Assessing the Liberal Studies Curriculum

Selected NSSE data associated with assessing the Liberal Arts at Saint Michael's includes: high quality non-classroom interactions with faculty, integration of ideas across courses, coursework in arts and sciences, and extracurricular involvement. Comparisons with Carnegie Classification and all NSSE schools are also included.

NSSE Data Regarding the Liberal Arts

Classroom and Residential Alerts

Saint Michael's has put in place an interdisciplinary team of academic and student life administrators and staff that meets on a regular basis to discuss confidentially individual students who present certain challenges or issues that need to be addressed in a comprehensive way.

The team determines what level, if any, intervention may be required and who will take responsibility for managing the intervention. The purpose is to address early on any potential problems that could impinge on a student's learning and well-being.

A Message from Dr. Karen Talentino, Vice President for Academic Affairs

While assessment of learning within classes has always been central to the work of teachers, it is only recently that a concerted effort has been undertaken to evaluate student learning outcomes for the entire institution. Such an initiative arises from both internal and external factors.

As higher education has clarified its purposes through differentiated curricula and pedagogies, assessment has become more refined; educators now seek to join their institutional educational philosophy to student learning outcomes. Thus, the oldest traditions of intellectual work between teacher and student are revisited on a larger scale and with a wider sense of purpose.

To be sure, this initiative arises within the larger context of assessment driven with a sense of responsibility to external constituencies, both public and private. Institutional assessment is an ongoing dialogue which constantly seeks to clarify the educational philosophy of institutions and to find the best ways to assess whether established goals are being achieved. Ultimately, the best institutional assessment systems will reflect the unique qualities of the institution examined within the appropriate best practices.

The Saint Michael's College Student Outcomes Assessment Program is crafted to ask the questions and provide the tools to attain key academic goals. Although no assessment program will demarcate fully the complexity of the educational process, we have chosen an aggregate of instruments that we deem will best inform our judgments and strategic planning. Current assessment activities are sustainable within our present resources, and we will continue to refine and expand our efforts in order to improve our academic programs and student outcomes.