Institutional Progress Report
Western Connecticut State University has been engaged in the assessment of student learning outcomes since 1988. Beginning in 2001, the Vice President for Academic Affairs required departments to submit plans for the assessment of student learning outcomes. All new program proposals submitted to governance review were also required to include assessment plans. The Assessment Committee developed guidelines, reviewed the plans, and issued a report of good practices in several departments. In most cases, however, there was little or no evidence of assessment-based change for improvement of learning. In 2004, the Commission on Institutions of Higher Education commended the University for the in-depth assessment practices undertaken by its programs that hold specialized accreditation. However, the Commission concurred with the visiting team that WestConn had yet to develop a “University-wide conversation on assessment.” The fall 2006 report provides an opportunity to report on the University’s success in developing and implementing a comprehensive approach to the assessment of student learning.
Efforts Made to Address Specified Concerns
The University has worked consistently and successfully to implement the assessment of student learning outcomes across the curriculum as stipulated by the NEASC reaccreditation report of March 2004. An Action Plan assigning responsibility for assessment policy and administration was adopted and implemented in the fall of 2004. To assist faculty and deans, an assessment coordinator was appointed for each of the three schools and the Division of Graduate Studies, and a special allocation of $25,100 was made for training and expenses. The Busy Person’s Guide to Assessment was distributed to provide guidelines for faculty and department Chairs. Several meetings were convened to clarify what constitutes evidence of student learning, and what is necessary to document faculty reflection on assessment results.
Deans and coordinators stressed the necessity of reporting all five stages of the assessment cycle:
- Determining program goals and objectives
- Gathering direct evidence of student learning outcomes
- Interpreting the evidence
- Making changes for improvement
- “Closing the loop” by gathering information on the effectiveness of changes and/or by focusing on different objectives and repeating the process.
Each academic dean was charged with the responsibility of: (1) reviewing the assessment status of all programs in the school or division; (2) providing copies of the NEASC statements on evidence to all department chairs; (3) asking departments which were already successful in assessing learning outcomes to agree on a common format for reporting assessment results; (4) conferring with the chairs of departments judged to be in need of assessment and explaining the procedures; and (5) ensuring annual reporting of assessment findings, including changes in instruction and curricula.
As a result of intensive efforts, 50 degree programs have implemented assessment plans (14 graduate and 36 undergraduate). Only the A.S. degree program in Liberal Arts and the M.A. program in Earth and Planetary Sciences are missing assessment plans. Work on assessment of the A.S. program is underway. The M.A. program has offered no coursework for two years and is being reviewed for continuance. The latest annual assessment reports were submitted to Academic Affairs in May and June 2006. Faculty have set goals, collected and analyzed assessment data for at least one year, discussed the findings in department meetings, and proposed curriculum modifications. Some recommendations can be implemented immediately, for example, standardizing syllabi across all sections of an introductory course, increasing emphasis on topics that prove most difficult for students in capstone courses, and advising students of alternatives to previously recommended junior-level courses. Other proposals include changing course prerequisites and designing new courses. As might be expected, scoring rubrics and other measurement instruments have been revised if found to be vague or otherwise inappropriate. The next section summarizes implementation in each of the three schools.
Examples of the Assessment of Student Learning Outcomes in Each School
In order to summarize trends in the implementation of assessment of student learning, we created a typology of outcomes and applied it to each degree program: (1) analyses were conducted and no substantive changes are considered necessary; (2) changes in instruction were implemented as a result of review of assessment findings; and (3) changes in curriculum were implemented as a result of review of assessment findings. This section contains a review of trends in each of the three schools, and the Appendix contains more detailed descriptions of recent assessment activities for each degree program.
Macricostas School of Arts and Sciences
Eleven programs were in the “analysis” category (1): American Studies BA; Medical Technology BS; Chemistry BA; English Literature BA; Professional Writing BA; Mathematics MA; Meteorology BS; Social Sciences BA; Anthropology/Sociology BA; Political Science BA; and Economics BA. Seven programs were in the “instruction” category (2): Art BA; Visual Arts MFA; Communication BA; Mathematics BA; Psychology BA; Earth and Planetary Sciences BA; and Theatre Arts BA. Seven programs were in the “curriculum” category (3): Biology BA; Biology and Environmental Science MA; Computer Science BA; English MA; History BA; History MA; and Spanish BA. Additionally, the new Professional Writing MFA program has plans to conduct a portfolio review in 2006-07.
Three examples of changes in Arts and Sciences programs. First, as a result of analysis of assessment data from the capstone course, the department of Biological and Environmental Sciences is revising the senior research project structure to better address several objectives, including: (1) design and execution of experiments; (2) understanding of the scientific process and methodology; (3) application of quantitative methods; and (4) demonstration of independence, creativity, and initiative in scientific design. Additionally, the course in Scientific Communication will be reorganized, expanded, and offered earlier in the course sequence for majors. Second, the B.A. program in History was evaluated by sampling final papers from 2002 through 2005 from the Senior Seminar. Analysis of the papers revealed proficiency in the students’ choice of appropriate topics and in the quality of scholarship, but the building of arguments was somewhat weaker. In response to these findings, the department is developing more rigorous writing requirements in two required courses, Introduction to Historical Research and Historiography. Third, the Theatre Arts senior portfolio (BA), comprising a written proposal, oral presentation, and written final paper, was used to assess three objectives: (1) students will perform competently on creative projects, technical projects, or theatre management; (2) communicate clearly orally and in writing; and (3) apply relevant theories and develop research tools. All full-time theatre faculty evaluated 13 portfolios using a department-developed rubric. Faculty concern focused on areas where substantial numbers of portfolios were rated below the “proficient” level. After discussion of the assessment findings, faculty resolved to implement four recommendations: (1) tell students to begin assembling the portfolio earlier in the semester; (2) integrate a greater diversity of plays across the curriculum; (3) develop a handbook to aid students develop a style of presentation; and (4) begin assembly of production books for all main-stage shows as a resource for students.
Ancell School of Business
Two programs were in the “analysis” category: Master of Health Administration and Management Information Systems BBA. Five programs were in the “instruction” category: Accounting BBA; Finance BBA; Management BBA; Justice and Law BS; and Master of Justice Administration. Two programs were in the “curriculum” category: Marketing BBA and Master of Business Administration.
Three examples of changes in the Ancell programs. First, authentic assessment of the Finance BBA program focuses on two types of portfolio management: an Internet investment simulation in Investment Analysis and Portfolio Management, and case studies in the capstone course, Cases in Managerial Finance. Changes to instruction include increased emphasis on cash flow analysis and problem solving, in response to demonstrated weaknesses in student projects. Second, in the Justice and Law Administration BS program, a sample of ten papers was selected randomly and student identification was redacted. Each of the papers was rated by three faculty members using the same rubric. Nine of the ten papers revealed an adequate understanding of concepts and eight demonstrated competent research skills. However, four papers were judged less than competent in written communication, and five were judged less than competent in critical thinking. To address the two areas of weakness, faculty will be encouraged to add writing components, put more emphasis on APA formatting, and stress critical thinking in each course. Third, the Marketing BBA program has a capstone course in which students are expected to develop a comprehensive marketing plan. One of the student learning outcomes is to transform the strategies into marketing programs that can be implemented. The plans are examined annually, both by faculty in the department and by external evaluators. Faculty review is documented in minutes of department meetings. External evaluators, most of whom are marketing plan clients, complete a rating instrument with several criteria. Instructional changes include an increased emphasis on secondary marketing research sources, data acquisition, analysis and statistics. Curricular changes include a two course sequence in Marketing Communications.
School of Professional Studies
Seven programs were in the “analysis” category: Elementary Education BS; Secondary Education BS; Counselor Education MS; Community Health BS; Music Education BS; Nursing BS; Nursing MSN. Four programs were in the “instruction” category: Education MS; Ed.D in Instructional Leadership; Music Education MS; and Social Work BA. Three programs were in the “curriculum” category (3): Health Education BS; Music BM; and Music BA. Additionally, the new BS program in Health Promotion has a well-developed plan for data collection and analysis.
Three examples of changes in Professional Studies programs. First, evaluation of the MS program in Education focused on two goals: (1) students in all options will demonstrate the ability to be both consumers of information and producers of educational research through the use of inquiry, critical analysis and synthesis; and (2) students will become proficient in the use of assessment techniques and analysis of student learning. Direct measures were applied to research papers produced in Introduction to Educational Research and to three assignments in Measurement and Evaluation. Two curricular changes were made as a result of reflection on the findings: (1) Introduction to Educational Research and the Educational Research Seminar have been redesigned to relate more directly to classroom practices; (2) although the combination of activities offered in Measurement and Evaluation satisfies the state and national standards for measurement and evaluation, all sections of the course do not emphasize the same concepts and outcomes. The course focus will be realigned across sections to ensure that students have a successful course sequence. Second, to measure changes in student abilities to assess individual and community needs for Health Education (BS), faculty devised a pretest-posttest approach. Content drawn from eight required courses was used to construct a test item bank. A 50-item pretest is given to all incoming students, and a posttest of similar difficulty will be given to all completers of degree programs. The pretest revealed a need for making two courses required instead of electives: Mental Health and Human Sexuality. Third, the Music and Music Education department maintains policies and procedures that contribute to the learning and assessment of its students. These include: (1) juries at the end of each semester for students enrolled in Applied Music; (2) upper division applied jury for students in Music Education, Performance and Jazz Studies; (3) interview for admission to the professional semester in music education, prior to junior year; and (4) ensemble auditions. Each of these performances provides opportunities for the collection of assessment data. One result of faculty reflection on assessment results is the adoption of stricter standards for admission to the B.M.-Jazz program as well as the sophomore barrier assessment prior to enrollment in upper division music courses. Another finding is that one-fifth of incoming majors were required to take Music Essentials. Because faculty consider this an unacceptably large proportion, plans are underway to implement an online theory assessment which will include tutoring in basic fundamentals.
Systemwide Changes. On June 10, 2005, the Connecticut State University (CSU) Board of Trustees adopted a resolution requiring the assessment of student learning for educational improvement at all CSU institutions, and annual reports from the presidents to the Chancellor. Additionally, the Board and Chancellor encouraged assessment efforts by providing funds for faculty assessment grants beginning with the 2003-04 academic year. Grant recipients are expected to present their findings at a system-wide assessment conference the following spring. A total of eight proposals from WCSU faculty have received funding.
Plans for the Future
After the Assessment Committee issued recommendations for the annual reporting of assessment plans and results, an Action Plan for implementation was approved by the Academic Council in 2004. Accordingly, every department with degree programs will continue to issue an Assessment Update as part of its annual report each spring. The Assessment Committee will continue to review plans and reports and issue recommendations for modifications of assessment practices throughout the University. The newly-formed General Education Committee, also a standing committee of the University Senate, will assume responsibility for the assessment of the general education curriculum.
Assessment of Student Learning Outcomes by School and Program
Macricostas School of Arts and Sciences
Art. Faculty focused on two learning objectives for students in the Visual Arts BA program. At the senior thesis exhibition, 25 students were assessed using a department-developed rubric. Evaluations of “proficient” or “superior” were given to 92% of the students on the first objective, “ability to craft work in a quality manner.” Evaluations of “proficient” or “superior” were given to 88% of the students on the second objective, “acquisition of compositional skills.” As a result of the assessment, the department will consider implementing a screening instrument such as portfolio review. The core of the Master of Fine Arts (MFA) in Visual Arts program is the student’s individual portfolio developed over four semesters of increasing intensity designed to develop individual originality and enhance expertise of technique and expression. Student portfolios are evaluated weekly by resident faculty and biweekly by visiting artists. Day-long midterm and final group critiques are conducted with resident faculty. The culmination of the program is a public thesis exhibition, which is open to peer review by visiting artists, advisory board members, university officials, faculty, students, media representatives, and the public. A curriculum modification initiated in 2005-06 was an exhibition of graduate work in New York. This external review has been formally added to the program. In 2006-07 faculty will work on tying evidence to specific measures and refining a standardized rubric for critiques.
Biological and Environmental Sciences. As a result of analysis of assessment data from the capstone course, the department is revising the senior research project structure to better address several objectives, including: (1) design and execution of experiments; (2) understanding of the scientific process and methodology; (3) application of quantitative methods; and (4) demonstration of independence, creativity, and initiative in scientific design. The course in Scientific Communication will be reorganized, expanded, and offered earlier in the course sequence for majors. In June 2006, three faculty members were awarded a CSU Assessment of Learning for Educational Improvement Grant for the "Pilot Project to Measure Student Achievement and Learning Gains in the B. A. Biology Degree Program." The ETS major field test in Biology will be used to measure content knowledge.
The B.S in Medical Technology (MT) program focused on three key objectives for student learning in 2005-06. First, successful placement in a hospital internship program was assessed by examining data for 25 students over a decade. All were successfully placed in professional settings. Second, a survey invited hospital supervisors to rate each intern’s analytical and critical thinking skills as well as technical competence. Only one major had an internship in 2005-06. Third, certification and employment data were used to track 25 students over 10 years, and all were found to be certified and/or employed as medical technicians. Because of the small number of MT majors, it is advisable to combine several years of data before making recommendations for changes in curriculum and/or instruction. No program changes are considered necessary at this time.
The M.A. program in Biological and Environmental Sciences has five learning objectives: (1) the student will demonstrate an understanding of the scientific process and be able to apply it to the design and analysis of biological experiments; (2) demonstrate an in-depth understanding of the factual and theoretical bases of one or more areas of biology; (3) be able to communicate effectively in both oral and written presentations; (4) demonstrate the continuing ability to acquire the specific knowledge and understanding appropriate to a professional career; and (5) develop the computer and library skills necessary to continue learning beyond the M.A. degree. A committee examination assesses the first four goals, and the graduate seminar assesses the first, third, and fifth goal. Preliminary results of a review of the M.A. program in Biological and Environmental Sciences indicate that more laboratory courses are needed (e.g. Advanced Molecular Biology). Faculty will address gaps in the rubrics for the evaluation of the seminar and the comprehensive exam in 2006- 07. The format of the comprehensive exam is also undergoing revision.
Chemistry. The department continues to comply with the standards of its accrediting body, the American Chemical Society. Independent laboratory research evaluations measure both student technical competence in laboratory practice and application of principles to research. Co-op evaluations, in conjunction with a literature research project, are used to evaluate three objectives: (1) appropriate use of scientific literature; (2) quality of scientific writing; and (3) accuracy of verbal presentation.
Communication. Fifty undergraduate student theses were rated by a faculty committee in 2005-06 using a department-developed rubric. The use of the senior thesis and capstone course for program evaluation is new to the department, and faculty plan to refine the process. Specifically, three changes are planned for 2006-07: (1) assessment will be integrated into the teaching of Senior Thesis rather than occurring at the end of the semester; (2) students who do not complete theses in the prescribed time frame will be included in the analysis in an effort to discover reasons for noncompletion; and (3) in order to generate better recommendations for each option within the major, the department will undertake separate analyses of student theses representing each program/option: Elementary Education; Media Arts option; Human Relations option; and General Communication option.
Computer Science. Faculty focused on three student learning objectives for assessment of the B.A. program: (1) students will understand the facts, concepts, principles and theories relating to computer science; (2) in-depth understanding of software applications; and (3) develop abilities to design and implement computer programs. The scoring rubric has specific criteria which should allow different coders to agree on scoring. Four faculty members rated student programming assignments. The methodology involves sampling student programming assignments from three courses. Bar graphs allow the reader to compare seven dimensions of student learning in the three courses. Two dimensions, Completeness and Testing, were shown to have lower mean scores for students in Software Engineering than in Web Applications Development. After reflecting on data collected in 2004-05, the faculty implemented several changes to instruction and curriculum. For example, a special session on testing and documentation was added to the early required course, Data Structures, and two alternatives to the Software Engineering course were developed. Data from the two alternative courses will be analyzed in 2006-07 to determine whether student deficiencies have been addressed.
English. Student attainment of four key learning objectives was measured: (1) ability to organize a lengthy piece of writing; (2) appropriate academic documentation; (3) maintain own voice while incorporating research; and (4) mastery of standard English. Analysis focused on the application of a rubric to student work sampled from baseline courses and compared to work sampled from advanced courses. Faculty concluded that the rubric needs to be customized for each program/option in order to make specific recommendations. For example, the American Studies rubric should be different from the rubrics for Literature and Professional Writing. Moreover, the choice of baseline courses needs to be reconsidered. The relatively small sample sizes should be expanded next year to allow for more detailed comparisons of student attainment of the learning objectives. The M.A. degree program has three primary learning objectives: (1) students will demonstrate knowledge of key literary periods and authors; (2) demonstrate historical, philosophical and social influences; and (3) demonstrate strong writing and critical thinking skills. A faculty review of exams and theses was conducted in December 2005. Beginning in Fall 2006, there will be a portfolio requirement for degree completion. During 2006-07 the M.A. program will establish a plan for review of portfolios.
The Master of Fine Arts (MFA) program in Professional Writing was begun in August 2005 and does not yet have any graduates. There are three main objectives for student learning: (1) students will display competence in one creative and one practical genre; (2) students will improve their skills through hands-on experience in practica and internships; and (3) students will produce publishable work in each genre. Faculty review of student portfolios will address the first goal, while a sample of theses will address the first and third goals. Writing mentors will submit written evaluations of student work for goals (1) and (3), and internship supervisors will submit evaluations for goal (2).
History. The B.A. program was evaluated by sampling final papers from Fall 2002 through Fall 2005 from the capstone course (Senior Seminar). Analysis of the papers revealed proficiency in the students’ choice of appropriate topics and in the quality of scholarship, but the building of arguments was somewhat weaker. In response to these findings, the department is developing more rigorous writing requirements in two required courses, Introduction to Historical Research and Historiography. Candidates for the M.A. degree in History have the option of a comprehensive essay examination or a thesis. After reviewing recent essays and theses, faculty identified student deficiencies in research and writing. As a result of discussions within the department, three curricular changes are underway: (1) revising standards and procedures for the comprehensive exam; (2) adding a required seminar in Research and Historiography; and (3) strengthening the area of United States history with four new courses.
Mathematics. In May 2004, three faculty members received a CSU grant for “Assessment in Mathematics.” Three key learning objectives were addressed in the analysis of course projects: (1) written mathematical communication; (2) problem solving; and (3) technology related to mathematics. Projects were sampled from a range of courses: Calculus I; Intermediate Statistics; and Numerical Analysis. Writing skills were somewhat below the “proficient” level, but with evidence of greater proficiency in advanced courses. The rubric for problem solving proved to be inadequate; therefore, rubrics will be reviewed before they are applied next year. After discussing the findings, faculty approved a departmental writing guide for use in all courses that have a written project component. Recent assessment of the M.A. program focused on comprehensive examinations and research papers graded by three faculty members to assess the two main goals: (1) ability to understand and apply theoretical concepts of mathematics; and (2) develop skills necessary for business, teaching, or doctoral programs. The graduate committee will continue to meet annually to review the comprehensive exam results, written paper grades, and oral defense results.
Psychology. A departmental examination was used to evaluate two main objectives: (1) student proficiency in experimental and statistical methods; and (2) knowledge of content areas of Psychology. Of 50 students sampled, 68% were judged “superior” or “proficient” on both objectives, and 32% were judged “less than proficient.” In response to the findings, faculty recommended increased standardization of the introductory course. Eight topics are now mandatory, and breadth requirements were increased from four to five.
Physics, Astronomy and Meteorology. Students in the B.S. program in Meteorology presented weekly television weathercasts and faculty judged their proficiency on three main objectives: (1) processing weather graphics; (2) forecasting and assessing weather situations; and (3) oral communication. An end of semester discussion and presentation was also used to assess forecasting proficiency, written communication skills. In addition, students prepared daily forecasts for a national contest. All but one student on one objective was judged to be “moderately proficient” or better. Students in the B.A. Earth and Planetary Sciences Astronomy option and the B.S. Earth Science-Secondary Education program were evaluated according to three main objectives: (1) conceptual understanding; (2) ability to develop original research; and (3) appropriate utilization of computer technology. Pre- and post-testing in the General Astronomy course showed satisfactory growth in competencies. Student research reports in the same course showed appropriate levels of performance. Reports on observatory service showed that more structure was needed in the kinds of service required. Accordingly, specific requirements and training procedures will be developed. Finally, student research reports in Solar and Planetary Astronomy showed weaknesses in methodology. Faculty recommended improvements in methodology preparation in General Astronomy to better prepare students for Solar and Planetary Astronomy.
Social Sciences. Four objectives are assessed for each of the degree programs in the Social Sciences: (1) appropriate use of theories and concepts; (2) knowledge and use of research methodologies; (3) application of skills and behaviors; and (4) information technology and literacy. The portfolio requirement is new, with the first portfolios collected in May 2006. Pretest-posttest comparisons of knowledge and skill self-assessment approach were administered to students in Researching Social Issues and the Social Sciences Research Seminar. All four objectives showed adequate growth in the dimensions of competence from pretest to posttest.
Theatre Arts. The senior portfolio, comprising a written proposal, oral presentation, and written final paper, was used to assess three objectives: (1) students will perform competently on creative projects, technical projects, or theatre management; (2) communicate clearly orally and in writing; and (3) apply relevant theories and develop research tools. All full-time theatre faculty evaluated 13 portfolios using a department-developed rubric. Faculty concern focused on areas where substantial numbers of portfolios were rated below the “proficient” level. After discussion of the assessment findings, faculty resolved to implement four recommendations: (1) tell students to begin assembling the portfolio earlier in the semester; (2) integrate a greater diversity of plays across the curriculum; (3) develop a handbook to aid students develop a style of presentation; and (4) begin assembly of production books for all main-stage shows as a resource for students. In June 2006, three faculty members were awarded a CSU Assessment of Learning for Educational Improvement Grant for “Assessment Rubrics for the Theatre Arts Department’s New Performance and Design/Technology Outcomes and a Curriculum Chart for Implementing a Process for Evaluating Learning Outcomes at Specified Markers within the Degree Program.”
World Languages and Literature. Student writing samples from Spanish conversation and literature courses were evaluated using a department-developed rubric based on national guidelines. Three key learning objectives were addressed: (1) variety in sentence components; (2) support of opinions; and (3) transitions between subtopics and distinctions between principal and secondary ideas. Student performance improved from 100% below proficiency at the beginning level to 70-90% “proficient-to-superior” evaluations at the upper level, depending on objective. After reflecting on the results of the assessment, faculty recommended more complex writing assignments for beginning level students. SPA 203, currently a conversation course, is being modified to include composition in 2006-07. Moreover, a research paper will be required in all 300-level courses, and there will be ongoing assessment of writing proficiency at all levels.
Academic Advisement Center (AAC). For the second successive year, the AAC has used assessment data to guide the development of its services. By University policy, Exploratory Studies majors (previously known as “Arts and Sciences Undeclared” students) are expected to choose disciplinary majors before completing 60 credits of coursework. They are also expected to make satisfactory academic progress – maintaining better than a 2.00 grade point average and completing 24 credits each academic year. All Exploratory majors were contacted by the AAC and offered advisement. The “Peer to Peer” (P2P) mentoring program was initiated in Fall 2005. Students on probation and others considered at risk of dropout were contacted biweekly and their progress was monitored by AAC staff. An electronic portfolio pilot program, “Major Discoveries,” was implemented in spring 2006. Students were guided by an advisor to develop ePortfolios that would help them to assess their interests and abilities. Seven students completed ePortfolios, and all of them declared majors by the end of the semester. One of the Assistant Directors was awarded a grant of $2,750 from the Connecticut Distance Learning Commission to support an ePortfolio project during 2006-07.
Ancell School of Business
Accounting. Faculty improved the assessment grid for the Bachelor of Business Administration (BBA) program by mapping learning objectives to the required courses in which attainment of the objectives is to be demonstrated. Analysis of the results of a comprehensive exam revealed a clear pattern of lesser competence in areas of cost behavior, job order, and process costing. These topics will receive greater emphasis in 2006-07. Supervisors rated the oral and written communication skills of 22 interns on a four point scale. The same set of internship evaluations revealed that all interns were rated competent or highly competent in computer technology skills. In 2006-07 the department will meet to discuss implications of the comprehensive exams for curricular change, as well as possible improvements to the internship program.
Finance. Authentic assessment focuses on two types of portfolio management: an Internet investment simulation in Investment Analysis and Portfolio Management, and case studies in the capstone course, Cases in Managerial Finance. Beginning in 2004, these projects were reviewed by faculty. The rubric for evaluating the presentation of case studies has six items, including knowledge of the financial topic, analytical competency, and ability to solve a financial problem. Each item may be judged inadequate, adequate, or proficient. More specific examples would help to clarify distinctions between adequate and proficient performances. In addition, student projects in Financial Decision Models and Quantitative Methods in Finance are reviewed by faculty teams. Changes to instruction include increased emphasis on cash flow analysis and problem solving, in response to demonstrated weaknesses in student projects.
Justice and Law Administration (JLA). The final research paper in the Research Seminar in Justice and Law was used to measure student attainment of four key learning objectives, including: (1) communicates effectively in written composition; (2) understands concepts of Justice and Law Administration; (3) exhibits critical thinking within the discipline; and (4) demonstrates research capabilities. A sample of ten papers was selected randomly and student identification was redacted. Each of the papers was rated by three faculty members using the same rubric. Nine of the ten papers revealed an adequate understanding of concepts and eight demonstrated competent research skills. However, four papers were judged less than competent in written communication, and five were judged less than competent in critical thinking. To address the two areas of weakness, JLA faculty will be encouraged to add writing components, put more emphasis on APA formatting, and stress critical thinking in each course.
In May 2005, one faculty member was awarded a CSU Assessment of Learning for Educational Improvement Grant for “Using the Final Graduate Research Project as a Tool for Student Learning Assessment.” A set of seven rubrics was developed for reviewing ten final research projects of candidates for the Master of Science in Justice Administration (MSJA). With some minor exceptions in the areas of leadership and political advocacy, all MSJA students demonstrated strong proficiency in each of the areas studied. The reviewer met with the professor who is currently teaching the research course. He will ensure that all of the rubric areas are reviewed with students early in the research class, and that the research papers will address each area.
Management. The department focused on two objectives for its BBA program in 2005-06. First, faculty developed and revised a class presentation instrument and administered it to students in the Current Issues in Management course. The primary purpose was to assess written and verbal communication skills for professional business settings. Second, faculty developed a Team Effectiveness Critique for application to group projects. Each team member was asked to rate 11 dimensions of effectiveness for its team. Endpoints of the scale were specific statements, such as “One person dominates…” and “There is full participation in leadership….” Curriculum mapping was also utilized to gather information on the methods for assessing student learning in all six program objectives. Plans for 2006-07 include working with the Career Development office to enhance students’ understanding of internship and co-op opportunities. Alumni surveys indicated that these forms of service learning were extremely beneficial. The department is also considering the development of a course embedded assessment for its capstone experience.
Candidates for the Master of Health Administration (MHA) degree are required to complete a final project in conjunction with the capstone course, Strategic Management in Health Care Organizations. Examinations and case analyses are also used to assess student attainment of three primary goals: (1) students will demonstrate a thorough understanding of the concepts, tools, and techniques of health services finance: (2) in-depth knowledge of the history and development of the American health care system; and (3) an understanding of health services management practices. In the spring of 2006, outside evaluators were chosen from the university and a nearby hospital. An assessment rubric was used both for peer review and external review, and the scores were examined for consistency. No program changes are considered necessary.
Management Information Systems (MIS). Three key objectives of the BBA program in MIS have been assessed in recent years: (1) students will be able to assume responsible MIS roles in public and private sectors; (2) students will display strong communication and teamwork skills; and (3) students will be able to solve business problems using analytical and computer skills. By mapping these objectives to the course sequence, faculty made explicit the assignments and projects designed to strengthen the appropriate skills. The Career Development Center provides feedback from employers and supervisors of co-op students. In the past two years, MIS majors were rated as “competent” or “highly competent” in oral communication, critical thinking, problem solving, group work, and productivity.
Marketing. The BBA program has a capstone course, Marketing Management, in which students are expected to develop a comprehensive marketing plan. One of the student learning outcomes is to transform the strategies into marketing programs that can be implemented. The plans are examined annually, both by faculty in the department and by external evaluators. Faculty review is documented in minutes of department meetings. External evaluators, most of whom are marketing plan clients, complete a rating instrument with several criteria, including thoroughness, usability, situational analysis, and plan integration. Aggregate ratings are converted to 100 point scales, and displayed as bar graphs for the criteria in two years. The capstone assessment method has been in place for five years. Several changes to both instruction and curriculum have been implemented based on the findings. Instructional changes include an increased emphasis on secondary marketing research sources, data acquisition, analysis and statistics. Curricular changes include a two course sequence in Marketing Communications, and specific prerequisites for the capstone course. Major revisions to the curriculum were implemented beginning in 2003 in order to strengthen the research, technology, and communication skills of marketing majors. In May 2004, the department chair was awarded a CSU Assessment Grant for “The Development of a Quantitative Instrument and Methods for the Assessment of Interactive Objectives for a Capstone Course Project.” The department has also monitored the performance of student interns since 1999. Supervisors consistently give excellent ratings to interns, judging them either competent or highly competent on all criteria. However, there is room for improvement in the area of written communication. The department continues to make written communication an important aspect of all Marketing courses and monitor the intern evaluations.
Master of Business Administration (MBA). Two key learning objectives for MBA students were assessed in 2005-06: (1) students will demonstrate knowledge of the fundamental business and organizational functions and activities, and (2) students will demonstrate ability to think strategically about business and organizational problems. The capstone course, Strategic Management, provided opportunities for faculty and external reviewers to evaluate student projects. Five faculty members served as evaluators of capstone projects in spring 2005, and gave each project a rating on a 100-point scale for each of 14 dimensions, including “understanding company needs” and “implementation detail.” Faculty will continue to refine the capstone rubrics as needed. After discussion of the project ratings, faculty modified the course sequence to strengthen students’ quantitative skills. Specifically, two prerequisites were mandated: (1) Statistics will be a prerequisite for Economic Analysis for Managers; and (2) either Statistics or Economic Analysis for Managers will be a prerequisite for Managerial Finance.
School of Professional Studies
Education and Educational Psychology. The department offers B.S. degrees in elementary and secondary education, M.S. degrees in Education and Counselor Education, and an Ed.D. in Instructional Leadership. Accredited by the Connecticut State Department of Education, the department is in the second year of a four-year plan to achieve NCATE accreditation. Accordingly, NCATE standards are used to evaluate student teacher competencies. Some work remains to be done in operationalizing NCATE standards and aligning them with state requirements. In May 2004, two faculty members were awarded a CSU Assessment of Learning for Educational Improvement Grant for “Assessment of the Dispositions of Teacher Candidates.” In June 2006, one faculty member was awarded a CSU Assessment Grant for “Assessment of Teaching: The BEST Portfolio.”
Three instruments were used for program evaluation of undergraduate elementary education. Two of the instruments, PRAXIS I and II, were designed and administered by the Educational Testing Service (ETS). PRAXIS I measures the level of candidate proficiency in reading, writing and mathematics. A passing score is required for admission to the Professional Development School experience (PDS). PRAXIS II measures knowledge and understanding of curriculum, instruction and assessment in language arts, mathematics, social studies and science. The third instrument is the state Student Teacher Evaluation Instrument (STEI). This instrument was designed to assess how well students accomplish their initial teaching experiences in actual classrooms around Connecticut. It is based on the Common Core of Teaching established by the state. For example, the question, “How did the teacher communicate with students about assessment and evaluate student performance?” is addressed by four performance indicators, each of which is rated on a continuum indicating levels of proficiency. PRAXIS II results for 2004-05 revealed that 96.4% of undergraduate elementary education candidates were proficient or better in language arts, mathematics, and science. Similar results were obtained using the STEI: 20.4% proficient and 79.6% superior in language arts, mathematics, and science. Similar results were obtained for undergraduate secondary education candidates. Based on the evidence of student proficiency, no changes in either curriculum or instruction are warranted at the present time.
Evaluation of the M.S. program in Education focused on two goals: (1) students in all options will demonstrate the ability to be both consumers of information and producers of educational research through the use of inquiry, critical analysis and synthesis; and (2) students will become proficient in the use of assessment techniques and analysis of student learning. Direct measures were applied to research papers produced in Introduction to Educational Research and to three assignments in Measurement and Evaluation. Two curricular changes were made as a result of reflection on the findings. First, Introduction to Educational Research and the Educational Research Seminar have been redesigned to relate more directly to classroom practices. In an effort to avoid conducting research with human subjects, the content of projects has tended to move away from action research to projects based on aggregate data which can be less meaningful for the student teacher. To counter this trend, an action research approach will be re-infused into the course sequence. Second, the combination of activities offered in Measurement and Evaluation satisfies the state and national standards for measurement and evaluation. However, all sections of the course do not emphasize the same concepts and outcomes. The course focus will be realigned across sections to ensure that students have a successful course sequence.
The Council for Accreditation of Counseling and Related Educational Programs (CACREP) provides standards for the assessment of student learning in the M.S. program in Counselor Education. Site supervisor evaluations of practica and internship experiences are used to measure the development of experiential skills important for the functioning of professional counselors. Student video sessions and clinical folders are reviewed for evidence of knowledge of the concepts, theories and practices of counseling. Results of the National Counselor Exam (NCE) are used to assess the students’ preparation for certification in counseling. Finally, faculty evaluation of the practicum research paper provides additional evidence of all of the competencies described above. All evidence gathered to date supports the efficacy of the program as designed and currently implemented.
The Ed.D. program in Instructional Leadership was designed to provide K-12 educators with the content knowledge and process skills to: (1) assume leadership roles; (2) become lifelong consumers and producers of scholarly research; (3) develop and implement innovative curricula; (4) implement school-wide professional development activities; and (5) implement activities consistent with emerging national standards. Evidence is being gathered to assess student expertise in four areas. First, all students have submitted Leadership Plans, and each plan was evaluated for content expertise. In addition, comprehensive exams will be conducted in the summer of 2006 and electronic portfolios will be evaluated. Second, all students have produced posters of their proposed research and program evaluation proposals. These materials will be evaluated for process expertise. In addition, dissertation proposals will be evaluated in December 2006 and defenses conducted in December 2008. Third, all students have developed plans to improve the use of higher order thinking skills in the classroom. These plans will be evaluated for curriculum development expertise. Fourth, all students have completed Professional Development Plans for their schools, and these plans will be evaluated using scoring rubrics. Even though no candidates have graduated yet, several changes have been made as a result of assessment activities. For example, a follow-up of the Leadership Plan is being incorporated into the comprehensive exam. Rubrics are being developed for program evaluation and professional development plans. The higher order thinking skills project is being revised to explicitly include a field experience component.
Health Promotion and Exercise Science (HPX) The HPX department offers a B.S. degree in Health Education, a B.S. in Community Health, and, beginning in Fall 2006, a B.S. in Health Promotion Studies. The department received Connecticut Department of Higher Education approval in January 2006 for the Health Promotion Studies Program, with options in Wellness Management and Community Health. In preparation for the new degree program, two faculty members applied for and were awarded a CSU Assessment of Learning for Educational Improvement Grant for “Assessing Senior Seminar Performance: Designing a Pilot Evaluation Program” (June 2006). Rubrics are being developed for assessment of student progress WCSU Report to NEASC Page 13 DRAFT 8/3/06 in key learning objectives, and implementation will begin in 2006-07. Students completing the degree in Health Education should be able to integrate the National Health Education Standards for students with relevant health content into a competency-based curriculum. Faculty are collaborating with the Connecticut State Department of Education in a student teacher pilot program. Internal and external assessments continue to evolve as necessary to keep state and national standards in alignment.
To measure changes in student abilities to assess individual and community needs for health education, faculty devised a pretest-posttest approach. Content drawn from eight required courses was used to construct a test item bank. A 50-item pretest is given to all incoming students, and a posttest of similar difficulty will be given to all completers of degree programs. First administered in January 2006, the pretest revealed a need for making two courses required instead of electives: Mental Health and Human Sexuality. The addition of a nutrition course as a major requirement is pending in the undergraduate curriculum committee. The same objective is also measured using evaluation forms completed separately by each student teacher and the supervising teacher. After completion of the student teaching assignment, each student has an exit interview with the faculty coordinator. The cooperating teacher assessment of the student’s performance is compared to the student teacher’s self-evaluation. The student’s strengths, weaknesses, and career goals are discussed during the exit interview. All eight student teachers were judged competent on all aspects of the Fall 2005 performance evaluation.
Music and Music Education. The department completed its fourth year as an accredited member of the National Association of Schools of Music (NASM). The department’s programs comprise a Bachelor of Music with Classical and Jazz Studies options, a B.A. in Music, a B.S. in Music Education, and an M.S. in Music Education. Key objectives of the B.A. program include: (1) performance ability, assessed at the end of each semester of applied study during jury exams; (2) knowledge of music history, theory, and analysis, measured during four semesters of music theory and two semesters of music history; and (3) skill-based areas, measured daily during four semesters of ear training and sight singing. At a jury exam, each student performs repertoire specific to his or her instrument or voice type. In addition, all students sight-read and perform rudiments (scales, arpeggios, percussion). There is a rubric for each area of competency.
Similar learning objectives exist for the professional degree programs (B.S. and B.M.). In addition, students enrolled in the B.M.-Jazz and B.M.-Classical programs perform recitals at the end of their junior and senior years. These recitals are evaluated by a faculty panel, and include extensive program notes which give the student an opportunity to display his or her theoretical, analytical and musicological knowledge of the works being performed. Students in the B.S. and the B.M.- Classical programs take a capstone course, Musical Form and Analysis, for which they are required to analyze 15 musical compositions and provide stylistic, theoretical and historical views of each work. Finally, B.S. students must demonstrate competence in planning, instructing and assessing the learning of K-12 students. They must also pass the PRAXIS I and II exams before receiving provisional certification (see the Education section of this document for details).
The department maintains policies and procedures that contribute to the learning, assessment, retention, and graduation of its students. These include: (1) juries at the end of each semester for students enrolled in Applied Music; (2) upper division applied jury for students in Music Education, Performance and Jazz Studies; (3) interview for admission to the professional semester in music education, prior to junior year; and (4) ensemble auditions. Each of these performances provides opportunities for the collection of assessment data. One result of faculty reflection on assessment results is the adoption of stricter standards for admission to the B.M.-Jazz program as well as the sophomore barrier assessment prior to enrollment in upper division music courses. Another finding is that one-fifth of incoming majors were required to take Music Essentials. Because faculty consider this an unacceptably large proportion, plans are underway to implement an online theory assessment which will include tutoring in basic fundamentals. Some work remains to be done before comprehensive assessment results are available. Jury sheets, currently hand-scored, are being adapted to scanner technology to facilitate analysis. The conversion should be fully implemented in 2006-07. Similarly, there is a proposal to administer the Advanced Measures of Audiation, a tool to measure music aptitude in adults, predict program success, and facilitate advising and placement.
The M.S. program in Music Education has three key learning objectives: (1) students will demonstrate proficiency in the teaching of music; (2) develop lifelong advocacy for music and the arts; and (3) develop the skills to be able to pursue advanced degrees in music and the arts. After review of student recitals, theses, and comprehensive examinations, faculty recommended one curricular change: the final project in the Music Department Seminar will include specifically the concepts and lessons learned in Introduction to Educational Research.
Nursing. The department continues to meet the standards of its national accrediting body, the Commission on Collegiate Nursing Education. The revised undergraduate assessment plan for the department encompasses two programs: (1) The B.S. program graduates nurses who have not yet obtained their RN licensure; and (2) The RN-B.S. program is for Registered Nurses who have been licensed and have completed an Associate degree program at another school. Both programs grant the same B.S. degree and have the same goals: graduates will be able to think critically, communicate effectively, and perform nursing interventions appropriate to their practice roles. Three types of direct measure of student learning are utilized: Preceptor Evaluation; Computerized Adaptive Testing (CAT); and Cumulative Professional Portfolio (CPP, for RN-B.S. only).
The preceptor evaluation was designed to be administered at the end of the capstone course (NUR 375). The CAT test is a standardized mock exam designed to prepare students to perform well on the National Council Licensure Examination for Registered Nurses (NCLEX-RN). CAT measures several dimensions of nursing competency, including pharmacological therapies, reduction of risk potential, physiological adaptation, safety and infection control, and basic care and comfort. One measure of the success of CAT and coursework is the 100% NCLEX-RN pass rate for WestConn students graduating in 2004 and 2005. In addition, all degree candidates scored 90% or better on CAT. The rubric for the Cumulative Professional Portfolio operationalizes the broad program goals. For example, critical thinking comprises use of the process of scientific inquiry and research findings to improve nursing care, in addition to implementing and evaluating the care provided to individuals families and communities. Application of the nursing process is also evaluated using reflective writing samples, including the capstone experience.
A survey of recent graduates provides indirect measures of student learning, including selfassessments of the same areas assessed in the Cumulative Professional Portfolio. Concurrently, a survey is mailed in to employers of graduates from the previous two years. Employers are also asked to rate the same skills and competencies used in the CPP. Over several years, one can compare the perceptions of faculty, employers and graduates to see the extent of agreement. During department meetings, faculty have reviewed recent results of preceptor evaluation, CAT, CPP, employer surveys, and alumni surveys. Nearly all measures reveal that graduates possess the necessary skills, knowledge, and values. However, the department is considering offering incentives to improve the return rates of the employer and alumni surveys. These programs continue to conform to the standards of the national accrediting body, the Commission on Collegiate Nursing Education.
The M.S. program in Nursing (MSN) is designed to prepare nurses for leadership roles as advanced practice nurses, with an option to prepare for certification as adult nurse practitioners. Theses and certification examination pass rates constitute the primary evidence for assessment of the following student learning objectives: (1) students will be able to use evidence-based nursing interventions to generate research for the purpose of expanding nursing science; (2) demonstrate expertise in the provision of care to individuals and groups from diverse backgrounds; (3) function competently in a variety of roles, collaborating with other disciplines to improve patient care outcomes; (4) continuously evaluate their nursing practice in relation to professional standards; (5) demonstrate comprehensive knowledge of policy formation and its impact on nursing practice and health care delivery; and (5) analyze ethical issues as they affect communities, society, the health professions, and their own practice.
Social Work. The department continues to meet the standards of its national accrediting body, the Council on Social Work Education. In 2005-06 the department focused on three student learning outcomes mandated by CSWE: (1) students will understand personal, professional, organizational and client system values and practice in a manner consistent with the NASW Code of Ethics; (2) use communication skills differentially across client populations, colleagues and communities; and (3) use supervision and consultation appropriate to social work practice. Each of the outcomes is measured by three instruments: (1) the Final Senior Field Practicum Evaluation (FSFPE); (2) assessments by field agency personnel; and (3) the Baccalaureate Social Work Education Assessment Package Exit Survey (BEAP). FSFPE, completed by instructors of the field practicum, evaluates student performance across nine areas in the two-semester practicum. This evaluation is conducted at the end of spring semester of senior year. Self-study data from 2003 compared to 2005 indicate that field instructors continue to give high ratings to students’ understanding of values and ethics, as well as communication and supervision/consultation skills. Assessments by field agencies are similar to internship evaluations used to evaluate of the degree programs. Every other year the department mails surveys to field agency personnel, including 15 items measuring students’ functioning in the context of program objectives. Nearly all agency staff (94%) responded that students “usually” or “consistently” applied professional values and standards to practice; and similar results were obtained for effective use of supervision and consultation (98%). However, the results for communication skills were somewhat less positive: 86% of agency staff who supervised seniors reporting that those students “usually” or “consistently” applied communication skills appropriately; and only 75% of agency staff whose observations were based on juniors reported the same levels of communication skills. After discussing these findings, faculty recommended keeping course content as designed but with more emphasis on exploring student perspectives on the scope and nature of all types of communication.