***********************************

Table of Contents

• Kathleen Condray, Graduate Student

• John F. Lalande II, Associate Professor of German and Committee Chair

• James McGlathery, Professor of German

• Karl-Heinz Schoeps, Professor of German and Director of Undergraduate Studies

• Aaron Wright, Assistant Professor of German and Director of Graduate Studies (although initially assigned to the committee, Prof. Wright was subsequently unable to participate for medical reasons).

The plan for assessing and improving student learning was developed by the committee with assistance provided by the Center for Teaching Excellence (hereafter referred to as CTE). Finally, the plan was discussed, modified, and endorsed by the department as a whole in early Spring 1999.

The Department of Germanic Languages and Literatures is one of the largest and most prestigious of its kind in the United States. Its faculty represent diverse areas of study in:

• German linguistics, literature, culture, and pedagogy

• Germanic philology and linguistics

• Scandinavian languages and literatures

• Yiddish language, literature, and culture

The history of the department's degree programs can be traced back to the beginning of the twentieth century. In 1900 GLL awarded its first MA, eight years later its first Ph.D. Currently, GLL offers the following degree programs:

• BA in German (with four options: German and Commercial Studies, Language Studies, Language and Literature, and Modern German Studies)

• BA in the Teaching of German

• BA in Scandinavian

• MA in German

• PhD in German

A number of undergraduate students also pursue minors in German and the Teaching of German. Additionally, GLL offers a number of general education and other service courses (at the undergraduate and graduate levels) for other units on campus.

The general competencies developed by the undergraduate programs in GLL, particularly in German, reflect an awareness of, and commitment to, the National Standards for Foreign Language Learning (ratified by such major foreign language professional associations as the American Association of Teachers of German, the American Council on the Teaching of Foreign Languages [ACTFL], and the Illinois Foreign Language Teachers Association). See Appendix A for a copy of these Standards.

Undergraduate programs in GLL aim to assist students to develop professional competence in the context of a liberal arts and sciences education, with reasonable expectations as promulgated by ACTFL and its Proficiency Guidelines, as well as in the context of the National Standards for Foreign Language Learning. The faculty of GLL have identified the following general competencies desired of their graduates:

• subject-matter expertise (i.e., in culture, language, linguistics, and literature)

• communication skills

• critical thinking skills

• problem-solving skills

• ability to conduct research and independent study

• successful completion of international study, travel, living, or work experiences

• ability to utilize technology for professional growth

• insight into the languages and cultures being studied

• evidence of a liberal arts and sciences education, humanistic values, and intellectual integrity

• ability to teach German (only for those students in the Teaching of German)

• ability to adapt to foreign cultures.

GLL has identified the following specific competencies for the assessment of its undergraduate program:

The department regularly receives considerable input on student performance in the priority areas identified earlier in this plan. However, the preparation and implementation of a formal assessment plan provide an opportunity to make the department's assessment activities more systematic and rigorous, and to create more explicit and productive feedback links between assessment and program development/improvement. Because of the department's relatively small size and its limited resources for new outcomes-assessment activities, the initial plan focuses on more systematic collection and reporting of assessment findings gathered via methods that are largely in place and have proven their value.

I. Evaluation of International Experiences

Many of our students, particularly our majors and minors, live, work, study, and/or travel abroad. These international experiences afford students opportunities to develop, demonstrate, and refine a range of competencies, including subject-matter knowledge, critical thinking and problem-solving skills, communication skills, and perhaps leadership skills (such as might be realized through certain internship experiences). Recognizing the value of such experiences, the department encourages all of its students to pursue such opportunities. The percentage of our majors and minors who realize such an experience abroad constitutes one measure of our programs' effectiveness. Another, more precise indicator is the number of students who have successfully completed study and work experiences abroad.

Because study-abroad experiences require students to develop and use subject-matter knowledge, these experiences also represent an ideal juncture at which to assess student development. The department's study abroad program in Austria (Austria-Illinois Exchange Program [AIEP]) furnishes considerable information about how adequately our undergraduate program prepares students for the linguistic and cultural challenges posed by study abroad.

To assess students' growth in linguistic, communicative, and cultural competencies, the department (through the AIEP) administers spoken and/or written measures to evaluate the efficacy of its study abroad program. On-site departmental representatives (faculty and/or graduate students) are also asked to complete a written evaluation of individual students. Careful monitoring of AIEP-students' grades constitutes a valuable indicator of incoming students' preparedness and the AIEP's overall effectiveness.

II. Evaluation of Employability and Admission to Professional Schools/Graduate Study

Information collected from exit questionnaires and surveys sent to graduates will be administered every two years and reviewed biennially by the faculty to determine program effectiveness and implement possible modifications.

The same data will be reviewed to gather information about graduates who wish to pursue advanced studies (viz., where they were accepted/rejected, whether or not financial awards were forthcoming, etc.).

III. Tracking Course Grades and ICES Responses in Key Areas

The grades students receive furnish a rough assessment of their development of subject-matter knowledge and competency in other areas. GLL students' aggregate GPA in a key course or group of key courses can provide indications of competency; changes over time in a course's aggregate GPA can also signal whether or not changes may be indicated for certain courses or curricula.

Student responses on course evaluation (ICES) questionnaires can also furnish valuable information about students' perceptions of how a course has contributed to subject-matter knowledge, understanding of cultures, the development of communicative skills, positive affect toward the subject matter, as well as how well the course fits in the curriculum and meets other curricular objectives. Aggregate ICES scores for a course can indicate the general level of student perceptions; significant changes over time in aggregate scores might also indicate potential sites for improvement.

An ad hoc committee will be charged to develop a list of courses and course types and monitor their grades to determine if (and how well) the objectives of various undergraduate programs are being met.

IV. Other Methods of Measuring Students' Subject-Matter Competence 

Depending on the students' field of concentration, various measures can be employed to assess the quality of learning outcomes:

1) For students in the Teaching of German curriculum, passing scores on the two certification examinations administered by the Illinois State Department of Education should be considered. One examination is German-specific, the other tests pedagogical skills and knowledge.

2) For all undergraduate students in German, any one or more of the following are considered:

• successful completion of nationally and internationally recognized examinations of German language proficiency

• successful completion of a course taught in the target language on the target culture

• completion of a senior honors thesis (written in the target language).

Throughout a student's undergraduate course of study in the department, the examinations and research papers represent additional measures of evaluation through which attainment of the department's general and specific learning outcomes can be evaluated.

V. Open-Ended Assessment of Programs: Student Focus Groups 

The monitoring of students' work and study abroad and the monitoring of key course GPAs and ICES scores limits learning outcomes assessment to topics specified in printed questionnaires and evaluation instruments. The creation of focus groups with students majoring and minoring in GLL undergraduate programs will permit exploration of more complicated outcomes-related issues.

The department will establish focus groups that are to be convened in each of the following areas:

A GLL committee will develop protocols for each Focus Group Meeting, with topics to include:

Finally, this plan calls for the development of alumni questionnaires and the establishment of lines of communication with companies that hire our graduates to assess the efficacy of our programs. In keeping with a proposed plan of the College of Liberal Arts and Sciences, the department will undergo periodic and systematic reviews by external advisors to support programmatic improvement.

Develop process for monitoring student grades in key courses

Develop process for monitoring ICES scores in key courses

Identify key courses in the various options in the BA and BAT degrees

Review ICES Questionnaire Core of department

Establish departmental Assessment Committee

Develop measurement and evaluation guidelines for study abroad

Review and amplify ICES core (if and where appropriate)

Develop protocols for student focus groups (graduate and undergraduate)

Pilot-test measurement and evaluation procedures for study abroad students

Draft exit interview/questionnaire for graduating seniors

Implement new ICES core

Meet with first student focus group

Implement study abroad measurement and evaluation

Pilot-test exit interview/questionnaire for graduating seniors

Tabulate and interpret ICES core data (for courses)

Tabulate and interpret 2nd year of student grade data (for courses)

Review study-abroad evaluations

Refine exit interview/questionnaire for graduating seniors

Meeting with 2nd student focus group

Prepare 1st outcomes assessment report

First meeting with 3rd student focus group

First meeting with 4th student focus group

The department already has in place an annual review of the progress of the graduate students. The report is submitted by the Director of Graduate Studies to the graduate faculty. The graduate students and faculty annually participate in a retreat that focuses on relevant outcomes issues, such as preparation for an academic career. Furthermore, the department has initiated an annual self-study which is used internally for assessment purposes and which is reported to the Graduate College. The department's graduate faculty will continue to review the data gathered and consider actions deemed necessary for improving graduate courses and curricula.

Desired Learning Outcomes

The committee, which has prepared this plan for the assessment and improvement of graduate-student learning, proposes the following profile of the ideal graduate. It does so with a view toward the standards of the profession, contemporary and future needs/directions of the profession, and the career profiles of successful graduate students.

The mechanisms for assessing outcomes have been developed by the graduate faculty, which also reviews the documentation gathered and considers the need for revising program requirements, course requirements, and course syllabi. The assessment of the graduate program occurs both at the unit and Graduate College levels annually.

I. General Competencies/Outcomes

• to be well versed in theory, research methods, and the application of knowledge in the graduate's area of interest and related areas

• to be able to design and conduct independent research

• to begin to establish a publication record in well respected journals

• to begin to establish a record of service contributions to the profession

• to begin to form professional contacts

• to gain experience in college teaching

• to gain experience in grant-seeking and grant-writing

• to adhere to high ethical standards

To assess the above, the following are considered:

In addition to the Expanded Campus Profile and other centrally collected data, the following departmentally generated instruments are used in measuring the qualities identified above:

1) Qualifying Examination Assessment:

• design of the dissertation project

• presentation of the dissertation project

• breadth of knowledge demonstrated

• depth of knowledge shown

• ability to analyze knowledge

• ability to synthesize knowledge

2) Thesis Assessment:

• grounding in and application of theory

• soundness of methodology

• quality of research design

• strength of analysis

• soundness of findings

• merit of interpretations and conclusions

• potential impact on the field

3) Short-term Follow-up Questionnaires from Graduates (2 to 5 years after receipt of the doctoral degree) reporting on the following: job placement; salary; publications; grants and fellowships; other awards; career satisfaction; tenure and promotion

4) Long-term Follow-up Questionnaires from Graduates (10 to 15 years, possibly also 20 to 25 years, after receipt of the doctoral degree) reporting on the following: career milestones; salary; publications; grants and fellowships; other awards; career satisfaction; tenure and promotion; reviews of, and citation rates for, publications

As noted previously, the monitoring of students' work and study abroad as well as the monitoring of key course GPAs and ICES scores limits learning outcomes assessment to topics specified in printed questionnaires and evaluation instruments. The creation of focus groups of students in GLL graduate programs will permit exploration of more complicated outcomes-related issues.

Three focus groups will be established, which are to be convened in each of the following areas:

1) literature

2) linguistics

3) SLATE (Second Language Acquisition and Teacher Education)

Groups will meet on a rotating basis once every three years, beginning in AY 1999–2000.

All students with a commitment to one or more of the above areas may participate in the focus groups.

A GLL committee will develop protocols for each Focus-Group Meeting, with topics to include:

1) students' satisfaction with the quality of instruction and the availability of formal and informal opportunities to develop the expected competencies

2) students' appraisals of changes in the quality of instruction, requirements, or opportunities during their time in the department

3) students' perceived view of departmental strengths and weaknesses

4) students' suggestions for improvement.

Finally, alumni questionnaires will be developed. Furthermore, in cooperation with the Graduate College, annual reviews of the graduate program will be supplemented by periodic in-depth reviews by an external team of reviewers.

Timetable