The College of Education plan for assessing outcomes in our graduate programs was prepared by the Graduate Programs Committee. This committee is comprised of faculty representatives who have graduate program responsibilities in each department. The current members of the committing include:

Scott D. Johnson (Chair), Human Resource Education

Terry Ackerman, Educational Psychology

Chris Brown, Education Organization and Leadership

Jan Gaffney, Special Education

Georgia Garcia, Curriculum and Instruction

Ralph Page, Educational Policy Studies

James Leach, Associate Dean (ex officio)

Members of the Graduate Programs Committee met on a regular basis to discuss outcomes assessment in the College and developed a work plan to formalize the process. Each committee member obtained information from their home department regarding existing methods of assessing student outcomes. Using this information, the committee identified areas of strength and limitations in the existing assessment process. A synthesis of the various departmental procedures for assessing outcomes led to the current College plan.

Our approach to outcomes assessment is designed to ensure that our graduate program produces graduates who meet specific characteristics. Although the actual characteristics vary somewhat across departments, the following list characterizes the program outcomes achieved by our "ideal" graduates.

1. Ideal graduates understand the scholarly literature of their area of interest and related areas.

2. Ideal graduates know how to conduct scholarship in their area of interest. They are able to do this both independently and in collaboration with others.

3. Ideal graduates are proficient in finding resources needed to conduct professional work. Resources include grants, investments, locations, staff, and permissions.

4. Ideal graduates have begun a publication record. They possess writing skills and know which publishers of books, journals, magazines, and newsletters are most important to their interests.

5. Ideal graduates have begun to assume collegial responsibilities. This includes regularly reviewing others’ writing, service to professional organizations, and membership in networks or support groups.

6. Ideal graduates display interest and ability in quality instruction.

7. Ideal graduates have developed interest and ability in communicating with a variety of audiences in appropriate ways. This variety includes audiences specializing in other areas of education and related fields, and it includes lay audiences.

The sustainability of an assessment system depends greatly upon the efficiency at which assessment data can be collected and analyzed. Our approach to assessment relies on existing data sources that are readily available and updated on a regular basis. We have also developed our assessment plan to rely heavily on the data sources and data collection methods commonly used by departments and College offices. This places little extra responsibility on faculty and staff to ensure that the desired assessment data are collected, maintained, and utilized for appropriate purposes. To further support the efficiency of the process the Instructional Programs Office in the College is currently designing a web-based data management and electronic forms system that will further support automation of our data collection and analysis process for outcomes assessment. The following section describes the measurement instruments and data collection and analysis techniques that will be used for the outcomes assessment process.

ANNUAL REVIEW OF STUDENT PROGRESS

Each department has implemented a plan to evaluate student progress on an annual basis. In addition to identifying students who are not performing at a desired level, the annual review process helps us identify problems in course scheduling, curriculum coverage, and the ability of our programs to adequately prepare students to reach the desired outcomes.

REVIEW OF INSTRUCTION

Evaluations of instructional quality are assessed using the campus ICES system and periodic peer reviews. Currently individual faculty and department heads receive the ICES results, but their use varies considerable across the College. We need to create a process to systematically study ICES results across the college and use the results to identify areas in need of instructional improvement. In addition, we need to consider creating a peer observation system focused on instructional improvement that is modeled after the approaches used in the College of Engineering and ACES.

PRODUCT AND PERFORMANCE ASSESSMENT

Product and performance assessment takes place at several points in a student’s program. Significant scholarly products are created through the early research requirement (Ph.D. only), research methodology requirement (Ph.D. only), cognate requirement (Ed.D. only), qualifying examinations, and thesis/dissertation project. Assessment of the final products and performances that result from these requirements are conducted by faculty committees using a categorical checklist. We recognize that the current summative approach of product assessment provides limited information for program improvement. Our plan is to create a more formative approach using new forms that more accurately align with the desired learning outcomes stated above. For example, preliminary orals are currently evaluated by a committee using a judgement of pass, fail, or decision deferred. Instead, we wish to create a system of formatively assessing dissertation quality using more specific criteria such as importance of the subject, originality or creativity of the work, organization and sequence of the dissertation, integrity of the writing, and preparedness for dissemination to academic or policy audiences. We will also implement a process of using review committees to discuss the quality of student work collectively and identify areas of program strengths and weaknesses. The resultant information will aid department’s in rethinking how academic material is taught to future students. Additionally, formative assessment of the dissertation can identify potential weaknesses in the overall curriculum, which can be rectified through the exchange or addition of courses and other requirements.

SHORT- AND LONG-TERM FOLLOW UP SURVEYS

Follow-up of College of Education graduates occurs at several stages by multiple units. The units most involved in the short and long term follow-up of graduates include the Instructional Programs Office (IPO), the Educational Placement Office (EPO), the Office of Development and Alumni Relations, and each of the six departments within the College. The majority of current follow-up efforts of our graduates serves demographic and public relations purposes.

The College needs to establish a more formal process to collect information from our graduates to more effectively assess program outcomes and improve future efforts. To meet this goal graduates should be contacted three times: (1) immediately after graduation, (b) five-years after graduation, and (c) ten-years after graduation. A small number of evaluative questions should be included in the contact materials. While one or two questions might be standard across the COE (e.g., value of the RMAC requirement), departments would construct the remaining items to specifically assess their master, advanced certificate, and doctoral programs.

The IPO should oversee the effort and work in conjunction with the Office of Development and Alumni Relations. In addition, the College should align its efforts with those of the all-campus alumni surveys that are distributed to selected graduation classes on 1, 5, and 10 year cycles by the University Office for Planning and Budgeting. Contact information for graduates is maintained in the University’s Alumni database. Changes in contact information would be coordinated by the College Alumni Office. Departments would send changes in contact information to the College Alumni Office and update the list on an annual basis. While departments have the greatest likelihood of maintaining current contact with alumni, the processing of the returns would be the responsibility of the IPO. In this way, efforts can be coordinated with already established procedures and results analyzed by departments and programs.

The assessment of graduate student outcomes is a valuable tool for the College of Education. Graduate students, faculty, and college administrators will receive formative information through a continuous process of outcomes assessment. This information can be used for three primary purposes. First, student outcome data provides graduate students with in-depth feedback regarding their performance and progress in the program. Second, the outcomes assessment process provides critical data to guide graduate program planning and identifies areas in need of improvement. Third, the information gained through the outcomes assessment process provides a greater understanding of the quality of the graduate program in the College of Education, which is used to support ongoing student recruitment and public relations efforts.

This plan for improving our outcomes assessment process and leading to program improvement will be implemented at three levels. First, the Instructional Programs Office (IPO), under the guidance of Associate Dean Leach and with the support of the Graduate Programs Committee, will coordinate the overall outcomes assessment process. IPO’s role will be to coordinate and conduct the administration and analysis of graduate follow-up surveys, review each department’s implementation of their annual student review plans, guide the development of new forms for evaluating student products, and oversee quality and progress dimensions of each graduate program milestone. IPO will also complete the construction of the web-based student record database that will allow the College admistiration, departments, and faculty analyze and share graduate student outcome data. Second, each department head/chair will be responsible for ensuring that graduate students reach the desired outcomes. This will involve collaborating with IPO to collect and analyze follow-up survey data, creating a department policy and procedures to ensure that faculty meet to discuss student attainment of outcome criteria, and ensuring that faculty discussions of outcome attainment leads to program improvements. Third, individual faculty will contribute to the outcomes assessment process through their role as individual evaluators of student work. Faculty are responsible for ensuring that their advisees are making timely progress toward their degree and assisting their students in the development of a graduate program that meets their individual needs. Faculty will assist IPO in the development of appropriate assessment forms for evaluating student performance on qualifying exams and conducting product assessments of their dissertations/theses. These forms will serve both a formative and summative purpose by identifying strengths and limitations in student attainment of the desired outcomes. Faculty will collaborate to synthesize the information collected from each milestone, discuss areas in need of program improvement and products, and identify program changes that address the newly identified needs.

All development activities should be completed during the 1999-00 academic year. Implementation of the outcomes assessment process will begin as various components are completed.

The College may request support from the Survey Research Lab and CTE to facilitate the process of creating better follow-up surveys. We may also request assistance to support the development of the web-based data management system.