Learning Outcome Assessment and Follow Up

Printer-friendly versionPDF version

Background

This section describes assessment procedures for measurement of learning outcomes in the Food Science Major, Department of Food Science, University of Guelph. The assessment procedures themselves will be evaluated and modified as appropriate.

Cross curricular learning objectives (LO) are described by year and by course in separate documents. The LO are those specified in the Undergraduate Calendar of the University of Guelph, grouped according to the learning realms prescribed by the Ministry of Education and Training, namely, knowledge, skills and affective. They also approximate the LO specified by the Education Committee of the Institute of Food Technology, which is the accrediting agency for the Food Science major. The LO are summarized in the following description of the Super Food Science Graduate.

  • Describes, explains and applies all basic sciences to all aspects of food from gate to plate.
  • Demonstrates a broad range of success skills.
  • Demonstrates positive affective qualities.
  • Has a sense of vocation and purpose.

Measuring Success

Learning assessment as prescribed here more or less follows the Kirpatrick Model for Summative Evaluation (www.e-learningguru.com/articles/art2_8.htm). This model defines four levels of assessment, namely, student’s reaction, learning results, behavior in the work place and business results. For our purposes the levels are defined as:

  • Level one. Students’ reactions to the program: likes, dislikes, best learning experiences, relevance etc.
  • Level two. Learning results: achievement of learning objectives as assessed during and soon after the program.
  • Level three. Behavior in the work place and generally. In what ways and to what benefit are graduates applying the knowledge and skills learned at Guelph?
  • Level four. Business results: in what ways has food science education helped employer organizations fulfill their missions or meet their objectives.

Four assessment tools are used. Student reactions (Level one) are obtained via group exit interviews and post graduate email surveys. Learning results (Level 2) are mainly assessed via individual student assessment. Behavior in the work place and business results (Levels three and four) are assessed by post graduate email surveys and employer interviews. These assessment tools are described below.

1. Individual Student Assessment

Course by course and cross curricular learning outcomes are assessed for individual students. Within course outcomes are measured by examinations, laboratory reports, technical reports, group projects, oral presentations etc. Course by course evaluation is the principal assessment basis for individual achievement of LO in the knowledge domain. In other words, achievement of course requirements is evidence that students also meet curricular learning objectives in the knowledge domain. Having said that, the port folios will provide opportunity for students to demonstrate achievement of learning objectives in all domains, including knowledge.

Vocational portfolios are the principal assessment tools of cross curricular learning outcomes for individual students. Students prepare portfolios in the fourth semester and update them for resubmission in the eight semester. The editing framework for the online portfolio can be viewed at (www.cfkeep.org/) and a sample student portfolio in its final presentation format can be viewed at (www.cfkeep.org/html/snapshot.php?id=9602977) . For the Winter 2005 offering of FOOD*2100, students used a modified version of the OAC portfolio guide. With permission from Fred Evers, Educational Development Resource Unit, Office of the Provost, I now plan to use a modification of the portfolio format prescribed in his course, ISS*432 Transition From School To Work. See (www.studentlife.uoguelph.ca/service/php/documents/SkillsPortfolioSpecifi... ).

Students use the portfolio to reflect on and illustrate learning achievements in all domains, but particularly in the Success Skills and Affective domains. To assist that process, both faculty and students will refer to performance criteria for assessment of numeracy, technical, and other success and affective skills.

A document describing technical skills, mostly related to laboratory and pilot plant performance objectives and another on numeracy are in preparation. With respect to other success and affective skills I propose to use a modified version of the base competencies criteria described by Evers et al., (The Base Competence: Skills for Lifelong Learning and Employability. San Francisco: Jossey-Bass, 1998). An evaluation rubric for base competencies is at: http://www.studentlife.uoguelph.ca/service/php/documents/SkillsChecklist.pdf 

2. Annual Group Exit Interview

The student email survey is followed by an informal discussion with the graduating class during the last semester of the final year of study. A senior faculty member (normally the undergraduate coordinator) leads the discussion and collates the comments. The discussion is initiated and primed at the discretion of the facilitator with questions such as:

  • Describe your most important learning experiences at Guelph?
  • Describe your most enjoyable learning experiences at Guelph?
  • What would you drop from the food science curriculum?
  • What would you add to the food science curriculum?
  • Comment on other aspects of your Guelph experience: Awards? Food Science club? Academic support?

3. Tri-Annual On-line Survey of Recent Graduates

An email survey of recent graduates (Appendix 1) is conducted by a senior faculty member, usually the Undergraduate Coordinator, with assistance from a faculty member in Teaching Support services. Recent graduates (2,3 and 4 years post graduation) are asked to make critical comments about the program. The survey is conducted on-line using an interactive website created and hosted by Teaching Support Services, University of Guelph.

When available, we will also evaluate the results of the Alumni Survey conducted by the Office of the Provost. This survey is made two years post graduation.

4. Tri-Annual Employer Survey and Focus Group

Ten employers who have hired at least one graduate from our program during the previous three years are invited to participate in an informal focus group facilitated by a faculty member external to the department and attended by the undergraduate coordinator and the department Chair. Continuous Program Improvement The following two processes are used to monitor course content, reflect on learning assessment results, and suggest and implement individual course and curricular improvements.

Continuous Program Improvement

The following two processes are used to monitor course content, reflect on learning assessment results, and suggest and implement individual course and curricular improvements.

4.1 Tri-Annual Faculty Survey of Course Content, Teaching Strategies and Learning Outcomes

Faculty are given descriptions of teaching strategies and learning objectives and asked to answer the following questions for each learning objective. A separate survey is completed for each course that the faculty member teaches or is responsible for.

a) Does this course contribute to this learning objective? If yes, answer questions b) and c).

b) What teaching strategy(ies) or learning opportunities, including assignments are used to meet the learning objective?

c) How is the skill level measured and evaluated?

This information is used to construct a table that for each course provides the course description, learning objectives grouped by learning realms, and principal teaching strategies. The result is a snap shot of existing content, teaching strategies and targeted learning outcomes. Through a process of gap analysis and further faculty feed back including the retreat described below, the table is edited to define and redistribute revised content, improved learning objectives and revised teaching strategies.

4.2 Tri-Annual Faculty Teaching Retreat

The results of all the surveys described above are presented and discussed at a tri-annual half day faculty retreat where faculty reflect on their course content and teaching approaches in the contexts of survey results and suggest micro and macro improvements to individual courses and the program. Those suggested changes are then translated into revised learning objectives, teaching strategies and evaluation techniques and applied in the subsequent academic years. In addition, regular undergraduate committee meetings and undergraduate updates at regular faculty meetings address curricular issues as they arise.