Use Canvas outcomes and rubrics for program assessment and improvement

On this page:


Overview

Learning outcomes in Canvas can be published at any level of the Canvas academic hierarchy--campus, school/college, department, or course. When outcomes at the department level or higher are incorporated into course rubrics to assess student learning, the results can be aggregated across all courses under the purview of the academic unit where the outcomes reside for purposes of longitudinal assessment, accountability, and program improvement. This document provides guidance on how to implement Canvas outcomes to support your assessment efforts.

Note:
UITS strongly recommends collaborating on your outcomes implementation with the instructional design and/or instructional technology consultants in your campus teaching and learning center. There are many technical and pedagogical decision points along the way, and they can help guide you through the process.

Outcomes and the Canvas academic hierarchy

Canvas outcomes make use of the academic hierarchy of accounts in Canvas, a branching organizational structure representing the university, its campuses, and their respective schools, colleges, and departments (see figure below).

  • Level 1: At the top of the hierarchy is Indiana University.
  • Level 2: Reserved for the nine IU campuses: BL, CO, EA, FT, IN, KO, NW, SB, and SE.
  • Level 3: Beneath each campus are the accounts representing the schools and colleges on the campus. These accounts begin with the two-letter campus code, followed by the SIS academic group code for the unit, and ending with a three letter "GRP" suffix (for example, BL-ADMIN-GRP, CO-BUS-GRP).
  • Level 4: These accounts correspond to the academic subject codes used by the courses offered through the parent school or college. Level 4 accounts begin with the two-letter campus abbreviation followed by the academic subject code (for example, BL-AERO, CO-EDUC). SIS courses are provisioned into the level 4 account that matches the campus and academic subject code in their course ID. For example, a course with the ID SP19-CO-EDUC-E201-18797 would be placed in the CO-EDUC account.
Tree diagram depicting sample units in the IU academic hierarchy

As soon as outcomes are created in a specific account within the hierarchy, they become available to any account or course that is a child or descendent of the account in which the outcomes were published. Outcomes published at the campus level, for example, can be incorporated into rubrics in any course offered on the campus, and the results can be aggregated at the department, school/college, and campus level. School or college-level outcomes, on the other hand, are only available to the courses and departments belonging to that school or college.

When creating outcomes, it's always best to place them in the account at the highest level to which they apply. In other words, campus-level outcomes should be created in the corresponding campus account, school and college-level outcomes should live in the associated school/college account, and so on. This approach will ensure that student performance results can be aggregated and reported upon by the academic units responsible for those outcomes at all relevant levels of the academic hierarchy.

Plan your outcomes assessment implementation

Before you begin to create outcomes for your unit, it's important to have a clear plan for how the outcomes will be organized, described, scored, promoted, and used to achieve the assessment goals of your unit as well as the academic unit(s) under your purview. This planning process often engages an assessment coordinator, assessment committee, or curriculum committee in consultation with the academic leadership of the unit. Topics and questions your planning group will need to consider are covered in the Canvas Outcomes Planning Worksheet. Use either of the links below to create a copy for your assessment team:

If you use the above worksheet to guide discussion and document your decisions, you'll have everything you need to start building your assessment framework in Canvas.

Embedded assessment vs. a centrally managed approach

The Canvas Outcomes feature was designed on an embedded assessment model. In this model, outcomes are published by a campus, school, or department. Faculty then incorporate them as criteria in rubrics, which are used to grade assignments in their classes. A key benefit is that no extra work is required on the part of the student or the instructional staff. Outcomes are assessed as part of the normal grading process. Under this distributed model, however, the academic unit has very little control in terms of when and how outcomes are used, the rigor with which they are scored, the nature of the student work being evaluated, etc.

School and programs subject to specialized disciplinary accreditation often want greater control over the assessment process in order to meet the requirements of their accrediting body. Implementing a centrally managed assessment process requires considerably more planning and oversight and typically increases the workload of individuals responsible for evaluating student work. To decide which approach is the best fit for your unit's needs, use your completed worksheet and the lists of characteristics below.

  • Embedded assessment:
    • Instructors assess their own students in their own courses.
    • Instructors determine which artifacts and outcomes are assessed.
    • Assessment takes place in regular course sites as part of the regular assignment submission and grading process.
  • Centrally managed assessment:
    • Assessment coordinator or committee determines who evaluates student work.
    • Assessment coordinator or committee determines which artifacts and outcomes are assessed.
    • All assessment takes place in a dedicated assessment site.
    • In addition to outcomes assessment, can be used to track student GPA, standardized test scores, completion of program requirement or milestones, etc.
    • Blind evaluation (student identity hidden) of student work can be enforced.
    • Two or more evaluators per student submission can be enforced.
    • Assessment coordinator or committee can easily track individual and aggregate student performance over time.
    • Students can easily review and be asked to reflect on their own intellectual development over time.
    • Multi-campus collaborative programs can be accommodated.
    • Aggregate assessment data is available in Canvas Gradebook and Learning Mastery Gradebook.

Before you begin implementation

Request Canvas admin rights

Special permissions are needed in order to create outcomes and rubrics, as well as generate reports at the academic unit level in Canvas. IU has created the Outcomes Manager administrative role for assessment coordinators or others who will be responsible for overseeing the unit's outcomes implementation. See Administrative access to Canvas at IU for information about and instructions on how to request access as an Outcomes Manager.

Review plans with colleagues and academic leadership

Once outcomes or rubrics are published at the account level, they are immediately visible and available for use by instructors in your unit. After they have been used to assess student work, Canvas outcomes are difficult to change and they cannot be deleted. Therefore, it's important to get them right from the start. Be sure to share your plans with colleagues and seek the approval of the academic leadership for your unit before publishing anything in Canvas.

Create account-level outcomes and rubrics

Create outcomes at the account level

General instructions on how to create and manage outcomes and outcome groups at the account level are available in the Outcomes section of the Canvas Admin Guide. The most labor-intensive aspect of creating outcomes is customizing the rating scale, mastery level, and mastery calculation method for each outcome. You can do this more quickly in Microsoft Excel; see Use Excel to create or modify outcomes in Canvas.

Create rubrics at the account level

Like outcomes, rubrics published at the account level can be used in any course that is a child or descendent of the account in which the rubrics were published. Providing instructors with pre-defined rubrics containing the outcomes for your unit can save time on importing outcomes and creating rubrics.

To create an account-level rubric:

  1. Log into Canvas and navigate to the administrative account containing your outcomes set.
  2. In the account navigation menu, click Rubrics.
  3. Create a new rubric and give it a meaningful name (such as "Undergraduate Spanish Outcomes").
  4. Delete the default criterion added to the rubric.
  5. Use the Find Outcome option to select and add the outcomes from the account-level set that you wish to add to the rubric. After selecting each outcome, be sure to uncheck the option Use this criterion for scoring before clicking Import.
  6. When you are finished adding outcomes, click Create Rubric to save the rubric.
Note:

When an instructor attaches the rubric to an assignment in a course:

  • The outcomes referenced in the rubric will automatically be imported into the course.
  • The instructor can delete criteria from the rubric that are not applicable to the assignment.
  • The instructor can add assignment-specific criteria to the rubric.

Facilitate embedded assessment activities

Once you've created your outcomes and rubrics, no additional setup in Canvas is necessary as long as your unit plans to use embedded assessment as the principal method of data collection. However, it's not enough to simply publish these assets in Canvas. If you want instructors to take advantage of them, someone (ideally your department head or assessment coordinator) needs to communicate clear expectations to faculty about how to locate them and when, where, and how they should be used. Otherwise, instructors who do happen to find them may use them in ways that aren't consistent with your assessment goals. You may also want to develop documentation for your faculty with instructions on how to find and make the best use of the outcomes and rubrics published for your unit.

Below is a list of Canvas Guides that can be shared with instructors who will be using account-level outcomes or rubrics to assess student work. You may also want to develop some custom documentation that provides specifics on how to locate and import the outcomes and rubrics for your unit.

Implement a centrally managed assessment process

Academic units that want more control over the assessment process can conduct assessment activities in one or more Canvas sites designed for and dedicated to this purpose. This approach requires more time and effort on the part of students, evaluators, and program managers, but allows the unit to regulate the types of student evidence collected as well as how evidence is evaluated, and by whom. Another advantage is enhanced tracking and reporting afforded by both the Learning Mastery Gradebook as well as the standard Canvas Gradebook.

Set up a Canvas site for assessment

Many of the tools and features used for teaching in Canvas can be used for outcomes or program assessment. Unlike Canvas course sites, however, which are typically active for one term, your assessment site can remain open indefinitely, allowing you to collect and analyze assessment data over time. Since assessment requirements and practices vary widely from one discipline to the next, UITS recommends working closely with a consultant from your teaching center to design an assessment framework that best meets your needs. In general, to set up a Canvas site for assessment:

  1. Create a new Canvas site: This site will serve as the center of all assessment activity for your program. Set the expiration date for the site far out into the future (for example, January 1, 2030). For instructions, see Create a new Canvas course site or change the expiration date for a course you created.
  2. Enable the Learning Mastery Gradebook feature option(s): The Learning Mastery Gradebook provides an overview of how the entire class and individual students are performing in relation to learning outcomes. This feature is disabled by default. To turn it on in your assessment site:
    1. Open the Settings tool in the site, and click Feature Options.
    2. On the "Feature Options" page, toggle the "Learning Mastery Gradebook" option from Off to On. This will enable the instructor view of the gradebook.
    3. If you would like students to be able to track their own performance, toggle "Student Learning Mastery Gradebook" to On as well.
  3. Remove unused tools from the course navigation: In a dedicated assessment site, students will interact with a small subset of the tools typically available in Canvas course sites, and may only need access to the Modules tool. To keep students focused on the tasks required of them in the site, disable the tools they don't need. (Disabled tools are still visible to instructors in the course navigation.) For instructions, see How do I manage Course Navigation links?

    For the course navigation in Canvas assessment sites, UITS recommends some or all of the following:

    • Home: Use the home page to provide a description of the purpose of the site and links to appropriate documentation if needed.
    • Announcements You can use Announcements to communicate to site members, if desired.
    • Modules: Use Modules in place of Assignments and Quizzes so that students will have one place and tool to view and participate in assessment activities.
    • Grades: This provides access to both the standard and the Learning Mastery Gradebook.
    • Outcomes: You can allow students to view the outcomes and their descriptions in the Outcomes tool, if desired; outcomes may also be visible in assignment rubrics and the Learning Mastery Gradebooks.
  4. Add your account-level outcomes: In order to use your account-level outcomes to assess student work, you'll first need to add them to the course site. For instructions, see How do I find an existing outcome to add to a course?
  5. Decide what types of evidence you want to collect: With your assessment committee, determine what types of artifacts you want to collect at what points in the program. Some programs are very prescriptive about the work that students submit (for example, signature assessments A,B, and C from courses X, Y, and Z), while others want students to make the selections in response to prompts that lay out the requirements.
  6. Create a Canvas assignment, quiz, or survey for each submission or piece of evidence: Begin by selecting the Canvas tool and submission type best suited to the type of evidence that you intend to collect. Common examples are provided in the table below.
    Evidence Canvas tool Submission type
    Representative artifacts (assignments, projects, etc.) uploaded by the student
    Assignments Online > File Upload
    Reflection Assignments
    Online > File Upload or Text Entry
    Web site or web resource created by the student
    Assignments
    Web Site URL
    (Note that web content can be changed after assignment submission or evaluation.)
    Video files uploaded directly to Canvas
    Assignments
    Online > Media Recording
    (Maximum file size 500 MB)
    Student videos stored in Kaltura
    Assignments Text Entry. See Create a video assignment in Canvas with Kaltura.
    Observations of student behavior (for example, student teaching observations)
    Assignments
    No submission. Evaluate with grade, rubric, and/or comments.
    GPA, exams, standardized test scores
    Assignments
    No submission
    (Can be hand entered or imported into the gradebook)
    Survey or self-assessment Quizzes
    Graded or ungraded survey
    (With graded surveys, you can track submissions in the Canvas Gradebook)
    Knowledge or skills assessment Quizzes Graded quiz

    As you set up each assignment, quiz, or survey for evidence collection and evaluation, also consider:

    • Assignment/quiz/survey instructions: Provide students with detailed directions for what they are required to submit for the assignment and when it is due (for example, at the end of the sophomore year, after completing a certain course, etc.), as well as contact information in case they have questions. Also, if students are expected to submit work products created in specific classes, ask the instructors of those classes to remind students to make the submission.
    • Rubrics: For evidence that will involve scoring students on learning outcomes, create or attach a rubric that incorporates your account-level outcomes as criteria. The rubric can also include criteria that are not outcomes, but the scores for those criteria will not be available in the student mastery gradebook or other reports. For information on how to work with rubrics, see:
    • Blind evaluation: If you don't want evaluators to know the identity of the person they're evaluating, enable the anonymous grading feature in the assignment as described in How do I add an assignment that includes anonymous grading? In the assignment instructions, ask students to remove their names and other identifying information from their work before they submit it in the assignment. This feature is not available for quizzes.
    • Multiple evaluators per submission: If your program requires two or more evaluators per submission, use the moderated grading feature in Canvas to set the number of graders and designate the person responsible for reconciling differences and posting the final scores. For instructions, see How do I add a moderated assignment to be graded by multiple reviewers? This feature is not available for quizzes.
    • Grading setup for assignments: The grading options in the assignment can be helpful for tracking how successful students are in meeting all of the outcomes associated with the task in the standard gradebook. You can either let the points in the rubric determine the total points for the assignment or ask evaluators to manually enter the score.

      To display performance descriptors in the gradebook (meets expectations, does not meet expectations, etc.) instead of point values, create a custom grading scheme in the course; see How do I add a grading scheme in a course? Then set the Display Grade as option in the assignment to Letter Grade and choose the custom scheme. Finally, since students won't be receiving a final grade in the assessment site, it's a good idea to check the box Do not count this assignment towards the final grade.

    • Restricting or granting access to an assignment: For requirements that only apply to a subset of the students in the site, you can use differentiation to restrict or grant access to specific individuals or sections. See:
    • Hiding evaluation results from students: If you don't want students to be able to see the evaluation results for all or specific assignments, mute them. See How do I use posting policies in a course?
  7. Organize assignments, quizzes, and surveys into Modules: Use the Canvas Modules tool to group the assignments in your site in a way that makes sense to both students and evaluators. Among the more common schemes for grouping assessment tasks evidence are:
    • Program year: (Year One, Year Two, etc.) Used to organize submissions chronologically, making it easier to monitor progress over time
    • Learning outcome: (Critical Thinking, Written Communication, Team Work, etc.) Used when students are expected to submit representative work for each outcome at specific points in their program; students may be required to upload specific work products or make their own selections
    • Decision point: (Decision Point 1, Decision Point 2, etc.) Commonly used in education programs. Performance in a given decision point module determines whether students are ready to advance to the next stage of the program.
    • Course: Used when students are expected to submit specific artifacts or representative work from specific courses in the program

    If desired, module prerequisites and requirements can be added to control how students progress through the modules.

  8. Publish all assignments, quizzes, surveys, and modules: Before you start adding students, publish all of the elements in your assessment framework so they are available to students.
  9. Create course sections: Canvas sections can be used to group students or students and evaluators for a variety of purposes, including:
    • Reporting: Grouping students by cohort, program year, concentration, etc. facilitates filtering by the characteristics in the standard and Learning Mastery Gradebook, as well as in unit level reports.
    • Assigning evaluators: To distribute the evaluation workload, evaluators can be assigned to specific students by creating a section for each evaluator and adding their students to it. To prevent evaluators from seeing and assessing students in their group, limit their section privileges as described in How do I limit a user to only interact with other users in the same course section? Evaluators who are permitted to see all students can use section filtering in SpeedGrader and the Gradebook to narrow the list to their assigned students.
    • Assignment differentiation: Student groups can also be used for assignment differentiation. For example, if students are grouped by entering cohort, the availability and due dates for assignment can be customized to reflect when students in each cohort are expected to complete the assignment.

    When setting up sections in your assessment site, keep in mind that any member of the course can belong to multiple sections. For instructions on how to create ad hoc sections, see How do I add a section to a course as an instructor?

  10. Communicate expectations to students, faculty, and evaluators: Before you add users to your assessment site and at regular intervals thereafter, communicate with each audience (students, faculty who will be facilitating student submissions, evaluators, etc.) to let them know the purpose of the site and their role in supporting your unit's assessment strategy. Ideally, evaluators and faculty facilitators will already be aware of your assessment plans and have a clear understanding of what they need to do to ensure its success. Communications to students should explain when and how they are expected to interact with the assessment site. And of course, reminders are needed at specific points in your program. Student reminders can come from you or from faculty who have been tasked with facilitating the submission of specific artifacts produced in your curriculum.
  11. Add the students and evaluators to the assessment site: You'll need to maintain the roster for your assessment site manually. This will involve adding new students each term and adding/removing faculty evaluators as needed. Leave students who have graduated or left the university in the site if you want them to be included in reports as well as the standard and Learning Mastery gradebooks.
    • To add or move an individual user to a specific section, locate the user in the People tool, open the Options menu at the right, and choose Edit Sections.
    • To add users to a section in bulk, in the People tool, click +People. Enter or paste the list of email addresses or usernames into the text box, and then select the role and section for the group. This method can be used to add new users to a specific section in the site, or to add existing users to another section.

    See the People section of the Canvas Instructor Guide for instructions on other roster management features.

Tools for tracking and reporting

Tools for tracking and reporting on learning outcomes are available at the Canvas account level as well as in individual Canvas sites. Descriptions of these tools and links to documentation are provided below:

Learning Mastery Gradebook (availability: course only)

The Learning Mastery Gradebook provides summary and detailed student performance data for the outcomes used in the course. Using the mastery calculation method defined for each outcome, the gradebook assigns a proficiency rating to each student using a customizable color-coded scale. As with the standard Canvas Gradebook, instructors have access to a grid view (which can be filtered by section) listing all students in the course as well as an individual student view. For more, see:

Learning Mastery gradebook, grid view
Grid view

Learning Mastery Gradebook, individual student view
Individual student view

Canvas Gradebook (availability: course only)

For centrally managed assessment processes, the Canvas Gradebook shows the grade, score, or proficiency level earned by each student on each assignment or quiz in the assessment framework. The value in the gradebook represents the students' overall performance on each assessment task, whereas the Learning Mastery Gradebook gives performance information on individual outcomes. The gradebook can also be used to track evaluation progress since it indicated which assignments have been graded and which have not. For more, see the Grades section of the Canvas Instructor Guide.

Canvas Gradebook with custom grading scales for proficiency levels, requirements, etc.

Outcomes Report for an individual student (availability: course and account)

The Outcomes Report lists the outcomes that apply to the selected student along with the aligned artifacts, number of attempts, latest score, and average percentage score. Available at both the course and the account level, this report can be accessed by finding or selecting the user with the Canvas People tool and navigating to the User Details page. For more, see:

Outcomes results for individual student

Outcome Export report (availability: account only)

The Outcomes Export is a CSV file that lists the outcomes and outcomes groups in the account where the report is run. This report conforms to the Outcomes Import Format and is especially useful for editing large outcomes sets in Microsoft Excel and importing the changes back into Canvas.

View a sample Outcomes Export report.

To generate the report:

  1. Navigate to your unit's account in Canvas.
  2. From the navigation menu, select Settings and then Reports to open the "Reports" screen.
  3. Locate the report named "Outcomes Export" and click Run Report.
  4. It may take some time for the report to be generated. Return to the "Reports" screen after a few minutes and look for a download link. Click the link to view and/or download the report.

For more, see How do I view reports for an account?

Outcome Results/Student Competency reports (availability: account only)

The Outcomes Results and Student Competency reports aggregate raw outcomes results data across all courses that use the outcomes published in the account. Whenever a student is assessed on an account-level outcome, a row with relevant information about the student, course, assignment, and outcome is added to the report. In addition to providing insights into student performance, the data in these reports can be used to generate a curriculum map showing which outcomes are emphasized and assessed in the courses that make up your program. Both the Outcomes Results and Student Competency reports are in CSV format and can be easily imported into Excel or statistical software for analysis.

The data elements available in each report are detailed below:

Data elements Outcomes Results Student Competency
student id Yes Yes
student sis id Yes Yes
assessment title Yes Yes
assessment id Yes Yes
assessment type Yes No
submission date Yes Yes
submission score Yes Yes
learning outcome name Yes Yes
learning outcome id Yes Yes
attempt Yes Yes
outcome score Yes Yes
assessment question Yes No
assessment question id Yes No
course name Yes Yes
course id Yes Yes
course sis id Yes Yes
section name Yes Yes
section id Yes Yes
section sis id Yes Yes
assignment url Yes Yes
learning outcome friendly name Yes No
learning outcome points possible
Yes No
learning outcome mastery score Yes No
learning outcome mastered Yes No
learning outcome rating Yes No
learning outcome rating points Yes No
account id Yes No
account name Yes No
enrollment state Yes No

To generate either of these reports:

  1. Navigate to your unit's account in Canvas.
  2. From the navigation menu, select Settings and then Reports to open the "Reports" screen.
  3. Locate the report named "Outcomes Results" or "Student Competency", and click Configure.
  4. Select the desired term(s) and sort order, and then click Run Report.
  5. It may take some time for the report to be generated. Return to the "Reports" screen after a few minutes and look for a download link. Click the link to view or download the report.

For more, see How do I view reports for an account?

This is document awaw in the Knowledge Base.
Last modified on 2022-08-05 08:28:45.