About Terracotta at IU

• Overview
• Use Terracotta
• Manage experiments in Terracotta
• Learn more and get help

Overview

Terracotta (Tool for Educational Research with RAndomized COnTrolled TriAls) is a platform designed to lower the barriers to conducting rigorous and responsible experimental research on teaching and learning. Terracotta is an external tool that integrates with the Canvas learning management system, making it possible to assign different versions of Canvas activities randomly to different students, while also protecting these students' rights and privacy as research participants.

Terracotta automates the informed consent process, which allows students to opt into or out of the research study; it also assigns participants to different experimental treatments at random. The platform's interactive assignment builder helps teachers create different versions of learning activities that correspond to experimental treatment conditions, and then maps scores from the LMS gradebook onto those treatment conditions, providing authentic outcomes for research studies. Terracotta also exports de-identified data, and excludes non-participants automatically.

Terracotta is developed and managed by the eLearning Research and Practice Lab at Indiana University, part of the Pervasive Technology Institute, with support from Institute of Education Sciences (IES) grant R305N210035.

Research considerations

Conducting research within one's own class is important for evidence-based teaching and professional development, but it should be done carefully, ethically, and with sensitivity to students' rights and privacy. Before deciding to conduct this research, teachers may wish to discuss plans with departmental leadership, contact a campus teaching and learning center, or participate in the Teaching for Student Success course to become familiar with evidence-based approaches to teaching and learning.

Terracotta is a tool for conducting experimental field research in authentic education settings. Experiments provide compelling evidence for causal relationships, showing that an instructional practice had a causal effect on student learning. However, there are other ways to conduct this kind of research, or to build evidence of causal relationships. By making Terracotta available, IU is providing helpful support for experimental field research, but this should not be misinterpreted as a suggestion that experimental research is a privileged method of conducting research on teaching and learning.

Use Terracotta

When conducting an experiment within Terracotta, a teacher differentiates Canvas activities for different students. In doing so, some students will receive versions of assignments and learning resources that differ from their peers' assignments. To minimize concerns about fairness, consider the following recommendations:

1. Use "Informed Consent" in Terracotta so that only students who volunteer to participate will receive the experimental manipulation.
2. Use the "All Conditions" design option in Terracotta so that all students receive all versions of manipulated assignments, but in different orders.
3. Ensure that all experimental versions reflect pedagogically sound instructional strategies. Do not expose students to variations that are known to be less effective.
4. Do not manipulate high-stakes assessments or pivotally important learning experiences.
5. Provide additional opportunities for students to learn the material outside the scope of the experiment.
6. Emphasize to students that you are available to answer any questions, and that your availability and your responses will not be manipulated as part of your research.

Terracotta walks a user through a sequence of design decisions, such as determining the number of treatment conditions, the experiment design, the informed consent procedures, and so on. The experiment builder has three sections: design, participation, and assignments.

Design

The "Design" section walks researchers through the basics of experiment design. Researchers give experiments a title, describe the experiment (this includes recording a research question and a hypothesis), name conditions to label experimental versions of assignments, decide how students will be exposed to these conditions (either students will receive all conditions or only one condition), and select a default condition (which is what students will experience if they do not consent to participate, or if they join the class after the experiment has already commenced).

Participation

In this section, researchers determine how students will become participants in the experiment. Researchers can invite students to consent to participate, decide who will participate and enroll them manually, or include all students automatically.

Assignments

In the "Assignments" section, researchers create different versions of their class assignments according to their experiment conditions. Terracotta will populate Canvas assignments with learning activities and materials that change depending on who's looking at them, automatically managing experimental variation within the buckets. Once the assignments are published within Canvas, Terracotta will determine which students see which versions of the assignments.

Manage experiments in Terracotta

Once the instructor publishes the assignments and the experiment is in the field, they can determine how they want to use the data Terracotta collects. Assignment data, including "Informed Consent" assignments, are de-identified. If, for instance, researchers are interested in how experimental treatments influence outcomes such as performance on future assignments, they could "Add Outcome" by entering a score manually, or by selecting existing gradebook items as relevant outcomes.

At any time, researchers can export experiment data as a ZIP file that includes several CSV files describing the items added to an experiment, how participants responded to those items, which treatment conditions participants received, and how these items related to outcomes.

Learn more and get help

For more information and updates about ongoing development, visit https://terracotta.education.