We recently updated our our published research page to include several ways we are seeing the Technology Integration Matrix (TIM) used in research studies. In this post I would like to give a little more information about each of these approaches, as well as some best practices that can help make your research and evaluation efforts successful.
Research and Evaluation Approaches
In general, the TIM describes dimensions that have been identified as important components of good pedagogy. The TIM levels generally describe concepts such as a gradual release of ownership of learning from the teacher to the student, a shift in focus from the technology to the content, and the journey toward higher-order thinking. The characteristics that make up the TIM are informative lenses for what is happening in a classroom, and there are bodies of research behind each of the characteristics (see the section on the Research page titled “Research on Concepts Central to the TIM” for selected sources related to each TIM characteristic). Research questions that focus on any of the TIM characteristics or levels of integration can serve to inform your understanding of the teaching and learning taking place in a classroom. In contrast to some other models of technology integration that focus on specific tools or tasks, the TIM can help you reveal the answer to whether or not a research effort or program it is positively impacting teaching and learning.
Here are a few ways that the TIM and TIM Tools can be used in your research and evaluation studies, along with an example for each general approach:
TIM as a theoretical framework for a study
The TIM could serve as an overall frame for a study to evaluate a one-to-one program initiative. Professional development activities could focus on different TIM characteristics. A variety of data collection methods might be used over the period of an academic year, with the interpretation of results related back to the TIM.
TIM as a coding scheme for analyzing data
In a research study focusing on a giving teachers strategies for active learning, a series of lesson plans might be collected from each teacher over the semester. These lesson plans could be reviewed with an eye on what the teachers and students would be doing during each lesson. The lesson plans could then be coded as to which levels of each TIM characteristic match the activities described, with particular interpretive emphasis on the active characteristic.
TIM-O as a tool for collecting classroom observation data
Within a grant project focused on providing training and resources to support teachers in more effectively integrating technology, evaluators might conduct multiple observations in each participating classroom over the study period, using the analysis provided in the TIM-O reporting tool to look for patterns and changes.
TUPS as a tool for collecting data about teacher perceptions and use of technology
In a research study in which different types of technology-related training are provided to matched groups of teachers, the TUPS could be administered to all participants before the training and again after the training to look for differences across the treatment and control groups.
ARTI as a framework and tool for conducting and recording an action research project
The ARTI provides the opportunity for teachers to design their own classroom-based research around a particular program, technology, or teaching approach being evaluated. This could serve as a practitioner-based extension of an evaluation study that would give end-user insight into a school or district-wide evaluation.
TIM-C as a tool for documenting coaching and mentoring activities
In evaluating a new coaching program within a district, the TIM-C could be a valuable tool for collecting and reporting data related to goals, activities, client/coach interactions, and end-of-cycle reflections from the client and coach.
Selected studies that have used the TIM and/or TIM Tools can be found on the TIM website Research page. In addition to giving you ideas as you design your study, these resources may be helpful as you conduct your literature review.
Whether used for research or program evaluation, here are a few things to keep in mind to contribute to the success and value of your work.
Using TIM Tools to Collect Data
One of the most important things to keep in mind when using any of the TIM Tools is to make sure that you are using the tool in the way it was designed. The tools were carefully created to help in the collection and use of data for specific purposes. For your research and evaluation studies to yield meaningful and helpful results, it is important to stick to the recommended uses. For example, the Technology Uses and Perceptions Survey (TUPS) is made up of different sections that are designed to provide helpful information about that particular aspect of an educator’s perceptions of technology. If in your analysis you assign numerical values to responses, these are not meant to be combined across sections to create a total, as each section measures a distinct concept.
Similarly, with the TIM Observation Tool we recommend that you separately interpret the results for each characteristic, and not sum across the characteristics. An observation results in a level for each of the five characteristics of the TIM. These characteristics are related but evaluate distinct concepts, and therefore should not be averaged or summed together. To help you interpret results from the TIM-O, we have created a report that provides different analyses that we think give much more meaningful and helpful interpretations (more on these in an upcoming post!).
Consider using more than one of the Tools to strengthen your study. For example, you might use a design that incorporates both the TUPS and the TIM-O or one that uses the TIM-O, TIM-R and TIM-LP to create a more complete picture.
Finally, we recommend that you take the time to create a strong design for your research or evaluation study that has a well-thought-out sampling strategy, and includes multiple observations if using the TIM-O.
Providing Descriptive Information and References for the TIM and TIM Tools
Do I need to cite the TIM? How do I cite the TIM? What source(s) do I reference? We get it! There are a lot of materials available on our website and many other sites that give mention to the TIM. In general, the most helpful thing you can do for your fellow researchers is to provide a reference to the the Matrix itself, as well as any additional resources you have used.
For example, if you mention the Technology Integration Matrix or use it as a substantive part of your study, it is helpful to provide an in-text citation as well as a listing in your reference list. In a typical paper following the APA format, this might look like:
The Technology Integration Matrix (TIM) served as a framework for our study (Florida Center for Instructional Technology, 2019).
In the reference list for the paper, you would then include this general reference:
Florida Center for Instructional Technology. (2019, June 1). Technology Integration Matrix. Retrieved from https://fcit.usf.edu/matrix/matrix/
If you use one of the TIM Tools in your study, you would end up citing three different things: the TIM itself, whatever source you used when paraphrasing your description of and background on the TIM, and the tool you used. For example, if you used the TIM-O you would include this reference in your list:
Florida Center for Instructional Technology. (2019, June 1). The Technology Integration Matrix Lesson Observation Tool. Retrieved from https://fcit.usf.edu/matrix/evaluation-tools/tim-o/
We have provided some samples and suggested references for you on the TIM for Research Page, as well as references formatted in styles other than APA.
If you’re wondering what resource will best help you get a deeper understanding of the foundational concepts and development of the TIM, we recommend:
Harmes, J. C., Welsh, J. L., & Winkelman, R. J. (2016). A framework for defining and evaluating technology integration in the instruction of real-world skills. In S. Ferrara, Y. Rosen, & M. Tager (Eds.), Handbook of research on technology tools for real-world skill development (pp. 137-162). Hershey, PA: IGI Global. Electronic version of the chapter available from publisher.
If you have any questions, or are using the TIM in your research or evaluation efforts, please let us know! We enjoy hearing about all the ways TIM and TIM Tools are being used in the field. You can email us at: TIM@fcit.us
Christine Harmes is a consultant on research, measurement, and evaluation, and an ICF-certified coach. Her research interests focus on improving teacher use of technology, computer-based testing and usability. At the Florida Center for Instructional Technology at the University of South Florida, Dr. Harmes focuses on research and tool development related to technology integration.
