TIM Tools 7.5 Administration Guide

Interpreting TIM-O Report Data

The TIM-O is a flexible tool that supports gathering data for a variety of different purposes. Each purpose may require a unique set of data collection and analysis procedures. As such, interpretation of the data in a TIM-O report depends upon the purposes for data collection, the procedures employed, and the conditions under which those procedures were undertaken.

When applied to the data you have collected, examining each of the concepts included in a report — Extensity, Flexibility, Characteristic Dominance, Highest Levels Observed, and Level Prevalence — can help you identify the professional development needs of your teachers. Different concepts may be more useful to you depending on the kinds of decisions you are trying to make. Each concept illuminates a different aspect of the pedagogical approaches used in the observed lessons. Together, they create a more complete view of the ways technology was used in these classrooms. We encourage you to explore the data in reports and compare the different charts, figures, and tables to develop a deeper understanding of the strengths and needs of your teachers.

The following sections describe general principles for understanding and interpreting the data in a report based on the following assumptions:

• Observers received training to conduct classroom observations.
• Observers applied the same criteria consistently across all observations.
• Observations were conducted within the same time period.

An effective use of TIM-O data is to identify a teacher’s current practices to help support that teacher’s professional development. For example, if a teacher is currently only using technology at the Entry level, he or she would be better supported by professional development aimed at Adoption level practices than at Transformation level practices. Classroom observations provide the best evidence of the TIM levels a teacher currently uses and therefore the levels of professional development that would be most beneficial.

Generalization is a concept that refers to drawing broader conclusions based on a particular set of observations. For example, unless you observe every lesson from a particular teacher, you are drawing conclusions about what is typical in that classroom based only on the results of the lessons that were observed. Similarly, you may observe only a sample of the teachers in the school or district and want to make decisions and draw conclusions for all of the teachers in the school or district. The degree to which these generalizations are supported depends on the rigor involved in collecting the data and choosing which data points (teachers and lessons) to include. For more information about collecting data to support generalizations, please see TIM Tools course offerings at iteach.usf.edu.

In the absence of the rigorous conditions necessary to support generalization, very useful evidence to support professional development decisions can still be found in your TIM-O data. Each observation describes a lesson using concepts from the TIM and can be used in coaching and professional development for the teacher of the observed lesson. Each observation provides evidence of conditions and practices that may be present in other classrooms and points the way toward further exploration that may clarify professional development needs of other teachers.

 

Extensity

What is Extensity?

The extensity value is the highest TIM level for a given teacher, across all observations and characteristics. For most teachers, the level of difficulty increases as one moves toward the right on the TIM, i.e., it’s easier to do an Entry level lesson than an Adoption level, and so forth. Therefore, we assume a teacher can implement lessons at every TIM level up to his or her Extensity level. Note that the percentages in the extensity table add up to 100%, meaning each teacher is represented in one and only one cell.

While extensity is the highest level at which that teacher has been observed, it’s important to remember that it may not be the highest level lesson that teacher has done or can do. The greater the number of observations of a given teacher, the greater the confidence that the extensity value truly represents the teacher’s ability.

How do I use this information?

The extensity distribution can be used to plan professional development by matching the instructional strategies of focus to the next highest TIM level. For example, for teachers at the Adoption extensity value, professional development should focus on instructional strategies at the Adaptation level, i.e. providing students with opportunities to choose from among appropriate options for technology use, allowing students time within lessons for exploration of the available technology tools.

Another use for the extensity distribution is to measure overall progress toward school-wide or district-wide goals. For example, if your goal was for every teacher to demonstrate a lesson at the Adaptation level or above during the observation period, the extensity distribution would tell you how many teachers met this goal. If you wanted to know the highest level the majority of your teachers reached, the extensity distribution would provide the answer.

An effective teacher will choose to incorporate lessons at all TIM levels, depending on student needs and curriculum demands. Extensity data can help you identify the upper limits of the pedagogical range teachers are using when teaching with technology, so that you can plan professional development to extend that upper range.

 

Flexibility

What is Flexibility?

In this context, flexibility refers to a teacher’s ability to use each TIM level within each characteristic. The TIM levels represent a range of different pedagogical strategies and the needs of students may call for the use of any of these levels within any of the characteristics of meaningful learning. Ideally, a teacher would be able to use all levels within all characteristics. The flexibility distribution provides evidence of the range of different pedagogical strategies used by teachers within the sample of lessons observed. Keep in mind that just because you haven’t observed a teacher using a lesson at a given level, the teacher still may have used those strategies in other lessons that were not observed. Note that the percentage in every cell of the extensity table can go as high as 100%. If all cells were at 100%, it would mean that every teacher in the sample was observed at each of the five levels.

How do I use this information?

Flexibility describes the variety of strategies evidenced in the observation data. While extensity data identifies the upper limit of the range of strategies used, flexibility data can help you identify gaps in the pedagogical strategies in use. In general, we might expect the percentages in the flexibility distribution to decline as we move toward higher levels of technology integration. For example, we might expect a Flexibility distribution to show that 100% of teachers use Entry level strategies, 70% of teachers use Adoption, 60% of teachers use Adaptation, 20% use Infusion, and 10% use Transformation, with the percentage at each level lower than the percentage at the previous level. If however, the Flexibility distribution showed a dip — a percentage that was lower than the cell to the right — that might indicate an opportunity for professional development. If Adoption strategies are underused compared with other strategies, teachers may need guidance about the circumstances in which Adoption strategies are most effective. Flexibility data helps us identify gaps in teachers’ understanding of application of lower TIM levels. Likewise, differences between the distribution for different characteristics may inform professional development.

 

Level Prevalence

What is Level Prevalence?

Level prevalence highlights the most frequently observed TIM levels within each characteristic for the selected observations. Level prevalence is calculated at the observation level. This contrasts with the Teacher Flexibility distribution, which organizes similar information by teacher rather than by observation. Level prevalence identifies the most frequently occurring level for each characteristic.

How do I use this information?

In combination with teacher extensity and flexibility, level prevalence helps describe typical classroom use of technology within the observed lessons. Level prevalence helps identify productive areas for professional development.

 

Highest Levels Observed

What information is included in the Highest Levels Observed chart?

Each observation includes up to five TIM levels, one for each characteristic. The highest level from each observation in the dataset is represented in the Highest Levels Observed chart. The Highest Levels Observed chart provides information about how common it is for each level to appear as the highest level on an observation. While Extensity groups observations by teacher, Highest Levels Observed includes each observation as a separate data point.

How do I use this information?

Because it provides a distribution of highest levels, this chart combines information about the range of observations with information about the typicality of observed levels. In combination with other charts, highest levels observed can inform your understanding of typical classroom technology use in your classrooms.

 

Characteristic Dominance

What is Characteristic Dominance?

Characteristic dominance refers to the characteristics of meaningful learning that stand out as having the highest and lowest ratings across the set of observations. A characteristic can be classified as Dominant if it received the highest TIM level ratings. Conversely, a characteristic can be classified as Least Dominant if it received the lowest TIM level ratings. It is possible for more than one characteristic to be rated at the highest or lowest level, and thus share the dominant or least dominant status. Characteristic dominance is calculated at the observation level, and is interpreted as the percentage of observations for which the characteristic was rated as the highest (for dominant) or lowest (for least dominant). This measure gives a sense of which characteristics tend to reach the highest and lowest TIM levels in the particular set of data.

How do I use this information?

Describing characteristics in terms of dominance provides a view of where professional development might be most beneficial for the teachers in the dataset. Looking at the dominant characteristic gives a view of where teachers are already strong. Looking at the least dominant characteristic gives insight into where teachers need the most training and support. For example, if Collaborative is found to be the dominant characteristic (i.e., the one most often rated the highest) then teachers have demonstrated that they can give lessons at higher levels related to the indicators for collaborative learning, and therefore professional development may be better focused on helping them reach higher levels on other characteristics. If the least dominant characteristic is Goal-Directed, this indicates that the concepts and strategies involved in goal-directed learning are those for which these teachers might most benefit from professional development and support in order to reach higher TIM levels.

 

Comments

What are comments?

You can collect valuable information in the comments field of an observation.

How do I use this information?

A qualitative analysis can be conducted on these comments that can show patterns and themes. It is also possible that your observation protocol instructs the observer to include a code in the comments field that will identify a particular type of observation, a particular subject focus, or related to a particular grant.