TEACHER EVALUATION OF GTV PROTOTYPE

March 25, 1987

by

Donald Holtgrieve, California State University, Hayward

and

David Lawrence, Lucasfilm Ltd.

Introduction

As part of the Apple Computer, National Geographic Society and Lucasfilm collaboration in the uses of computers and optical media, a demonstration of research findings to date was presented to a select group of social studies teachers.

The purpose of the evaluation session conducted on Saturday, 1-24-87, was to assess the initial reaction of teachers to the prototype and to determine if the direction the project is taking is valid in terms of teacher's expressed needs and expectations for new ways to improve the teaching of geography. The evaluation session began with a Pre-Demonstration Questionnaire. This was followed by a demonstration of the GTV prototype. Finally, a variety of opinions about GTV were solicited by means of a written questionnaire followed by an informal discussion.

The responses on the questionnaires and the major ideas offered during the informal discussion were compiled by the evaluators and are presented below.

Respondent Profile

The eleven participating teachers were selected from the mailing list of the Northern California Geographic Alliance, the criteria being that they be from different schools representing middle and high school grade levels. All are social studies teachers and most have other duties as well. Teachers of grades 4 through 12 were represented. The most common grade taught was 9th grade World History-Geography (6). The second most frequently taught grades were 7th grade World History-Geography and 11th grade U.S. History-Geography (4).

One teacher reported previous experience with videodisc, nine with videotape and five with computers. Nine of the eleven teachers reported that they have total control over the selection of A/V materials in their classrooms. One stated that he had some control and one reported no control over selection or purchase. Funding was an important issue for all. Where asked about the best materials they were currently using the eleven teachers gave seventeen responses as follows:

general departmental collections4

computer software3

PBS videotapes3

classroom simulations2

NGS materials3

self generated materials2

In response to the question of what things are difficult to teach students that A/V or computers could help with, eight respondents reported concepts as the primary concern. Other responses were; places or regions (4), facts (2), and data manipulation (1).

In general, the teacher respondents seemed very much in control of what they teach and how they teach it. They appeared to be very interested in possibilities for improving their own teaching situation, but were unwilling to speak for other teachers.

Post-Demonstration Discussion

Following a demonstration of the GTV Prototype and the solicitation of responses via written questionnaire, teachers were given an opportunity to informally discuss their ideas and reactions. Below is a summary of their feedback.

 

Practical Concerns

Based on their experience with A/V and computer resources in the classroom, many teachers expressed concern over the security and reliability of the equipment. Some responses:

"A thing that's scary about looking at a thing like this is a lot of times teachers leave their classroom open...they have to go somewhere and kids like to tinker [with equipment]."

"We have a problem with security and a lot of times we don't get to use the computer equipment because it's locked away so no one can steal it."

"What do you do when it doesn't work correctly Easy, functional troubleshooting so that a teacher can refer to it and it's written in such clear English, not written by a technician but written by an educator in their language, so they understand which pieces to put together to make it work..."

"It's real frustrating, even if you do know a lot about technology, to put everything together. A teacher in the classroom doesn't have a lot of technological savvy."

Concerns such as these suggest a need to make the hardware system simple and tamper-proof. Teachers indicated a preference for a single box that could do everything. There was some apprehension at the number of devices involved in the current plan for GTV. Until there is a one box system, the Apple Cart will be an important package for the xTV components. The Apple Cart should secure the various pieces of hardware so that they're safe from theft and tampering. Its design should emphasize integration so that a person using The Cart is thinking in terms of using The Cart, rather than using components. The system should be as "intelligent" as possible in providing self diagnosis and troubleshooting advice. One teacher commented on being impressed with the way that newer copying machines had icons that lit up to tell where a problem was. The xTV hardware should be at least as helpful in identifying and offering solutions to basic problems such as disconnected cables, open drive doors, wrong or missing discs, etc.

 

 

Using GTV

Flexibility was important to teachers. GTV's modular design and ease of customization were well received. Here are some of the comments:

"Flexibility is the main thing. So many times with the material I'm using I wish I could just use that little part. If there's an introduction that's important or if it's not everything I want, or if it's too much...The flexibility we have here...to pull out three or six minutes [of what I want]...is probably the most valuable thing."

"The segment idea is wonderful because you could put together...various segments for different uses."

While teachers indicated that flexibility was what they wanted, whether they would actually use it was a different matter. One teacher summarized the situation with this observation:

"Hope it's not too cynical, but my view of teachers is that they may say "yeah I'm gonna do all this programming" and "yes I'm gonna arrange this" and so on, but in the real world they're probably more likely just to plug it in and turn the damn thing on and hope it works."

Other teachers similarly remarked that they would probably not be willing to spend much time reprogramming the system. This seems to confirm early GTV design notions that an automatic, turnkey level of operation be the entrance, with more powerful functionality available as desired.

 

GTV in the Classroom

Teachers felt that GTV could work well with large groups. Here's a typical comment:

"I see this as optimum for the whole class. Here we have one computer and the whole class can operate from it and it can be used individually, but I see it more as a broad thing."

They also thought GTV could be a powerful resource for small groups and for individuals. For example:

"I have lab days where I set up activity stations and the kids go from one to another in groups of three or four. This could be another good use of the system."

"I see it as something that could potentially be in the library the way we have reference materials in a library. This could be one more reference resource in the library that students could have access to [when] they're doing projects."

Thus it seems that while GTV's group presentation model is fully supported by the teachers, there is still a desire for closer, one-on-one interactions between students and the material. One teacher had this criticism of the GTV format:

"The thing that bothered me was that this is passive learning. You're sitting and you're looking and you're being fed information. You're not required to use the information. I want them to be interested and to use the information. So maybe there are two presentations and your job is to put the third together. Then you have something that would work very well."

It is important that the GTV product support and encourage classroom interactions between teachers and students, students and students, and students and material. The supplemental teacher's guide material can provide examples of discussion topics and class activities tied to specific segments and segment playlists.

 

Segments

Response to the segment model was positive, although there was disagreement regarding length, pace and informational content. For example, one teacher had these words:

"I like the short segments. I like the length because you can use them as kick off things. You can use them as review. They can get two or three concepts in ten minutes."

Other teachers countered:

"You said they were good intro and review, but I didn't see too much as far as the body went."

"I would like to see some longer segments. I had trouble retaining, even following the information as well as I do. I didn't remember too much of it."

These teachers reacted positively to the notion of a series, a group of thematically and stylistically linked segments that could be played together to make a longer presentation. They also were receptive to the idea of repeatability in segment presentation, however these reservations were expressed:

"First impressions are very important. You might not have the motivation on the part of a student or a teacher to repeat it."

On the whole, the segment model appealed to teachers. They liked the idea of working with modules and having control over the structure of classroom presentations. They liked the possibility of having many moods, paces, and textures to choose from.

Teachers could appreciate the importance of mood and clarity in a segment. One teacher had this to say about the Mt. St. Helens segment:

"The volcano piece was what, three minutes That had more impact on me that most of a full length film does...The points that were made were made so clearly and left not only information but an impression of the scope of the thing."

A less succinct segment was criticized:

"There was too much information in the logging segment. [The pictures and the narration] had absolutely no relationship. For bright kids it would be okay. For slower kids there was too much information, particularly when the voice [says one thing and the pictures something else]."

This argues for clear, focused segments. Teachers could easily recognize the difference between narration used as a contextual aid for interpreting visual information, and narration used to recite facts. They clearly preferred the former. Teachers were pleased that material could be presented in a dynamic, engaging manner. Mood and rhythm were cited as important factors in creating overall impressions.

Teachers supported the notion that visual presentations are valuable teaching tools. One teacher offered this observation:

"So much of teaching is auditory and here is something that you're seeing, that is visual. You tend to hang your ideas on a picture and remember a picture quicker than you would a word in a textbook."

 

Other Comments and Ideas

Teachers liked the idea of large text. One teacher commented:

"I liked the size of the text because there are a lot of kids who go to school who should supposedly be wearing glasses and they don't, and they can't see diddley squat from the other side of the room. Plus, it's an unreasonably large size [room]."

Comments on the music varied. Some teachers found parts of the music distracting. Others liked what they heard. One teacher had this comment:

"I think the fact that you are using original music is good because they're not singing along, mouthing the words or thinking about that rockstar or that band."

Teachers expressed a strong desire for visual indexing and a concept map for the GTV material. The GTV product and xTV Workshop should be designed to reflect these needs.

One teacher made this comment about who GTV educational material should be aimed towards:

"All those people are well taken care of, you know...people in the upper end, everything's great for them...Don't lose sight of the middle, the kid that may not go onto college. All this is great for those people [people at the top]. They're gonna succeed anyway...but the kid down below is the one were trying to reach."

The teacher's point is that the underachievers should not be neglected. GTV can be a good way to reach students who have not responded to traditional educational material. It can be a way to get them back into the process of education.

This point is important. In every classroom there will be students sitting in the back thinking "So what". The big challenge is to motivate them, as well as the quick learners.

There was disagreement as to GTV's target grade level. Responses varied with some teachers saying it would be useful for grades 3 and 4 while others felt it would be appropriate for high school. Some typical comments:

"Everyone was saying it's for middle school, but for the geography classes I teach it's ideal, and I teach in the high school section."

"I can see the segments being very useful with grades 1 through 3 or 4, perhaps even without comments, just with pictures."

The interest and desire for this class of educational product is strong. For many, the biggest problem seemed to be in finding the money to purchase a system. Overall, the teachers were excited about the possibility of having GTV in the future. One teacher put it this way:

"We will give up an awful lot of material if the thing we want is good."

Post-Demonstration Item Analysis

In this section, each question on the post viewing questionnaire is listed, a breakdown of responses follows and a quoted response from the questionnaire is cited.

Question: What were the best features of GTV

Responses:

teacher control 7

segment with concept idea 6

music 4

picture clarity 3

pace 3

overall potential 2

high interest 2

interactivity 1

up to date 1

Note: There may have been some confusion about whether or not the question referred the best features of the prototype or the best features of the GTV concept.

Typical Response:

"Flexibility- being able to use anything on the disc in any sequence- rearrange segments, or even frames. Use "as is" or build your own. Quality of picture, sound. Variety of voices and sounds utilized."

Question: What were its greatest drawbacks

Responses:

cost 4

potential teacher resistance 2

possible maintenance problems 1

music too dominant 1

passive (no interaction among viewers) 1

too simple in content 1

pace 1

use of children narrators 1

Typical Response:

"We use anything we can afford. Price is the key. If the material is good, great or even excellent, price is still the key. In some cases, something is better than nothing at all."

 

 

Question: If GTV were available to you, would you use it in your classroom this semester How would you use it (If not, why not)

Responses:

use in class presentations 5

use in lab situation 4

review and refinement 2

enrichment 1

evaluation 1

introduction 1

Typical Response:

"I would love to use it, especially the map segments. It would help students to understand how regional geography determined the way of life of the people who settled in their various locations- at present they are trying to get this information from atlases, encyclopedias etc."

Question: What is your reaction to the GTV format

All agreed that format was excellent. Suggestions for improvement were:

longer segments needed 4

introduce concepts in PSAs 4

keep large text style 1

something (don't know what) is missing 1

Typical Response:

"Segments are great. Every kid in the country knows every major commercial and watches them even when known. I assume the GTV format will aim for the same kind of awareness, use, and reuse of the materials."

Question: What do you think of the potential of GTV to motivate students about geography

All respondents agreed that the product was motivational.

Typical Response:

"I use geography as only one component. GTV will motivate students in an interdisciplinary approach designed to develop the student's critical thinking skills."

Question: Can you imagine this medium being used to deliver concepts as well as factual information

All respondents agreed to the potential of teaching concepts with GTV.

Typical Response:

"A series of stills that contrast two ideas such as conflict and resolution could prompt a lively discussion. The less concrete ideas would take more effort, but none seem impossible."

Question: Please comment on the length of segments and the pace of presentation.

Responses:

longer segments needed 6

pace OK 5

segment length OK 3

pace too fast 2

vary the pace 2

Typical Responses:

"Total daily presentation length should be 25-30 minutes."

"Depending on the subject matter, some longer segments (5 to 10 minutes) would be appropriate."

Question: Do you think students would enjoy these segments What kinds of things do you think students would learn from these segments

Seven of the eleven teachers stated that they thought the segments would be enjoyable to students. One doubted the value for learning content.

Typical Response:

"Yes! Students learn visually as well as aurally. Frequently we teachers lecture students without showing examples - and the students immediately forget what we said."

Question: How would you use today's segments in your classroom

Answers:

introduce ideas 6

general use 4

develop ideas 2

examples 2

discussion topics 2

review 2

tutorial 2

compare/contrast places 2

Typical Response:

"Some I would use to introduce a concept, some to develop an idea and illustrate it, and some, like the cities segment, to assess understanding."

Question: Which segments do you think worked the best, and why

Preferred segments:

Mt. St. Helens 8

Natural Hazards 4

Logging 3

50's 1

Cities 1

Reasons:

action/impact 4

most information 4

conceptual message 2

connections 1

related to lesson plan 1

Typical Responses:

"Mt. St. Helens worked best because of the sense of impact. You got a feel for the amount of force involved. It was exceptionally attention getting."

"The one on Mt. St. Helens and Harry Truman worked well, because of its subtlety. It lets kids "finish" the story themselves."

Question: Which segment was the worst, and why

Responses for worst segment:

Logging 4

50's 2

Mt. St. Helens 1

Reasons given:

narration 2

picture quality 2

not enough info. 2

too much info. 2

personal taste 2

Typical Response:

"Logging. It offered too many things. It started off with the pencil but never showed a pencil being produced. This can be confusing and caused mixed messages."

 

 

Question: What additional segments would you like to see

Responses:

physical geography processes 5

comparing world peoples/regions 3

resources 2

current events 1

culture traits 1

map skills 1

population/migration 1

Typical Responses:

"I would like to see more segments on map work."

"I would like to see some U.S. History related geography which relates our population growth and movement patterns to economic and political developments."

Question: How would GTV affect your teaching style

Responses:

Generally positive 7

open new options 2

same as before 1

use as slide resource base 1

student operation 1

Typical Response:

"Since I support cooperative learning and the role of the teacher as facilitator of learning processes, I would use GTV as a student tool. I would have students work in groups to access a segment and the do an in-depth study related to the segment topic. The end product would be presented to a large group of students."

 

Question: How would GTV affect the teaching style of other teachers

Responses:

generally positive 7

Typical Response:

"Most would find a place in the curriculum for it. It's too good not to use."

Question: What information should be in the teacher's handbook

Responses:

general directions 7

user demonstrations and classes 4

teaching tips 4

concept guide 3

menus 1

hotline 1

guide to match state curriculum 1

Typical Responses:

"Some specific strategies for how to use it to open a lesson, how to manipulate the stills."

"Very clear instructions following a hands on demonstration so that all of the possibilities would be used."

"Descriptions of concepts, examples of lessons and a demonstration tape showing how it could be used but leaving application to the creativity of the teacher."

Question: What are potential problems of GTV

Responses:

need for simplicity 3

size of screen 3

security 2

cost 1

improper uses 1

updating 1

Typical Responses:

"Not setting the stage for its use. Not debriefing the segment, just turning it on and letting kids watch without being directed watchers."

"This varies as much as the type of classroom and abilities of teachers. Keep the equipment simple!"

 

Question: Would you take advantage of GTV's reprogrammability How would you use it

Responses:

generally positive 6

use "what if" questions 1

use as slide collection 1

use as evaluation 1

Typical Responses:

"Move images to fit my lesson. Cut off sounds and have kids make narrations."

"To create my own segment sequences to fit our curriculum."

 

Question: How much time would you be willing to spend each day in programming GTV

Responses:

30 minutes 3

15 to 20 minutes 3

as little as possible 2

Typical Response:

"Due to my job demands, I probably would be lucky to find a spare 30 minutes/day."

 

 

Other comments on response form:

field testing recommended

product must not be boring

should not be too much "fun"

integrate with other subjects in social studies

stress world themes

integrate with other subjects in science

longer segments needed

music is not always necessary

Evaluator's Observations

D. Holtgrieve:

There is a demonstrated need for the product to be concept based.

World rather that U.S. topics were most requested.

Teacher's technical knowledge was higher than expected.

Segmental learning system is valid.

Back up support for users is critical.

Consider use in elementary grades.

Consider use of printer in system.

Longer segments should be produced.

Map based segments are necessary.

Possible new segment topics that were discussed: physical geographic processes, world political hot spots, careers in geography, world regional overviews, population and migration, data base, major rivers of the world, map skills, map projections, land use change through time, boundaries, settlement patterns, oceans, bridges and barriers, religions.

Recommend that master list of future segment topics be approved by project planners as soon as possible so that content consultants may create uniform reports for each concept. Concept outline reports would contain title, places focused on, major concepts, major facts to support concepts, suggested maps and other graphics, references for researchers and suggested place in project concept map. Outline reports would not contain story lines or detailed research findings and facts.