animated clay talking head image

Talking Books as Assistive Technology
TED 503 Project - J. Black
Fall 2006
 

 

A talking book provides access to literature for individuals who may otherwise have difficulty reading independently. Since the availability of software and hardware often governs whether or not AT (Assistive Technology) resources are utilized, it should be noted that talking books can easily be created using a computer with PowerPoint (or other widely available software) and a scanner or digital camera. (Talking books may also be created for the WWW using a basic web editor such as Front Page or Nvu.)

The beauty of talking books is that they can be personalized to target individual interests and abilities. Sound effects and speech can be added. Customizable avatars may be used to coach or to provide assistance to students with reading difficulties. Newer technologies even make it possible to include "signing avatars" for deaf or hearing impaired individuals.

Consequently, depending on how the books are set up, they have the potential to assist:

Learners that are:

  • beginning readers
  • individuals with reading difficulties
  • ESL learners
  • individuals with handicapping conditions that impair their abilities to hold books and turn pages
  • learners with visual impairments
  • persons who are deaf or hearing impaired

In addition, this same technology may be used to encourage students to create their own books, and thus improve literacy from both a writing and reading perspective.

Solutions

Let's look at some sample talking books created with PowerPoint. The first book would be suited for beginning readers, young ESL learners, early learners with reading difficulties, or those with difficulties turning pages. This book  is considered to be on a B Level for "guided reading". Its use is popular with students in an early intervention literacy  program, as well as  in a regular K-1 setting. [Additional book information: The Tree House (The Wright Group: The Story Box Series) by Joy Cowley.]

The Learning Environment ~The students targeted for early intervention receive language arts/reading instruction on a daily basis in their classrooms with other classmates, plus additional individual or small group instruction with a reading specialist each day (or every other day) using an instructional approach similar to a Reading Recovery. There are many books and print resources available. Each classroom has 2-3 computers. Frequently a reading specialist pushes in to the classrooms to work with students. The "reading room" has 4-5 computers (most computers in the school are on a network with Internet access and have Windows XP with Microsoft Office, among other applications). The atmosphere in the reading room is collegial, positive, and encouraging. Students are encouraged to demonstrate increased fluency and comprehension, hopefully on guided reading levels appropriate to their grade.

Sample Talking Books

  • The Tree House  -- a sample talking book created using a CrazyTalk avatar. The CrazyTalk software utilizes both a model component  that is animated, and a script component to develop the avatar. The avatar used with this talking book is a small tiger cartoon. The  avatar reads the text using TTS (text to speech) whenever the play control button is clicked. The two CrazyTalk component files for each slide are created and saved in the same folder with the PowerPoint file. 

    Then both files are inserted into the PowerPoint using the procedure outlined below. (Note: It is necessary to embed the files in order for the PowerPoint presentation to be used on other computers.) The learners need only click on the avatar when they wish to listen to the page being read.
     
  • The Tree House -- a sample talking book using PowerTalk. Note: Click and save the PowerPoint file, then open it with PowerTalk  (must have PowerTalk installed on your computer!). In this talking book,  the text already appeared on the scanned book pages. So, additional text was added to the  slides using a "white" font to make them invisible to the viewer but still enable PowerTalk to read them (PowerTalk will not read words on the scanned book page images). This way PowerTalk reads the words and the viewer sees only the words on the scanned book page. Using "invisible" text might also provide an alternative method for including questioning or prompting for strategies rather than merely providing story narration.

Other Examples

  • All The Places to Love - talking book with recorded sound  developed by H. Johnson for use with Kindergarten
  • Ira Sleeps Over -- talking books with recorded sound developed by H. Johnson for use with Kindergarten

Task 1 -- Read Talking Books Click here for sample reading lesson for The Tree House

Task 2 -- Create Talking Books Click here for details

The Tools

What are the no tech, low tech, and high tech options that are available for using and creating Talking Books with regard to the learners described above?

Consider that low tech and high tech depends on a person's facility and expertise using the available technology, as well as its affordability. The other part of this question involves making philosophical and/or aesthetic decisions.

In an increasingly virtual world would we rather hear an artful teacher read a story (where interactions and opportunities for learning are anticipated based on years of experience and educated insight and embedded in the recording), or listen to Microsoft Sam mechanically read (minus any nuance the author wove into his/her artfully crafted text)? (Would one rather talk to a real person for customer service or be stuck in a phone mail maze?) When  looking at a beautiful illustration, do we really want Peedy intruding? Can less obtrusive systems be developed? When is obtrusiveness outweighed by necessity, increased student performance, a motivation for learning, or increased user independence?

Some would argue that using a pencil to illustrate a point, maintaining eye contact, or asking the right question constitutes "technology" --  technology is any tool used to enhance communication and promote learning. Partners reading together is probably a low tech approach to talking books. Creating PowerPoint talking books and utilizing PowerTalk kicks it up a notch or two. Next would be recording the narration in conjunction with the PowerPoint. Adding avatars and interactivity to the PowerPoint raises the bar some more. Using "signing avatars" increases the level of technology further.

These levels of technology are considered in the context of  making use of hardware and software that is somewhat readily available to teachers. As with everything, it depends on what you have to start with and the uniqueness of what is required by your individual students.

Given all the variables, you  keep UDL in mind, and go as far as you can with what you have.

Using PowerPoint

Here's everything you need to get started:

Adding Audio

  • Use Audacity to record your own audio. (Free Download) The advantage to recording a human voice reading the text is that you will undoubtedly end up with more expressive speech and more realistically controlled inflection, and emphasis. This is difficult to get with most TTS computer voices. There are many providers for additional voices (albeit for a fee).
  • Use PowerTalk to provide automatic speech for PowerPoint. PowerTalk speaks the text on the slides. (Free Download!) It reads ALL of the text! (all of the time!)
     
  • AT Tutorials for other products

Using Avatars

What are the advantages and disadvantages to using avatars as assistive technology? Are they a distraction or do they really assist and/or motivate learners?

  • Peedy, the talking parrot, is a popular Microsoft Agent avatar. Click here to see Peedy in action.  Also find some other excellent Universal Design for Learning resources.
    • Click Microsoft Agent Technology to learn more about how to add an interactive personality like Peedy to a learning application.
    • Learn more: Microsoft Agent Ring, an organization featuring websites utilizing Microsoft Agent technology
    • Get Mash, a Microsoft Agent Scripting Helper
    • Read Stories: Microsoft Agent characters read stories
       
  • Signing Avatar demos The software from the www.vcom3D.com site looks incredible, albeit a little on the expensive side.
     
  • Crazy Talk is a program that allows you to import your own photos or audio and/or create a TTS script to customize your avatar. You make adjustment to four key points on the face of the photo. Then when speech is added, the avatar appears to speak. (See Rosie sample.) The avatar can then be inserted into a web page, PowerPoint presentation, etc. You can animate objects, drawings, cartoons, and photos. Viewers may need the install the CrazyTalk Player.  ActiveX is involved.  Crazy Talk may be problematic on some computers.
  • VSign Virtual Sign Project -- software that can be used to create and play 3D animations of sign language. (Click "Software" > "Prototypes" to get to the Free Download page) For the most part, you would need to know ASL (or other) signing in order to build and author signing animations. So, you can see the appeal for affordable software that draws on an extensive database of animations that can be activated automatically via the text one types!!
     
  • eSign - the eSign project also uses a signing avatar. The eSign plug-in is free. However, the editing tools require a license that is free for educational and research uses.

Issues of Copyright

Most books are copyrighted. So this needs to be taken into consideration when scanning books. How copyright affects creating resources for persons with disabilities is explored in this Talking Books tutorial: http://www.atto.buffalo.edu/registered/Tutorials/talkingBooks/powerpoint.php

Ideally, you would contact the author/illustrator of the book and seek permission. This is not always the most timely solution since it might take weeks to get a response (and you may not even get a response). However, this would be a must if you intended to make the books accessible from the Internet. Creating talking books for use offline is more up to your interpretation of "Fair Use".

How does Fair Use affect the creation of Talking Books? Try this checklist.

Other Articles and Resources

ASL Animations as Visual Supports in Multimedia Literacy Software

RMTC-D/HH -- links to a number of useful AT literacy resources including accommodations for learners.

MultiReader Project -- access to many topic related conference papers

Adaptation of Multimedia books

Reading Multimedia Documents

MultiReader: a multimodal, multimedia reading system for all readers, including print disabled readers eBooks.

This article discusses problems of navigating eBooks and designing prototypes that address these issues - multimodal interfaces. Researchers looked at user requirements as well as habits and strategies in order to design a multi-reader.

Daisy - Digital Talking Books -- give readers greater access to navigating book structure that was not possible with analog talking books

CAST Universal Design for Learning

Universal Design for Learning

Books Online

Children's Books Online ~ The Rosetta Project

Accessible Books Online - provides digital text for persons with disabilities

Full Text Books Online Carnegie Library of Pittsburg

Interactive eBooks for Children

Online Talking Books  List of resources

International Children's Digital Library

Talking Books Articles

Wood, R., Rawlings, A., & Ozturk, A. (2003). Toward a new understanding: The 'Books Alive! Multimedia project'.  Reading , Retrieved October 20,2006 from Academic Search Premier database.

This article discusses how Talking books (CD ROMs such as the Living Book Series by Broderbund) give primary children opportunities to read a stories independently (or listen and watch). Words may be highlighted and phrases can be heard. Research indicates there is a benefit from the interactive nature. Word accuracy increases in story context. Beyond these noted benefits there are possibilities for multilayered interactions with content via many new technologies. However teachers indicate they aren't always aware of the potential for new technology (as they are not necessarily multimedia experts). There needs to be collaboration between multimedia experts and skilled teachers to create quality talking books/software. Thus the project to create quality multimedia materials that can be accessed via the WWW is in progress. The potential for talking books is not yet fully realized.

Oakley, G. (2003). Improving oral reading fluency (and comprehension) through the creation of talking books. Reading Online, Retrieved October 22, 2006 from Academic Search Premier database 

A small group of students at a private school in Australia created electronic talking books using Illuminatus software. A "formative experiment" research methodology was used. The results of this study indicated that creating multimedia talking books provided a context for improving oral fluency and might even be preferable to some of the more traditional strategies -- especially for phrasing and expressiveness. Students who had been at the bottom of their class  for oral reading ability and comprehension were able to perform on par with the rest of the class at the conclusion of the project. Students were motivated and eager to share their talking books. While this approach does not replace other strategies it does provide an additional option useful for struggling readers and reluctant writers. Because of other variables it wasn't conclusively determined that the improvements indicated by the outcomes were solely the result of the interventions undertaken.

Wood, C. (2005). Beginning readers' use of 'talking books' software can affect their reading strategies. Journal of Research in Reading, v.28, i.2, pp 170-182. Retrieved October18, 2006 from Academic Search Premier database.

This research study explored whether phonics-based talking books compared favorably to one-to-one reading with an adult tutor for improving phonological awareness. Two groups of beginning reader students were used for this small study. The results indicated that using the speech feedback component of the software was associated with gains  in ability for rhyme detection, and changes in some of the reading strategies the children utilized. The software, however, did not outperform the one-to-one tutoring with an adult with regard to attainment of phonological awareness on an overall basis. More research is indicated to determine the conditions for software that will be of benefit for "reading attainment as well as interpersonal and cognitive" skills.