Multimedia in Education: What Needs to be Done?

A Position paper by Jack Schwartz, New York University


This position paper focuses on the application of currently available computer technologies to education, particularly education in basic and applied science and technology. It reflects the author's belief that as long as education remains a simple service supported by little effective capital investment other than textbook publication, its cost, like that of all other services, remains certain to rise. For the same reason the quality of education available to those unable to pay premium prices is likely to fall. Extensive capitalization of education by development of interactive computer-based multimedia-augmented teaching/learning systems offers an opportunity (in the author's opinion, one of very few opportunities) to reverse this trend.

Technology has created the opportunity we see. Hardware capable of supporting very effective educational applications is already available and is certain to become much more widely available over the coming decade. The computational power of this hardware and the functionality which it provides will also expand. Major software tools greatly easing the creation of cognitively powerful educational applications have become available and seem certain to improve. Thus, even though the hardware and software systems presently deployed in schools and homes are still seriously limiting, we see the whole field as more and more 'poised for takeoff'.

A key problem: the overburdened teacher. College level faculty know their subject matter well enough to deal confidently with it, and their teaching loads, which are very light relative to those of teachers in the lower grades, allow them the time needed to develop or collect supplementary materials whenever they want to deviate from their main textbook. The situation in elementary and middle-schools is very different. Once one passes beyond the basic daily skills of reading-writing-arithmetic, teachers will often be minimally trained in their subjects, will rarely have reflected deeply upon them, and thus will often depend totally on textbooks and other presupplied materials. Classroom loads are too heavy for teachers to be able to absorb new materials or approaches at any very substantial rate. Hence teachers can be expected to adopt new approaches and materials only if these are very 'teacher friendly', that is, reduce their existing burdens rather than adding to them. Approaches raising difficulties for the teacher will either sink into a swamp of incomprehension or be reduced to veneers of new terminology further confusing the body of traditional material with which teachers feel comfortable. The collapse of the 'New Mathematics' after a national effort and the expenditure of tens of millions of dollars shows the force of this remark.

We believe multimedia expositions to be resistant to these difficulties. Since such expositions are 'Lecture/Books', they relieve part of the teacher's burden by supplying explanation (including spoken explanation) directly, and by rehearsing student skills on-line. Multimedia educational materials can free teachers from the need to function as primary sources of exposition (which is a role in which educational research shows they are not fully effective) and let them work as sources of individualized help, as coordinators of progress, and as knowledgeable guides to the use of instructional systems. By bringing students into more direct intellectual contact with primary sources and authors of educational materials, multimedia can revolutionize the teacher's classroom role. In particularly challenging subject areas, multimedia expositions can also 'teacher-proof' instruction by supplying very carefully worked out materials preserving nuance and subtlety better than overburdened teachers can. This is the 'capitalization of education' to which we have referred.

We see this 'capitalization of education', realized in comprehensive libraries of multimedia educational materials, as revolutionizing the teacher's role. From the teacher's point of view, multimedia materials resemble the comforting and friendly wall-chart, but scaled up by a factor of 1000.

Why multimedia materials can be expected to surpass books and conventional classroom teaching as a means for education.

Education today uses few means that educators have not used for centuries. Students read books, as they have since the invention of writing. We still lecture from and about books, as professors at the University of Bologna did in the twelfth century. The formidable strengths of this established approach, and particularly the great advantages of the printed book as a means for conveying information, suggest a cautious view of all new educational technologies. Nevertheless initial experience has convinced us that multimedia expositions will have substantial advantages even over books, in many (but not in all) areas. These advantages are worth enumerating.

(a) Multimedia expositions can integrate the many aspects of a concept or phenomenon. In many cases, the correspondence between distinct representations, descriptions and/or models of a phenomenon is critical to understanding it. Integration of principles with illustrations of them is also very helpful. Multimedia exposition allows artful juxtaposition of text, formula, diagram, aural recording, animated sequence, picture, image, video clip, and can be used to show such correspondences, greatly facilitating student grasp of the relationship between general principles and their particular instances.

Among very many possible examples we may cite a few: use of position versus time graphs integrated with animated and/or video sequences showing the motion of a falling body and leading to the formula for this motion; general discussion of the concept of tolerance in machining, integrated with multi-color rendition of 3D models showing their various tolerance envelopes; narrative presentations of basic statistical concepts, integrated with illustrations of survey results illustrating their use; etc.

(b) Multimedia can show all concrete and abstract motions far more clearly than a book can, and can show transitions and motion directly. By using video and animation to show and analyze real world phenomena, a multimedia author can be effective with a much enlarged student audience. Graphic animation allows an author to abstract the aspects of the real world that he/she wants to discuss. Such expositions prepare the student to understand the relevant equations of a field; once this has been done, multimedia allows the author to show changing numbers in a series of worked out examples.

Whenever motion and change are themselves part of the subject to be taught this is of obvious advantage, but even when physical motion is not directly involved, motion acts powerfully to illuminate abstract connections. For example, a lecturer explaining the manipulation of formulae in calculus often translates his own somatic 'feel' for the abstract operations involved into helpful gestures; computer-based animations can capture these gestures but books cannot. The intellectual effort required to understand how objects are moving when they are seen in a stroboscopically illuminated scene tells us how important continuity is for comprehension of the external world. Computer-based animations can mimic these cognitively vital continuities; books can show 'before' and 'after', but authors must ask their readers to tie 'before' and 'after' together by imagining the transitions which books cannot show.

(c) A third noteworthy advantage of multimedia exposition is its ability to animate the iterative processes which sometimes underlie static final results. For example, one can show the process of convergence which expresses an integral as the limit of sums, illustrate the central limit theorem by animating the accumulation of small increments, show how distributed bargaining converges to market equilibrium, etc.

(d) Diagrams and color are themselves useful tools of exposition. Books can certainly use diagrams and color, but their use of these tools is limited by the expense of multicolor printing. Computer-based expositions can use diagrams and color far more lavishly, employing hundreds of diagrams where even a profusely illustrated book could only use dozens, repeating diagrams whenever they need to be referenced, fading between diagrams to show structural relationships, showing 3-D structures from a moving perspective, etc. All of this greatly strengthens an author's ability to communicate graphically.

(e) Computer-based educational materials can combine text and graphics with speech. This turns them from illustrated books into a new form of 'Lecture/Book' which communicates more powerfully than either lectures or books can in isolation. Spoken summaries of material also presented as text and graphics can underline main points and make them more memorable; direct availability of illustrated/animated text makes a spoken lecture easier to follow and lets it move at an accelerated pace. Since the user controls the pace of presentation and can review any part of it at any time, the lecture-room problem of 'dropout' is alleviated; also, student hesitations and repetitions remain comfortably private, avoiding embarrassment and withdrawal.

(f) Computer-based materials can combine video with text, graphics, and speech. Note that video, while very powerful in some contexts, is not always appropriate: the essential aim of any educational exposition is to concentrate the learner's attention on the central points to be mastered, and if these points are abstract it can sometimes be better to use simplified cartoons which eliminate extraneous details than real images full of potentially distracting details. But whenever a concrete presentation is wanted the availability of video allows easy transition from the abstract to the concrete and back.

(g) In a computer-based exposition one can concentrate a student's attention on new or crucial aspects of an algorithmic skill by supplying part of what needs to be done while leaving other parts to be supplied by the student. One can also make a subject area's abstract tools visible as icons or in lists, applicable by simple graphical choice. For example, in a lesson on differentiation of complicated formulae one can handle algebraic simplifications and the application of basic differentiation formulae automatically while requiring the student to indicate the order in which the parts of a formula are to be processed.

(h) Any number of unobtrusive quizzes and small graphical experiments can be embedded in an instructional narration, along with larger experiments and simulations where these are wanted. These can serve either to confirm and encourage progress or to warn the student of some developing misapprehension that needs correction; detected misapprehensions can point at correcting supplements.

(i) Computer-based instructional narrations can include flexible simulations designed to help a student to round out his/her understanding of an area. Simulations, e.g. of laboratory procedures and/or experiments, can be made very realistic by use of artfully designed 'branching video'. As desktop systems become more and more powerful and software better integrated, all the scientist's computational and visualization tools, and all the engineer's design tools, can be made available to students as tools for exploration and for development of mastery. Software systems of this kind can give students direct exposure to exactly the professional tools which they will use later on the job.

(j) The mass of material presented in a book is ultimately limited by considerations of bulk; electronic presentations can evade this limitation. This allows footnotes and digressions in arbitrary number to surround a main narrative line without detracting from it. The student can either follow a main narrative line closely, or, once having mastered it, use it as the doorway to a world full of browsable information and freely usable construction tools. Given suitable networks, multimedia systems can even link their users to increasingly massive on-line resources compiled and maintained for other purposes, e.g. to government data bases, major new feeds, on line encyclopedias, etc. (But note that the situation in regard to video is much less advantageous than that encountered in dealing with text and cartoon graphics. Extensive educational use of computerized video materials will therefore require either distribution of substantial videodisk libraries, or development of an extensive networked software system capable of downloading video materials to gigabyte desktop storage systems).

(k) Since it is nearly as easy to manufacture copies of software individually as in bulk, the author of a multimedia electronic presentation can expect to revise it much more extensively than is feasible for a book, especially if only a little photographic or video material is involved. Thus 'infinitely many editions', into which one can readily incorporate the results of experience with students, become feasible (at least for text and cartoon graphic materials; but less readily for video, as said above).

(l) The student group most in need of help is accustomed to get information visually (from video). They relate well to this medium and their retention rate is higher than it is for spoken words or printed text.

Key aspects of interactive multimedia educational materials.

Ideally, multimedia educational materials combine the three fundamental kinds of materials, namely (a) educational narrations, which present the content of a subject area to a student in a form which, although visual and suitably enriched with sound, video, and some interactivity, is nevertheless didactic in essence; (b) electronic 'museum databases' for free browsing by the student. Examples might include hypertext libraries of biological images for study, similar libraries of historical, geographical, architectural, or manufacturing-related materials, organized collections of literary or musical materials, etc. These may provide some guidance to the student, but basically they allow the student to browse freely, so as to acquire general familiarity with key subject area facts. (c) free-form multimedia 'laboratories', which organize the tools of an area of study and put these at a student's disposition, and invite him/her to use these tools to build something. This kind of 'laboratory' software may appropriately suggest general goals and provide some guidance but nevertheless they allow the student very wide latitude in selecting a specific 'build target' and choosing an approach. In view of their flexibility and generality, electronic 'workrooms' of this kind are comparable to tinkertoy sets, or to boxes of blocks. When well designed, they stretch the imagination and tempt to independent creativity, but without overwhelming. 'Laboratories' of this kind will often incorporate sophisticated simulations or other elaborate computations. sequence of activities. Their authors can readily supply graded sequence of problems, experiments, and other relevant learning activities, and can correlate these with prior narrative presentations. Mechanisms for giving help where this is wanted, for verifying success, for storing draft and final versions of projects and reports, and for communicating with local or remote teachers and knowledgeable peers are all easy to provide.

Most current multimedia materials focus on one of these three approaches. That is, some of these materials are in effect illustrated lectures, with some provision for user-steered navigation around topics; others are a compilations of reference materials a the chosen topic, intended for student browsing; while others are electronic tinkertoy sets which provide students with a laboratory environment which they can explore freely. None of these individual approaches is complete: none by itself leads to optimally effective learning, though all can be used with some success by active, creative teachers. As tools improve, materials authors will move to production of modules which incorporate all three approaches, i.e., giving students packages in which review of preplanned expositions, free browsing, and workspace exploration alternate in an orchestrated way.


In spite of the formidable advantages enumerated above, development and massive deployment of multimedia educational applications still faces significant obstacles. Since a central aim of the activities I should like to propose is to break though these obstacles, they need to be defined.

Basically, I see three principal obstacles, namely (i) the pragmatic obstacle: Presently the multimedia market is fragmented among various manufacturers: Apple, IBM, foreign and domestic manufacturers of “IBM clones”, Amiga, NEXT, etc. These hardware capabilities are accessed through an incoherent melange of occasionally compatible, but more ordinarily incompatible, software tools, which differ from platform to platform. This fragments the market and breaks up the cash flows needed to pay for first-rate products by limiting the circulation of educational software products to particular platforms and software environments. (ii) the fragmentary state of currently available software tools. The best current software tools serve well for the production of educational narrations and for assembly of browse-oriented 'museum databases', but are grossly inadequate for production of more highly interactive materials, particularly for development and integration of free-form multimedia laboratories incorporating sophisticated simulations. This shortfall contributes directly to the final obstacle I wish to list, namely to (iii) the sheer cost of materials production: even with better tools, the cost of producing real electronic educational classics can be expected to be quite high, given all the video production, graphic design, and interactive materials development that they are certain to entail. Without much improved software tools these costs will remain prohibitive. (And, even when such tools become available, potential materials authors will need to learn their effective use. This requires skills which educators will have to learn: production of effective multimedia lessons requires not only understanding of the content area to be addressed and its pedagogical problems, but also special technical skills in the use of multimedia computer tools, as well as the design skills which enter into production of professional-level image and video. Another skill that is wanted is the art of producing base educational modules which are portable across platforms and which remain as "open" as possible to extension and to rework for a new audiences. People, and even teams, combining all these skills are still rare).

What Needs to be Done.

To address the complex of issues outlined above, an ambitious effort is needed. What is wanted is a nationally visible effort which will announce U.S. intent to (a) become the dominant player in a worldwide, multimedia-based transformation of education; (b) make maximal future use of multimedia educational technology; (c) raise the quality of the educational software products available to schools. This must reach levels of comprehensiveness and excellence which can command the full allegiance of educators. (d) Move to experimental, and then to full, deployment of multimedia educational systems as soon as materials of suitable quality becomes available.

To create this effort, one needs to shape a Center which gives institutional form to its goals. This center must aim to 'push the margin' of the developing technology by encouraging the formation and efforts of major materials and tool development organizations and consortia. (Its operations should also “stress the envelope” in regard to educational use of telecommunications). To educational materials developers the Center must say: 'If your materials are good and comprehensive enough, we will pave the way for their broad adoption by the $300 billion education market'. To software tool developers it must say: 'We are in process of greatly enlarging the market for the products which your tools support. Increasingly sophisticated products are being designed, but your tools in their present form are inadequate for development of these products. Improve these tools and you can expect to profit greatly as the educational software market grows'.

How should such a Center be structured? To be credible in market terms, it needs to be rooted closely in the realities of mass teaching. To be technically credible, while remaining commercially neutral, it probably needs university roots. Several possibilities are worth contemplating, the ultimate choice being dependent on the funding agency focus and the identification of suitable partners:

(i) a community college / adult education facility

(ii) a middle school, the middle school level chosen because it appears to be most amenable to innovation.

Many topics for a multimedia approach can be ideal are routinely offered in all these institutions. These include algebra, statistics, machining related theoretical topics such as geometric tolerancing, various computer literacy / computer skills topics, etc.

My own preference tends toward creation of a new, or reorganization of an existing, junior high school dedicated to the pioneering role outlined above. Why junior high school? Because this creates a particularly broad charter, ranging from the lower grades to the later high-school grades. But the other possibilities enumerated above all need examination.

The Center created is to operate with the close involvement of (and perhaps under the supervision of) a university or university consortium willing and able to lead the technical/organizational outline. The work of this 'lead group' will involve continuing contact with schools nationally, and with other universities and university consortia; ongoing efforts to develop exemplary materials; work with hardware developers and tools providers; participation in standards efforts; an active program of conferences and publications; and network dissemination of qualified materials.

Economic Impact.

It is generally agreed that the weakness of U.S. elementary education hurts the nation's competitiveness by denying it a work force having the 'shop-floor engineering' capabilities available to its strongest rivals, notably Germany and Japan. This national weakness begins in the elementary grades and mounts year-by year through high school. We believe that development and deployment of multimedia educational systems along the lines we have described can remove this weakness, both by strengthening elementary/middle school education and by enabling new forms of lifelong learning available outside the conventional schools system. If this is true the economic benefits of the program proposed can be enormous.

Another economic point is worth considering. In the U.S. alone, education is a $300 billion dollar industry. Over the next few years, electronic/multimedia capitalization of this industry can easily generate multi-billion dollar market for educational materials. Dominance of this industry will fall to whoever perceives its requirements and opportunities most quickly and clearly. By accelerating this process of discovery, the activity we propose can make a second major contribution.