Syntactic Theory of Visual Communication

Paul Martin Lester, Ph.D.
Professor
Department of Communications
California State University at Fullerton
Fullerton, California 92634
VOX: 657/278-4604
FAX: 657/278-2209
VAX: lester@fullerton.edu
(c) 2006

There can be no words without images.
---Aristotle

Archeologists in the year 3706 uncovering the buried ruins of any major city in the world will no doubt find text on billboards, storefronts, traffic signs, and so on in the languages we know and use today. These words however will probably not be understood by 38th Century scientists because languages of today will eventually become obsolete and forgotten. Luckily, there will be an energetic and tenacious researcher with a well-used digging tool who will find along the viaducts and abandoned highways in the old cities evidence of writing that will be instantly recognized and easily read. For amid the buried rubble of civilizations long past will be elaborated and brightly colored signs and symbols created by graffiti artists that will last through the millennia. This often scoffed and criminalized form of visual communication will in the future become the one, universally accepted language. Therefore, the future of mass communications does not rely on the preservation of pens, paper, computers, or satellites. In the vast future, we will understand ancient civilizations because of compressed paint in spray cans.

Before we are four years old, most of us have learned The Alphabet Song. Sung to the same tune of Twinkle Twinkle Little Star, it is unlike any other song because no pictures come to mind when singing it. With Twinkle, we can look up in the night's sky an imagine a little star out of the billions shining just for us. But a song about the letters in the alphabet do not carry any visual equivalents. Children soon match up, however, concrete nouns with images for each letter in the song. Children's books help to solve the mystery. "A is for apple ...." Each letter of the alphabet becomes a picture that corresponds with a complex set of direct and mediated images. We no longer have to think of an actual red, juicy apple. We can simply see the letter 'A' and know that it stands for that fruit.

Before children learn to read and write, they do not know the difference between a line drawing and a letter. When an adult writes an 'A,' it is simply another drawing. It is a picture, different than a face or a house, but it is still just another image drawn with a colored pencil on while paper. Soon children learn that combinations of these letter-pictures mean more complicated things. When the drawings 'A-P-P-L-E' are combined, they form another picture which we learn stands for the name of the fruit. Now the letter-pictures become word-pictures that can spark other images in our minds of the thing they stand for. We further learn that these word-pictures can be combined with other word-pictures to form sentence-pictures. To a child, there is no difference between words and pictures -- they are one and the same.

Early on however, we are taught to make distinctions between words and pictures and to not think of them in the same way. We are taught that although we can gain meaning from each, reading words is valued more than reading pictures. We are taught that pictures play a separate and subservient role to the words. And although we are taught how to make pictures with our colored pencils and our watercolor paints, we get much more instruction on how to form, with our large lead pencils, the lines and curves that make letters and words. We get one class where we make pictures -- art. The other classes are devoted to writing or reading stories whether in a grammar or in a geography class. We are taught to read stories, but we are never taught how to read images.

In the Disney classic, Beauty and the Beast, the macho Gaston satirizes Belle's reading habits. "How can you read this," he asks, "there are no pictures?" She answers with a condescending, "Well, some people use their imagination." And yet, when the viewer of the animated movie is shown a close-up of a page in her book, she points to a picture of a castle that illustrates the story.

There are strong indications that the status of images is improving. We live in a mediated blitz of images. They fill our newspapers, magazines, books, clothing, billboards, computer monitors and television screens as never before in the history of mass communications. Something is happening. We are becoming a visually mediated society. For many, understanding of the world is being accomplished, not through reading words, but by reading images. Philosopher Hanno Hardt warns that the television culture is replacing words as the important factor in social communication. Shortly, words will be reserved for only bureaucratic transactions through business forms and in books that will only be read by a few individuals. Reading is losing to watching because viewing requires little mental processing.

Critics blame everything from the rise in the crime rate to the deterioration of educational institutions on the concurrent rise in the number of mediated images that can be seen daily. Rebellious youth cling to visual symbols because words are associated with old ways of communicating and old ways of establishing social order. Words are repressive while pictures are fascinating, easily understood within a particular culture and can be made personal mediums of expression. One of the first acts in 1917 by the new Russian government was to transform churches into motion picture theaters in order to show propaganda films. One of the first tasks of a rebellious force is to capture the television station. All rebellions understand that to control a country, the pictures must also be controlled.

Several years ago, a headline alarmed many around the country: Why Johnny Can't Read. The question is often asked by educators who worry about the thinking and writing capabilities of their students. The answer is often simplistic: Too many pictures and not enough words. But there are other answers. Maybe there is little written that persons want to read. Maybe persons feel there is no point in reading when there are no jobs that require reading. Maybe parents do not read and do not support that habit in their children. Or maybe individuals are reading, but what they read is not understood by those from other cultures.

Visual messages, with their own rules of syntax, are being read, but this language means nothing to those who can only read words.

The wall space and signs in many cities often are coated with multi-colored spray painted messages. Termed vandalism, graffiti or tagging depending on the speaker, these visual messages are actually a complex written form of communication. Graffiti messages may mean the mark of a territorial border by a gang, a plea for understanding and hope for the future, grief for a killed loved one, anger toward an enemy, a show of playfulness and humor as part of a national fad, an act of criminal vandalism or simply an individual expression that signals the writer's existence. As with any symbolic communicative system, if you do not know the language, you will have trouble deciphering the message.

A brave new (visual) world

Although it is unclear what may be the social, religious and educational affects this visual culture will have upon the world's society, the use of images may foster a return of the word's importance. Or rather, a communication medium in which words and pictures have equal status may be a result of the recent explosion in pictures. Because television images cross all international borders, they become more easily understood by almost everyone. Words are easily forgotten, but pictures stay in our minds. We may not remember many of the facts that led to the brief student uprising in China's Tiananmen Square in 1989, but you can never forget the image of the lone protester standing defiantly in front of a line of menacing, green Chinese tanks. If you have seen the picture, you remember it not only because it is a highly emotional image, but because you have thought about the image in your mind with words. Words and pictures become one powerfully effective communicative medium inside your own mind.

John Sculley, the former CEO of Apple Computers, Inc. is not afraid of the world's dependence on images for communication. "We live in a visually intensive society," he asserts. Look at the most common medium for visual expression -- television. Programs can be watched from direct broadcast, from cable and fiber optics, from satellites, from VCRs and from laser disks. On some television sets you can even watch more than one program at a time. In a television commercial for Kodak's Photo CD technology in which still images can be viewed on the screen by a CD disk player, the announcer reports that "Pictures have never been so powerful." When televisions and computers are linked, viewers will actually be able to alter the content and technical considerations of programs to suit their individual interests. The combination will indeed be a powerful medium for pictures.

Computers make the production and distribution of images available to almost anyone at an incredible speed. More than any other technological innovation, computers are responsible for the explosion in images. Like the 'Big Bang' that many scientists think sent the universe on its course, computers will serve to expand the use of pictures. Today, 20 percent of the U.S. population can use a computer. But 80 percent of school-age children have learned to become computer literate. By the turn of the century, Sculley predicts that 98 percent of all the words and pictures created in the world will be computer mediated. By that time, virtual reality -- the ultimate fusion of computer and television technologies in which viewers become active users of the medium -- will be inexpensive and accessible.

Educational psychologist Jerome Bruner of New York University cites studies that show persons only remember ten percent of what they hear, 30 percent of what they read, but about 80 percent of what they see and do. When all members of society whether at home, in school and on the job learn to use computers for word and picture processing, the switch will be made from passive watching to active using. There will no longer be the barrier between the two symbolic structures. Words and pictures will become one, powerful and memorable mode of communication.

The syntax of symbols

Linguistic theorists categorically assert that since pictures are presentational and not discursive, they have no formal grammar. Without a grammar, images cannot be considered a language. Without a language, pictures cannot be read. Traditionally, syntax and grammar refer to the system of rules that are used to turn words into sentences. The syntactic rules applied for proper sentence structure are culturally based -- they change from language to language and over time. Once learned, the rules make it easy for an individual within a culture to write and read stories on an infinite variety of subjects about the myths that shape a culture. Words are the basis for human understanding because of their narrative structure. Philosopher Susan Sontag writes, "Only that which narrates can make us understand."

There are two major stumbling blocks that linguistic philosophers impose as to why images cannot be considered a language:

1) Images do not have common elements similar to a written language's alphabet and

2) Images have no recognized syntax.

Linguistic philosopher, Noam Chomsky writes that "a language is defined by its alphabet (i.e. the finite set symbols out of which its sentences are constructed) and its grammatical sentences."

American philosopher Suzanne Langer wrote "Photography ... has no vocabulary .... There are no items that might be called, metaphorically, the 'words' of portraiture." For Langer, the problem with admitting that there is a visible language is not that there are no words -- pictorial elements -- in still or motion picture photography, but that there is no universally acceptable language of visual description. However, some have devised such a language for visuals with well-defined alphabets.

French semiotician, Fernande Saint-Martin attempts to create an alphabet for images in her book, Semiotics of Visual Language. For Saint-Martin, color is the basic visual element. Color, as a form of light, gives shape and substance to the visible world. Her basic visual alphabetic letter is called the "coloreme." A coloreme is the smallest element within a direct or mediated image that can be focused within the foveal field of the retinas. A coloreme can be composed of a picture's actual color, texture, size, boundaries, direction or position in the frame of view. These physical attributes of the image, once noticed and identified, find meaning through successive viewings, similar to the ideas expressed by the constructivists.

In 1987 Irving Biederman published his theory of visual perception in which he outlined the alphabet of objects. Any written alphabet is simply a collection of symbols that correspond to the sounds made during speech. A basic unit of speech is called a phoneme. For all the languages in the world, only 55 phonemes are needed. The 44 phonemes for the English language are simplified into 26 characters -- the alphabet. Webster actually lists 50 alphabetic symbols that are used to describe all the sounds needed to pronounce the more than 50,000 words in the Dictionary, but 26 letters suffice. Biederman recognized that every object is composed of primitive shapes or parts. He called these basic components "geons," short for geometrical ions. Through his research, he discovered that only 36 geons are needed to make all objects. But because of previous experience with an object, most persons can recognize it by viewing only three geons. An object that is unusual, obscured, seen at an odd angle or out of context will require more geons. It seems that the mind may store images symbolically within the brain in the form of geons in order to make recognition quick and simple. Biederman's work may be an important link between the way words and images are thought to be viewed. Once a written language is learned, a reader usually does not have to consciously analyze every letter within a word. Whole words become a part of a "verbal geon."

The problem with the theories of Saint-Martin and Biederman is that their schemes could never fully describe all of the elements that make up a detailed image. Coloremes and geons offer indications of a symbolic storage of images, but the processing of coloremes and geons become so individualized to the point that no two persons would identify the same pictorial elements. A visual 'alphabet' should be easily recognized by all those using it every time. Coloremes and geons are far too abstract for such a purpose. There can probably never be an alphabet for images. Nevertheless, the brain does respond to an image's basic visual elements of color, form, depth and movement. Although these elements may not have the simple symbolic meaning of alphabetic characters, a difficulty in finding an alphabet for images indicates the richness of communication possibilities of pictures over words.

The other major problem linguists have over regarding images as a language is that there is no recognized syntax or order in which to view the elements within a picture.

Noam Chomsky writes that languages contain surface and deep structures. Surface structure refers to the rules of grammar while deep structure involves knowing the meaning of each word within a sentence. A sentence is considered to have proper grammar if it follows specific rules and if the words are understood. A chief rule of syntax is the arrangement of the words in their proper order. Wilson Hicks, an early picture editor for Life magazine writes that "the verbal medium is discursive." Words must follow a linear, horizontal order between two points. Langer asserts that a visual structure is taken in all at once by the viewer without any specific rules of order.

The sentence, "I like green or red apples" obeys the rules of proper syntax. The words are in the proper order and convey meaning. Now consider the sentence, "I like said or verb apples." It is grammatically incorrect. The word choices are confusing. The sentence makes no sense. But within this context, the sentence, "I like said or verb apples" has meaning because it is used as an example of improper syntax. Context, then, is a key factor in determining meaning. Words do not always obey the standard rules of syntax, yet they communicate. Poetry is an excellent example of words playing with syntactic rules.

The sentence, "Smoc moccoa crylie schemee" may or may not be of proper syntactic orientation because the phrase is foreign -- the words have an unknown meaning. The sentence, "My quoin gave me pneumoconiosis" may or may not mean something to you depending on your knowledge of the English language. Still, you would recognize the two groups of 'words' as sentences.

Words do not always need to be known in order to form sentences. Author David Lodge once wrote that "Language is a net that holds thought trapped within a particular culture." Imagine all that is lost to you because you do not know the meaning of the individual words.

A basic rule of syntax is that words that make up a sentence follow a linear, discursive pattern. The sentence, "Apples like or green I red" violates that rule. And yet, like poetry or a foreign language, and helped by a previous experience with the sentence in its proper grammatical order, the sentence does have meaning. (Try reciting the mixed up sentence to friends and see if they can figure it out.) Words within a sentence do not necessarily have to follow the rule of discursiveness in order for there to be meaning. Grammar is simply a set of rules invented to make life easier for those who use them.

The great documentary photographer, Lewis Hine, who often used words to accompany his photographs once said, "If I could tell the story in words, I wouldn't need to lug a camera." It is beyond question that words and pictures are different animals. But each possess a language that some can interpret better than others. Photographic historian Helmut Gernsheim wrote that "Photography is the only 'language' understood in all parts of the world, and bridging all nations and cultures." On the other hand, photography philosopher John Berger admits that "photographs supply information without having a language of their own. Photographs quote rather than translate from reality." Sol Worth, an expert of visual communication wrote of a compromise between the two points. "Pictures are not a language in the verbal sense. Pictures have no lexicon nor syntax in a formal grammarian's sense. But they do have form, structure, convention and rules."

The semiotic approach to visual communication stresses the idea that images are a collection of signs that are linked together in some way by the viewer. The study of semiotics, as detailed by Morris, divides itself into three areas: pragmatics, semantics and syntactics. Pragmatics is the study of the origin, common uses and communicative effects of signs. Semantics is an area of semiotics in which the researchers attempt to determine the significance of signs within and throughout various cultures. Syntactics is the study of the ways signs are combined with each other to form complex messages.

Since semiotics grew out of linguistic theory, pragmatics, semantics and syntactics are terms used in analyzing written or spoken communications. However, since a direct or mediated image is nothing more than a collection of signs, the area of syntactics is of most interest to visual communicators because an image is a collection of graphic elements that convey meaning for the viewer. Peirce wrote that a sign always refers to another sign and never to the actual object "in itself." Agreeing with him, de Saussure noted that the meaning communicated by a sign depends not on actual objects but on other signs. For him, a sign by itself means nothing.

Roland Barthes described the "chain of associations" or signs that make up a picture's narrative. In verbal language the narrative is linear. One word follows the next in a specific rule-based order. In that regard, verbal communication is considered discursive. Pictures, on the other hand, are presentational. Signs within an image are presented in various ways, many times depending on the style of the image-maker. The chain of signs is more tightly controlled with text than with images. One exception might be poetry in which the order of the words can have non-linear, presentation qualities. In fact, the Greek poet Simonides around 500 BC wrote that paintings were "silent poetry and poetry painting that speaks."

Individual symbols within a picture have no precise alphabetic relationship, but when used in combination, meaning is found for an image through a traditional, discursive method. That method happens inside the mind of the viewer. Suzanne Langer wrote that "grammatical structure is not a symbol of itself, but it ties together several symbols, ... to make one complex term." In a picture, that term or idea, as Langer writes, is expressed through "visual forms -- lines, colors, proportions, etc. and are just as capable of articulation, i.e. of complex combinations, as words." Some might argue that words express ideas through "visual forms -- lines, colors, proportions, etc. and are just as capable of articulation, i.e. of complex combinations," as pictures. The syntactic theory of visual communication

The writer William Saroyan once remarked that "One picture is worth a thousand words. Yes, but only if you look at the picture and say or think the thousand words." Suzanne Langer writes that because language names relationships rather than illustrating them, "one word can take care of a situation that would require a whole sheet of drawings to depict it." For her, one word is often worth a thousand pictures. The photographer Minor White wrote a little poem about the two forms of communication:

I delight in photographs.
I delight in words.
I delight in mixing both
To see what happens if they blend.

Words and pictures are intricately linked in journalism, advertising and educational contexts. Words printed under a photograph, the caption, signal the importance of the common sense representations portrayed in the image. Words beside a picture in an advertisement explain a product and its attributes clearly to a potential customer. Words spoken by an instructor give explanations and life to slides projected from the back of a darkened classroom. Words and images work together for most successful communications. As Hicks wrote in his book Words and Pictures, "It is not correct to say that either medium supplements the other. The right verb is complements. Even pride of authorship are mingled."

The words that are associated with a picture and the pictures that accompany words should never be chosen casually. Both symbolic systems have enormous impact upon the reader. Roland Barthes noted that "The text loads the image, burdening it with a culture, a moral, and imagination."

Perception of the importance attached to words or pictures in publications is often communicated by the size, position and proximity of the words to the visuals. On the front page of a newspaper, the most important story and photograph of the day often takes up the most space and is above the horizontal fold. The headline, story text and caption are near one another. Since the caption and picture credit are nearest to the image and often duplicate the picture's content, Barthes asserts that they become part of the picture's denotative message. The headline and story, because they are printed farther away, have a slightly different connotation than that of the picture. Newspaper reading research has found that readers look at a photograph first, scan the caption, read the headline and if they are still interested, read the story.

For most publications, a photojournalist is strictly responsible for the image, a reporter writes the story, a copy editor makes up the headline, another editor writes the picture's caption and a layout artist puts the words and pictures together on the page. It is a wonder that there are not more connotative problems between words and images with so many different individuals involved in the process. If a caption, for example, does not adequately explain the content of a picture, confusion among the readers can easily be the result.

Despite occasional problems in discerning the meaning from pictures and words in publications, the combination of the two symbolic systems is one of the most powerful communicative strategies known. Wilson Hicks writes that when words and pictures are equally expressive, the two become one medium where "the meaning of the work can be achieved in one perceptual act." John Berger also celebrates the word and picture collaboration:

In the relation between a photograph and words, the photograph begs for an interpretation, and the words usually supply it. The photograph, irrefutable as evidence but weak in meaning, is given meaning by the words. And the words, which by themselves remain at the level of generalization, are given specific authenticity by the irrefutability of the photograph. Together the two then become very powerful; an open question appears to have been fully answered.

The first tenet of the syntactic theory of visual communication is:

Mediated words and pictures have equal
importance in the communication process.

Words and pictures are both collections of symbolic images. Words are signs composed of lines, curves and open and closed shapes. Words, as with pictures can be presented in a variety of colors, forms, depths and movements. Words have their historical roots as images and are still thought to be works of art by typographical designers and calligraphers.

Words have such a long history and are used by so many that it is easily forgotten that they are actually equivalent to line drawings. Words, made up of individual letters, are pictures that have completely lost the resemblance between what they are and what they stand for. In that regard, words are the quintessential symbolic image. It is important to consider the history of words because that says much about the way we process information in our minds.

At one time words were icons. It took about 25,000 years or 1,250 generations for words to change from iconic to symbolic signs. Duncan Davies with Diana and Robin Bathurst in their book The Telling Image write that human communication in one form or another has occurred for about 30,000 years. For 25,000 of those years, the chief mode of written communication was paintings on the walls of caves. Most of the preserved drawings show animals of various sizes and types. As with any icon, the connection between the picture and the object it stands for is clear.

About 5,000 years ago the Sumerians in what is now the Middle East assigned meanings to more simplistic versions of the cave drawings. The Sumerians used about 2,000 different images called pictograms (or pictographs) and ideograms (or ideographs or logographs). Pictograms are icons that resemble objects while ideograms express abstract ideas. The name for the Sumerian language, cuneiform, came from the shapes made in clay tablets by a stylus-type writing tool. Cuneiform is Latin for wedge-shaped. At about this same time, the Egyptians had their hieroglyphics printed on papyrus while the Chinese and Japanese civilizations were simplifying their written languages as well.

About 3,000 years ago, the Phoenicians further simplified the list of pictograms through an ingenious breakthrough. Instead of drawings related to every possible object and idea in the world, they spread the new idea that pictures could symbolize the sounds made when they spoke. This invention was the birth of the phonetic alphabet. For the next 15 centuries, except in China and Japan where calligraphers still used an object-to-picture based language, alphabets evolved into present-day forms throughout the world.

About 500 years ago Johannes Gutenberg invented the moveable type printing press. His machine destroyed the monopoly over writing that was enjoyed by the kings and religious leaders of the day. Reading and writing became activities that anyone, not just a chosen few, could use. Fifty years after Gutenberg's press was introduced, 12 million books were printed in Europe alone. Printing aided literacy not only by the books produced that made persons want to read, but also by standardizing the shape of letters. Readability improved as the individual styles of handwritten characters, the pictorial elements of writing, were replaced by the mechanical sameness of the printed letters. Gutenberg's invention marked the decline in visual literacy. Words achieved dominance over pictures. But as visual literacy improves, typeface fonts become more individualistic and visual as in their early beginnings.

From this brief introduction to the history of words it is clear that the oldest form of communication is by pictures. But when it became clear that pictures were awkward for keeping long, complicated messages, the drawings became simplified so that they could be more portable. Once linked to the sounds of speech, the pictures lost forever their visual links to their direct image roots.

Nevertheless, words maintain their pictorial link with their past through the size, style, color, use of italics, boldface, reverse, etc. variations. Meaning not only comes from their symbolism as words, but also from their symbolism as pictures.

Suzanne Langer writes that the reason words have such a hold for literate societies is that they have such a "long history of understood meaning." Words are economical and easily combined symbols that have no other value except as symbols. If you become interested in the writing of the word itself, as in the case of creative typographic arrangements, the word loses some of its meaning. Computer technology, however, makes it possible for anyone with a desktop workstation to manipulate typeface fonts and to even create their own designs. Newspaper and magazine publishers hire graphic artists who are trained to turn words into visual statements that have a symbolic meaning apart from that of the words themselves. Canadian philosopher Marshall McLuhan once said that the medium is the message. The new age in communication technology makes obvious what has been known about images since they were first created. For pictures and now for words it is clear that the medium and the message form one symbolic statement. The medium -- color, form, depth and movement -- are characteristics that help define the message. Roland Barthes asserts that "lines satisfy the eyes while symbols satisfy the mind." But when the lines, shapes and colors have meaning, the mind is satisfied by both. Abstract painters were the first to recognize the link between lines and symbols. The eye and the mind are one. Understanding symbols is what separates humans from other animals. It is the fundamental necessity for rational thought. For Langer, "dogs scorn our paintings because they see colored canvases, not pictures."

The second tenet of the syntactic theory of visual communication is:

As symbols with similar historical roots,
mediated words and pictures are both symbolic representations.

Before Helen Keller, who was blind and deaf learned to connect words with the objects she touched in her environment, her mind was filled with non-visual, non-verbal emotions. Once she learned the name for water, that wet substance that made her feel good, the symbol and the object were forever linked. To feel good again, she did not need to actually drink water. Now she only needed to think of its name. The connotation remains in the mind through the word symbol without help from the denotation. The symbol and the object of its meaning become fused in the mind. Denotation and connotation become one.

Langer writes that "All genuine thinking is symbolic, and the limits of the expressive medium are, therefore, really the limits of our conceptual powers. Symbols in order to be thought must be verbalized." She was not referring to spoken verbalizations, but mental thoughts in the form of words. The perceptual elements of color, form, depth, and movement make it possible to turn signs within direct or mediated images into remembered thoughts. A perception has meaning only when the signs are recognizable and can be related, through memory, to personal and cultural experiences.

It is well known that previously viewed perceptions can be recalled through verbal stimuli. You can easily see for yourself how symbols are used to retain events and ideas in your memory. Next time you read a newspaper, wait a few hours and try to remember the stories that were printed on the pages. You probably will not recall many. Ask a friend, with the newspaper in hand, to give you clues from each story or to show you a photograph, if one accompanied the article. Through this recall method, you will prove to yourself that the facts of the day's events are still within your mind. Whether you remember the words and pictures in the newspaper over a much longer period of time depends on how often you recall them mentally. Dreams quickly fade from memory because they are purely visual experiences that have not been translated into a verbal format. If you want to remember a particularly moving or significant dream, psychologists advise that you immediately tell the dream to someone or write the content of the dream down on a sheet of paper next to your bed. Only by verbal reinforcement of the dream's content will it become a part of your long-term memory.

Psychologists have found that concrete nouns are much more effectively remembered and an aid to recall than abstract ones. The concrete nouns of ball, book, bottle, baby, etc. are easier to visualize in the mind because of their link to real objects. Abstract concepts of freedom, peace, ethics, love, etc. are harder to link to a single image. But in 1836 for his book Nature, Ralph Waldo Emerson noted that the roots of all words -- even abstract concepts -- are concrete in nature. "Every word which is used to express a moral or intellectual fact," he wrote, "if traced to its roots, is found to be borrowed from some material appearance." As the Greek poet Simonides wrote, "Words are the images of things." Giving names to objects enhances the memory of those objects.

When you carefully analyze a visual message, you consciously study each visual symbol within that picture's frame. The act of concentration is a verbal exercise. Without verbal translations of the signs within an image, there is little chance of it being recalled in the future. The picture is lost from your memory because you have learned nothing from it. Images become real property of the mind and remembered only when language expresses them. Linguistic experts do not need to argue that images have no alphabet or syntax because such assertions are true. The alphabet and the syntax of images reside in the mind, not in the picture itself.

Wilson Hicks calls the mind of the readers who look at published words and pictures as the "X factor." It is the mind that evokes meaning through memory and imagination. Others have termed this factor "the third effect." Mental activity frames the words and pictures viewed through direct or mediated images in a new context that facilitates memory and creative thought. Even cultures that have no written language or persons who cannot read or write think in alternative symbol systems in order to remember what they have experienced. Their thoughts still maintain the verbal characteristics of the spoken language. When there is no oral language as in the case of Helen Keller or young children, the symbolic form of spoken words is replaced by the symbolic code of raw emotions.

Societies with only a spoken language had better recall of objects and events because their mental symbol systems had to be highly developed. Writing and the literacy that spread throughout the world decreased the need for memory because once an event is recorded on paper, it does not have to be remembered exactly as it originally happened. Consequently, the need for a complex system of signs within the mind has decreased over time as words have dominated communication. As images become even more prevalent, memory may once again reach the level of 'primitive' cultures.

Signs have no meaning outside of their context. Visual and verbal thoughts combine to create the context that links signs together to form symbols that can be remembered and recalled. Context is the glue that binds visual and verbal symbols together. Recognizing context is the chief function of a rational mind. The face of someone you know can be immediately recalled as imagery of the mind. But that face is not a literal, detailed, photographic image. The mind's picture is a combination of the perceptual elements -- color, form, depth and movement -- that are needed to describe appearance combined with the verbal thoughts that define the context in which the person appears.

Memorable images, either directly experienced or seen through a mediated format, are those that you think about. They are usually simple compositions with immediate impact. They are images that trigger the emotional and rational aspects of your mind's personality. They are pictures you recall again and again long after the original object of perception has faded from your retinas.

The third tenet of the syntactic theory of visual communication is:

Images are remembered by thinking about them in words.

Whether pictures are not a language because there is no easily definable and reproducible alphabet or because the elements that make up a picture do not follow a discursive, linear flow, most experts agree that images are a collection of signs and as such, become a language when read in the mind. When words and images have equal status within all media of communication, the cultural cues that define a society will not only be more efficiently passed from one generation to the next, but within this generation, here and now, diverse cultures will be able to understand each other a little better.


Sources

Altschul, Charles. (1992). "The Center for Creative Imaging and the Influence of Technology on Creativity," in Ethics, Copyright, and the Bottom Line. Camden: Center for Creative Imaging, pp. 59-61.

Arnheim, Rudolf. (1974). Art and Visual Perception. Berkeley: University of California Press.

Barnhurst, Kevin. (December, 1991). "News as Art," Journalism Monographs.

Barthes, Roland. (1981). Camera Lucida. New York: Hill and Wang.

-----. (1977). Image Music Text. New York: Hill and Wang.

Berger, Arthur A. (1989). Seeing is Believing An Introduction to Visual Communication. Mountain View: Mayfield Publishing Company.

Berger, John and Jean Mohr. (1982). Another Way of Telling. New York: Pantheon Books.

-----. (1977). Ways of Seeing. London: Penguin Books.

Biederman, Irving. (1987). "Recognition-by-Components: A Theory of Human Image Understanding." Psychological Review, Vol. 94, no. 2, pp. 115-147.

Bloomer, Carolyn M. (1990). Principles of Visual Perception. New York: Design Press.

Barnhurst, Kevin. (December, 1991). "News as Art," Journalism Monographs.

Berger, John. (1977). Ways of Seeing. London: Penguin Books.

Bloomer, Carolyn M. (1990). Principles of Visual Perception. New York: Design Press.

Bolton, Richard (ed.). (1992). The Contest of Meaning Critical Histories of Photography. Cambridge: The MIT Press.

Bruce, Vicki and Green, Patrick. (1985). Visual Perception Physiology, Psychology and Ecology. London: Lawrence Erlbaum Associates, Publishers.

Bryson, Norman, Holly, Michael Ann & Moxey, Keith, eds. (1991). Visual Theory Painting & Interpretation. New York: Harper Collins.

Conover, Theodore E. (1985). Graphic Communications Today. St. Paul: West Publishing Company.

Davies, Duncan, Bathurst, Diana & Bathurst, Robin. (1990). The Telling Image The Changing Balance between Pictures and Words in a Technological Age. Oxford: Clarendon Press.

Deely, John. (1990). Basics of Semiotics. Bloomington: Indiana University Press.

Dondis, Donis A. (1973). A Primer of Visual Literacy. Cambridge: The MIT Press.

Fineman, Mark. (1981). The Inquisitive Eye. New York: Oxford University Press.

Fiske, Susan T. and Taylor, Shelley E. (1984). Social Cognition. Menlo Park: Addison-Wesley Publishing Company.

Foss, Sonja K. (1992). "Visual Imagery as Communication." Text and Performance Quarterly, Vol. 12, pp. 85-96.

Friedhoff, Richard Mark & Benzon, William. (1988). The Second Computer Revolution Visualization. New York: W. H. Freeman and Company.

Fulton, Marianne. (1988). Eyes of Time Photojournalism in America. New York: New York Graphic Society.

Gardner, Howard. (1982). Art, Mind, and Brain A Cognitive Approach to Creativity. New York: Basic Books, Inc. Publishers.

Ghiselin, Brewster. (1952). The Creative Process. New York: The New American Library, Inc.

Gibson, James J. (1979). The Ecological Approach to Visual Perception. Boston: Houghton Mifflin Company.

Goldberg, Vicki. (1991). The Power of Photography. New York: Abbeville Press.

Gregory, R.L. (1970). The Intelligent Eye. New York: McGraw-Hill Book Company.

Hardt, Hanno. (1991, April). "Words and images in the age of technology." Media Development. Vol. 38, pp. 3-5.

Hatcher, Evelyn P. (1974). Visual Metaphors A Methodological Study in Visual Communication. Albuquerque: University of New Mexico Press.

Hicks, Wilson. (1973). Words and Pictures. New York: Arno Press.

Hochberg, Julian. "Attention, Organization, and Consciousness, " in Mostofsky, David. (ed.) (1970). Attention: Contemporary Theory and Analysis. New York: Appleton-Century-Crofts.

Hoffman, Howard S. (1989). Vision & the Art of Drawing. Englewood Cliffs: Prentice Hall.

Hoistad, Gunnar. (1991, April). "How vulnerable are children to electronic images?" Media Development. Vol. 38, pp. 9-11.

Holmes, Nigel. (1985). Designing Pictorial Symbols. New York: Watson-Guptill Publications.

Hunter, Jefferson. (1987). Image and Word The Interaction of Twentieth-Century Photographs and Texts. Cambridge: Harvard University Press.

Jean, Georges. (1992). Writing The Story of Alphabets and Scripts. New York: Harry N. Abrams, Inc., Publishers.

Kepes, Gyorgy, ed. (1966). Sign Image Symbol. New York: George Braziller.

Kling, J.W. and Riggs, Lorrin A. (eds.) (1971). Experimental Psychology. New York: Holt, Rinehart and Winston, Inc.

Koole, Wim. (1991, April). "Imagination depends on images." Media Development. Vol. 38, pp. 16-17.

Langer, Suzanne K. (1960). Philosophy in a New Key. Cambridge: Harvard University Press.

-----. (1953). Feeling and Form A Theory of Art. New York: Charles Scribner's Sons.

Lester, Paul Martin. (2006) Visual Communication Images with Messages 4th Edition. Belmont, CA: Wadsowrth Publishing Company.

Lodge, David. (1984). Small World. New York: Warner Books, Inc.

Margolin, Victor, (ed.). (1989). Design Discourse History, Theory, Criticism. Chicago: The University of Chicago Press.

McCafferty, James D. (1990). Human and Machine Vision Computing Perceptual Organisation. New York: Ellis Horwood Limited.

Morgan, John and Welton, Peter. (1992). See What I Mean? London: Edward Arnold.

Paivio, Allan. (1971). Imagery and Verbal Processes. New York: Holt, Rinehart & Winston.

Parker, D.M. & Deregowski, J.B. (1990). Perception and Artistic Style. Amsterdam: North-Holland.

Phelan, John M. (1991, April). "Image industry erodes political space." Media Development. Vol. 38, pp. 6-8.

Richards, Stan. (1974). Hobo Signs. New York: Barlenmir House, Publishers.

Ritchin, Fred. (1992). "An Image-Based Society," in Ethics, Copyright, and the Bottom Line. Camden: Center for Creative Imaging, pp. 19-35.

Saint-Martin, Fernande. (1990). Semiotics of Visual Language. Bloomington: Indiana University Press.

Sculley, John. (1992). "Computers, Communications and Content," in Ethics, Copyright, and the Bottom Line. Camden: Center for Creative Imaging, pp. 15-21.

Sebeok, Thomas A. (1991). Semiotics in the United States. Bloomington: Indiana University Press.

Shepard, Roger N. (1990). Mind Sights Original Visual Illusions, Ambiguities, and Other Anomalies. New York: W.H. Freeman and Company.

Sontag, Susan. (1978). On Photography. New York: Farrar, Straus and Giroux.

Stoops, Jack & Jerry Samuelson. (1983). Design Dialogue. Worcester: Davis Publications, Inc.

Stroebel, Leslie, Todd, Hollis & Zakia, Richard. (1980). Visual Concepts for Photographers. New York: Focal Press Inc.

Thiel, Philip. (1981). Visual Awareness and Design. Seattle: University of Washington Press.

Tufte, Edward. (1983). The Visual Display of Quantitative Information. Cheshire: Graphics Press.

Wade, Nicholas. (1990). Visual Allusions Pictures of Perception. London: Lawrence Erlbaum Associates, Publishers.

Worth, Sol. (1981). Studying Visual Communication. Philadelphia: University of Pennsylvania Press.