Psychon Bull Rev (2017) 24:652–665 DOI 10.3758/s13423-016-1145-z

Gesture as representational action: A paper about function1

THEORETICAL REVIEW

Gesture as representational action: A paper about function

Miriam A. Novack1 & Susan Goldin-Meadow1

Published online: 7 September 2016 # Psychonomic Society, Inc. 2016

Abstract A great deal of attention has recently been paid to gesture and its effects on thinking and learning. It is well established that the hand movements that accompany speech are an integral part of communication, ubiquitous across cul- tures, and a unique feature of human behavior. In an attempt to understand this intriguing phenomenon, researchers have fo- cused on pinpointing the mechanisms that underlie gesture production. One proposal––that gesture arises from simulated action (Hostetter & Alibali Psychonomic Bulletin & Review, 15, 495–514, 2008)––has opened up discussions about action, gesture, and the relation between the two. However, there is another side to understanding a phenomenon and that is to understand its function. A phenomenon’s function is its purpose rather than its precipitating cause––the why rather than the how. This paper sets forth a theoretical framework for exploring why gesture serves the functions that it does, and reviews where the current literature fits, and fails to fit, this proposal. Our framework proposes that whether or not gesture is simulated action in terms of its mechanism––it is clearly not reducible to action in terms of its function. Most notably, because gestures are abstracted representations and are not actions tied to particular events and objects, they can play a powerful role in thinking and learning beyond the par- ticular, specifically, in supporting generalization and transfer of knowledge.

Keywords Gesture . Action . Learning . Representations

* Miriam A. Novack [email protected]

Department of Psychology, University of Chicago, Chicago, IL 60637, USA

Gestures are spontaneous hand movements that accompany speech (Goldin-Meadow & Brentari, in press; Kendon, 2004; McNeill, 1992). They have the capacity to portray ac- tions or objects through their form (iconic gestures), to repre- sent abstract ideas (metaphoric gestures), to provide emphasis to discourse structure (beat gestures), and to reference loca- tions, items, or people in the world (deictic gestures). Children gesture before they can speak (Bates, 1976; Goldin-Meadow, 2014) and people all over the world have been found to ges- ture in one way or another (Kita, 2009). Gestures provide a spatial or imagistic complement to spoken language and are not limited to conventions and rules of formal linear-linguistic systems. Importantly, gestures play a unique role in commu- nication, thinking, and learning and have been shown to affect the minds of both the people who see them and the people who produce them (Goldin-Meadow, 2003).

There are many questions that arise when we think about gesture: What makes us gesture? What types of events make gesture likely? What controls how often we gesture? These sorts of questions are all focused on the mechanism of gesture production––an important line of inquiry exploring the struc- tures and processes that underlie how gesture is produced. Rather than ask about the mechanisms that lead to gesture, we focused on the consequences of having produced ges- ture––that is, on the function of gesture. What effects do ges- tures have on the listeners who see them and the speakers who produce them? What features of gestures contribute to these effects? How do these features and functions inform our un- derstanding of what exactly gestures are?

We propose that gestures produce effects on thinking and learning, because they are representational actions. When we say that gestures are representational actions, we mean that they are meaningful substitutions and analogical stand-ins for ideas, objects, actions, relations, etc. This use of the term representational should not be confused with the term

1

Gesture as representational action: A paper about function2 653 Psychon Bull Rev (2017) 24:652–665

representational gesture––a category of gestures that look like the ideas and items to which they refer (i.e., iconic and meta- phoric gestures). Our proposal that gestures are representa- tional is meant to apply to all types of nonconventional ges- tures, including representational gestures (iconics, meta- phorics), deictic gestures, and even beat gestures. Iconic ges- tures can represent actions or objects; deictic gestures draw attention to the entities to which they refer; beat gestures re- flect discourse structure. Most of this paper explores the func- tions of iconic and deictic gestures, but we believe that our framework can be applied to all (non-conventional) gestures.

Gestures are representational in that they represent some- thing other than themselves, and they are actions in that they involve movements of the body. Most importantly, the fact that gestures are representational actions differentiates them from full-blown instrumental actions, whose purpose is to affect the world by directly interacting with it (e.g., grabbing a fork, opening a canister). In addition, gestures are unlike movements for their own sake (Schachner & Carey, 2013), whose purpose is the movement itself (e.g., dancing, exercis- ing). Rather, gestures are movements whose power resides in their ability to represent actions, objects, or ideas.

Gestures have many similarities to actions simply, because they are a type of action. Theories rooted in embodied cogni- tion maintain that action experiences have profound effects on how we view objects (James & Swain 2011), perceive other’s actions (Casile & Giese, 2006), and even understand language (Beilock, Lyons, Mattarella-Micke, Nusbaum, & Small, 2008). The Gesture as Simulated Action (GSA) framework grew out of the embodied cognition literature. The GSA pro- poses that gestures are the manifestation of action programs, which are simulated (but not actually carried out) when an action is imagined (Hostetter & Alibali, 2008). Following at least some accounts of embodied cognition (see Wilson, 2002, for review), the GSA suggests that when we think of an action (or an object that can be acted upon), we activate components of the motor network responsible for carrying out that action, in essence, simulating the action. If this simulation surpasses the Bgesture threshold,^ it will spill over and become a true motor expression––an overt gesture. The root of gesture, then, according to this framework, is simulation––partial motor ac- tivation without completion.

The GSA framework offers a useful explanation of how gesturing comes about (its mechanism) and the framework highlights gesture’s tight tie to action. However, this frame- work is primarily useful for understanding how gestures are produced, not for how they are understood, unless we assume that gesture comprehension (like language comprehension; Beilock et al., 2008) also involves simulating action. More importantly, the framework does not necessarily help us un- derstand what gestures do both for the people who produce them and for the people who see them. We suggest that view- ing gestures as simulated actions places too much emphasis on

the action side of gesture and, in so doing, fails to explain the ways in which gesture’s functions differ from those of instru- mental actions. The fact that gesture is an action is only one piece of the puzzle. Gesture is a special kind of action, one that represents the world rather than directly impacting the world. For example, producing a twisting gesture in the air near, but not on, a jar will not open the jar; only performing the twisting action on the jar itself will do that. We argue that this repre- sentational characteristic of gesture is key to understanding why gesturing occurs (its function).

Our hypothesis is that the effects gesture has on thinking and learning grow not only out of the fact that gesture is itself an action, but also out of the fact that gesture is abstracted away from action––the fact that it is representational. Importantly, we argue that this framework can account for the functions gesture serves both for producers of gesture and for perceivers of gesture. We begin by defining what we mean by gesture, and providing evidence that adults sponta- neously view gesture-like movements as representational. Second, we review how gesture develops over ontogeny, and use evidence from developmental populations to suggest a need to move from thinking about gesture as simulated ac- tion to thinking about it as representational action. Finally, we review evidence that gesture can have an impact on cognitive processes, and explore this idea separately for producers of gesture and for receivers of gesture. We show that the effects that gesture has on both producers and receivers are distinct from the effects that instrumental action has. In each of these sections, our goal is to develop a framework for understanding gesture’s functions, thereby creating a more comprehensive account of cause in the phenomenon of gesture.

Part 1: What makes a movement a gesture?

Before we can unpack how gesture’s functions relate to its classification as representational action, we must establish how people distinguish gestures from the myriad of hand movements they encounter. Gestures have a few obvious fea- tures that differentiate them from other types of movements. The most obvious is that gestures happen off objects, in the air. This feature makes gestures qualitatively different from object-directed actions (e.g., grabbing a cup of coffee, typing on a keyboard, stirring a pot of soup), which involve manip- ulating objects and causing changes to the external world. A long-standing body of research has established that adults (as well as children and infants) process object-directed move- ments in a top-down, hierarchical manner, encoding the goal of an object-directed action as most important and ignoring the particular movements used to achieve that goal (Baldwin & Baird, 2001; Bower & Rinck, 1999; Searle, 1980; Trabasso & Nickels, 1992; Woodward, 1998; Zacks, Tversky, & Iyer, 2001). For example, the goal of twisting the lid of a jar is to

654 Psychon Bull Rev (2017) 24:652–665 Gesture as representational action: A paper about function3

open the jar––not just to twist one’s hand back and forth while holding onto the jar lid.

In contrast to actions that are produced to achieve external goals, if we interpret the goal of an action to be the movement itself, we are inclined to describe that movement in detail, focusing on its low-level features. According to Schachner and Carey (2013), adults consider the goal of an action to be the movement itself if the movement is irrational (e.g., moving toward an object and then away from it without explanation) or if it is produced in the absence of objects (e.g., making the same to-and-fro movements but without any objects present). These Bmovements for the sake of movement^ can include dancing, producing ritualized movements, or exercising. For example, the goal of twisting one’s hands back and forth in the air when no jar is present might be to just stretch or to exercise the wrist and fingers.

So where does gesture fit in? Gestures look like movements for their own sake in that they occur off objects and, in this sense, resemble dance, ritual, and exercise. However, gestures are also similar to object-directed actions in that the move- ments that comprise a gesture are not the purpose of the ges- ture––those movements are a means to accomplish something else––communicating and representing information. Gestures also differ from object-directed actions, however, in their pur- pose––the purpose of an object-directed action is to accom- plish a goal with the object (e.g., to open a jar, grab a cup of coffee); the purpose of a gesture is to represent information and perhaps communicate that information (e.g., to show someone how to open a jar, to tell someone that you want that cup of coffee). The question then is––how is an observer to know when a movement is a communicative symbol (i.e., a gesture) and when it is an object-directed action or a move- ment produced for its own sake?

To better understand how people know when they have seen a gesture, we asked adults to describe scenes in which a woman moved her hands under three conditions (Novack, Wakefield, & Goldin-Meadow, 2016). In the first condition (action on objects), the woman moved two blue balls into a blue box and two orange balls into an orange box. In the second condition (action off objects with the objects present), the balls and boxes were present, but the woman moved her hands as if moving the objects without actually touching them. Finally, in the third condition (action with the objects absent), the woman moved her hands as if moving the objects, but in the absence any objects.

In addition to the presence or absence of objects, another feature that differentiates object-directed actions from gestures is co-occurrence with speech. Although actions can be pro- duced along with speech, they need not be. In contrast, ges- tures not only routinely co-occur with speech, but they are also synchronized with that speech (Kendon, 1980; McNeill, 1992). People do, at times, spontaneously produce gesture without speech and, in fact, experimenters have begun to

instruct participants to describe events using their hands and no speech (Gibson, Piantadosi, Brink, Bergen, Lim & Saxe, 2013; Goldin-Meadow, So, Özyürek, & Mylander, 2008; Hall, Ferreira & Mayberry, 2013). However, these silent ges- tures, as they are known, look qualitatively different from the co-speech gestures that speakers produce as they talk (Goldin- Meadow, McNeill & Singleton, 1996; Özçalışkan, Lucero & Goldin-Meadow, 2016; see Goldin-Meadow & Brentari, in press, for discussion). To explore this central feature of ges- ture, Novack et al. (2016) also varied whether the actor’s movements in their study were accompanied by filtered speech. Movements accompanied by speech-like sounds should be more likely to be seen as a gesture (i.e., as a repre- sentational action) than the same movements produced with- out speech-like sounds.

Participants’ descriptions of the event in the video were coded according to whether they described external goals (e.g., Bthe person placed balls in boxes^), movement-based goals (e.g., Ba woman waved her hands over some balls and boxes^), or representational goals (i.e., Bshe showed how to sort objects^). As expected, all participants described the videos in which the actor moved the objects as depicting an external-goal, whereas participants never gave this type of response for the empty-handed videos (i.e., videos in which the actor did not touch the objects). However, participants gave different types of responses as a function of the presence or absence of the objects in the empty-handed movement con- ditions. When the objects were there (but not touched), ap- proximately 70 % of observers described the movements in terms of representational goals. In contrast, when the objects were not there (and obviously not touched), only 30 % of ob- servers mentioned representational goals. Participants increased the number of representational goals they gave when the actor’s movements were accompanied by filtered speech (which made the movement feel like part of a communicative act).

Observers thus systematically described movements that have many of the features of gesture––no direct contact with objects, and co-occurrence with speech––as representational actions. Importantly, participants made a clear distinction be- tween the instrumental object-directed action, and the two empty-handed movements (movements in the presence of ob- jects and movements in the absence of objects), indicating that actions on objects have clear external goals, and actions off objects do not. Empty-handed movements are often interpreted as movements for their own sake. But if the conditions are right, observers go beyond the movements they see to make rich inferences about what those movements can represent.

Part 2: Learning from gestures over development

We now know that, under the right conditions, adults will view empty-handed movements as more than just movements

Gesture as representational action: A paper about function4 655 Psychon Bull Rev (2017) 24:652–665

for their own sake. We are perfectly positioned to ask how the ability to see movement as representational action develops over ontogeny. In this section, we look at both the production and comprehension of gesture in the early years, focusing on the development of two types of gestures––deictic gestures and iconic gestures.

Development of deictic gestures

We begin with deictic gestures, because these are the first gestures that children produce and understand. Although deic- tic gestures have a physically simple form (an outstretched arm and an index finger), their meaning is quite rich, representing social, communicative, and referential intentions (Tomasello, Carpenter & Liszkowski, 2007). Interestingly, deictic gestures are more difficult to produce and understand than their simple form would lead us to expect.

Producing deictic gestures Infants begin to point between 9 and 12 months, even before they say their first words (Bates, 1976). Importantly, producing these first gesture forms signals advances in children’s cognitive processes, particularly with respect to their language production. For example, lexical items for objects to which a child points are soon found in that child’s verbal repertoire (Iverson & Goldin-Meadow, 2005). Similarly, pointing to one item (e.g., a chair) while producing a word for a different object (e.g., Bmommy^) predicts the onset of two-word utterances (e.g., Bmommy’s chair^) (Goldin-Meadow & Butcher, 2003; Iverson & Goldin- Meadow, 2005). Not only does the act of pointing preview the onset of a child’s linguistic skills, but it also plays a causal role in the development of those skills. One and a half-year- old children given pointing training (i.e., they were told to point to pictures of objects as the experimenter named them) increased their own pointing in spontaneous interactions with their caregivers, which led to increases in their spoken vocab- ulary (LeBarton, Goldin-Meadow & Raudenbush, 2015). Finally, these language-learning effects are unique to pointing gestures, and do not arise in response to similar-looking in- strumental actions like reaches. Eighteen-month-old children learn a novel label for an object if an experimenter says the label while the child is pointing at the object but not if the child is reaching to the object (Lucca & Wilborn, 2016). Thus, as early as 18 months, we see that the representational status of the pointing gesture can have a unique effect on learning (i.e., language learning), an effect not found for a comparable in- strumental act.

Perceiving deictic gestures Children begin to understand other’s pointing gestures around the same age as they them- selves begin to point. At 12 months, infants view points as goal-directed (Woodward & Guajardo, 2002) and recognize the communicative function of points (Behne, Liszkowski,

Carpenter, & Tomasello, 2012). Infants even understand that pointing hands, but not nonpointing fists, communicate infor- mation to those who can see them (Krehm, Onishi & Vouloumanos, 2014). As is the case for producing pointing gestures, seeing pointing gestures results in effects that are not found for similar-looking instrumental actions. For example, Yoon, Johnson, and Csibra (2008) found that when 9-month- old children see someone point to an object, they are likely to remember the identity of that object. In contrast, if they see someone reach to an object (an instrumental act), 9-month- olds are likely to remember the location of the object, not its identity. Thus, as soon as children begin to understand pointing gestures, they seem to understand them as represen- tational actions, rather than as instrumental actions.

Development of iconic gestures

Young children find it difficult to interpret iconic gestures, which, we argue, is an outgrowth of the general difficulty they have with interpreting representational forms (DeLoache, 1995). Interestingly, even though instrumental actions often look like iconic gestures, interpreting instrumental actions does not present the same challenges as interpreting gesture.

Producing iconic gestures Producing iconic gestures is rare in the first years of life. Although infants do produce a few iconic gestures as early as 14 months (Acredolo & Goodwyn, 1985, 1988), these early gestures typically grow out of parent- child play routines (e.g., while singing the itsy-bitsy spider), suggesting that they are probably not child-driven representa- tional inventions. It is not until 26 months that children begin to reliably produce iconic gestures in spontaneous settings (Özçalışkan & Goldin-Meadow, 2011) and in elicited labora- tory experiments (Behne, Carpenter & Tomasello, 2014) and, even then, these iconic forms are extremely rare. Of the ges- tures that young children produce, only 1-5 % are iconic (Iverson, Capirci & Caselli, 1994; Nicoladis, Mayberry & Genesee, 1999; Özçalışkan & Goldin-Meadow, 2005). In con- trast, 30 % of the gestures that adults produce are iconic (McNeill, 1992).

If gestures are simply a spillover from motor simulation (as the GSA predicts), we might expect children to begin produc- ing a gesture for a given action as soon as they acquire the underlying action program for that action (e.g., we would expect a child to produce a gesture for eating as soon as the child is able to eat by herself). But children produce actions on objects well before they produce gestures for those actions (Özçalışkan & Goldin-Meadow, 2011). In addition, according to the GSA, gesture is produced when an inhibitory threshold is exceeded. Because young children have difficulty with in- hibitory control, we might expect them to produce more ges- tures than adults, which turns out not to be the case (Özçalışkan & Goldin-Meadow, 2011). The relatively late

656 Psychon Bull Rev (2017) 24:652–665 Gesture as representational action: A paper about function5

onset and paucity of iconic gesture production is thus not predicted by the GSA. It is, however, consistent with the pro- posal that gestures are representational actions. As represen- tational actions, gestures require sophisticated processing skills to produce and thus would not be expected in very young children.

Perceiving iconic gestures Understanding iconic gestures is also difficult for toddlers. At 18 months, children are no more likely to associate an iconic gesture (e.g., hopping two fingers up and down to represent the rabbit’s ears as it hops) or an arbitrary gesture (holding a hand shaped in an arbitrary con- figuration to represent a rabbit) with an object (Namy, Campbell, & Tomasello, 2004). It is not until the middle of the second year that children begin to appreciate the relation between an iconic gesture and its referent (Goodrich & Hudson Kam, 2009; Marentette & Nicoladis, 2011; Namy, Campbell, & Tomasello, 2004; Namy, 2008; Novack, Goldin-Meadow, & Woodward, 2015). In many cases, chil- dren fail to correctly see the link between an iconic gesture and its referent until age 3 or even 4 years (e.g., when gestures represent the perceptual properties of an object; Hodges, Özçalışkan, & Williamson, 2015; Tolar, Lederberg, Gokhale, & Tomasello, 2008).

The relatively late onset of children’s comprehension of iconic gestures is also consistent with the proposal that ges- tures are representational actions. If gestures were simulations of actions, then as soon as an infant has a motor experience, the infant ought to be able to interpret that motor action as a gesture just by accessing her own motor experiences. But young children who are able to understand an instrumental action are not necessarily able to understand a gesture for that action. Consider, for example, a 2-year-old who is motorically capable of putting a ring on a post. If an adult models the ring- putting-on action for the child, she responds by putting the ring on the post (in fact, children put the ring on the post even if the adult tries to get the ring on the post but doesn’t succeed, i.e., if the adult models a failed attempt). If, however, the adult models a put-ring-on-post gesture (she shows how the ring can be put on the post without touching it), the 2-year-old frequently fails to place the ring on the post (Novack et al., 2015). In other words, at a time when a child understands the goal of an object-directed action and is able to perform the action, the child is still unable to understand a gesture for that action. This difficulty makes sense on the assumption that gestures are representational actions since children of this age are generally known to have difficulty with representation (DeLoache, 1995).

As another example, young children who can draw infer- ences from a hand that is used as an instrumental action (e.g., an object-directed reach) fail to draw inferences from the same hand used as a gesture. Studies of action processing find that infants as young as 6-months can use the shape of someone’s

reaching hand to correctly predict the intended object of the reach (Ambrosini et al, 2013; Filippi & Woodward, 2016). For example, infants expect someone whose hand is shaped in a pincer grip to reach toward a small object, and someone whose hand is shaped in a more open grip to reach toward a large object (Ambrosini et al, 2013)––but they do so only when the handshape is embedded in an instrumental reach. Two-and-a- half-year-olds presented with the identical hand formations as gestures rather than reaches (i.e., an experimenter holding a pincer handshape or open handshape in gesture space) are unable to map the hand cue onto its referent (Novack, Filippi, Goldin-Meadow & Woodward, 2016). The fact that children can interpret handshape information accurately in instrumental actions by 6 months, but are unable to interpret handshape information in gesturing actions until 2 or 3 years, adds weight to the proposal that gestures are a special type of representational action.

Part 3: Gesture’s functions are supported by its action properties and its representational properties

Thus far, we have discussed how people come to see move- ments as gestures and have used findings from the develop- mental literature to raise questions about whether gesture is best classified as simulated action. We suggest that, even if gesture arises from simulated action programs, to understand fully its effects, we also need to think about gesture as repre- sentational action. Under this account, simulated actions are considered nonrepresentational, and it is the difference be- tween representational gesture and veridical action that is key to understanding the effects that gesture has on producers and perceivers. In this section, we examine similarities and differences between gesture and action and discuss the impli- cations of these similarities and differences for communica- tion, problem solving, and learning.

Gesture versus action in communication

As previously mentioned, one way in which gestures differ from actions is in how they relate to spoken language. Unlike object-directed actions, gestures are seamlessly integrated with speech in both production (Bernardis & Gentilucci, 2006; Kendon, 1980; Kita & Özyürek, 2003) and comprehen- sion (Kelly, Ozyurek, & Maris, 2010), supporting the claim that speech and gesture form a single integrated system (McNeill, 1992). Indeed, the talk that accompanies gesture plays a role in determining the meaning taken from that ges- ture. For example, a spiraling gesture might refer to ascending a staircase when accompanied by the sentence, BI ran all the way up,^ but to out-of-control prices when accompanied by the sentence, BThe rates are rising every day.^ Conversely, the gestures that accompany speech can influence the meaning

Gesture as representational action: A paper about function6 657 Psychon Bull Rev (2017) 24:652–665

taken from speech. For example, the sentence, BI ran all the way up,^ is likely to describe mounting a spiral staircase when accompanied by an upward spiraling gesture, but a straight staircase when accompanied by an upward moving point. We discuss the effects of gesture-speech integration for the speakers who produce gesture, as well as the listeners who perceive it.

Producing gesture in communication Gesture production is spontaneous and temporally linked to speech (Loehr, 2007; McNeill, 1992). Moreover, the tight temporal relation found between speech and gesture is not found between speech and instrumental action. For example, if adults are asked to explain how to throw a dart using the object in front of them (an instrumental action) or using just their hands with no object (a gesture), they display a tighter link between speech and the accompanying dart-throwing gesture than between speech and the accompanying dart-throwing action (Church, Kelly, & Holcombe 2014). Other signatures of the gesture-speech system also seem to be unique to gesture, and are not found in instrumental actions. For example, gestures are more often produced with the right hand (suggesting a link to the left- hemisphere speech system), whereas self-touching adaptors (e.g., scratching, pushing back the hair), which are instrumen- tal actions, are produced with both hands (Kimura, 1973).

The act of producing representational gesture along with speech has been found to have an effect on speakers them- selves. Gesturing while speaking can improve the speaker’s lexical access and fluency (Graham & Heywood, 1975; Rauscher, Krauss, & Chen, 1996), help the speaker package information (Kita, 2000), and even lighten the speaker’s work- ing memory load (Goldin-Meadow, Nusbaum, Kelly, & Wagner, 2001; Wagner, Nusbaum, & Goldin-Meadow, 2004). Moreover, movements that are not gestures, such as meaningless hand movements, do not have the same load- lightening effects on the speaker as gestures do (Cook, Yip, & Goldin-Meadow, 2012).

Perceiving gesture in communication The gestures that ac- company a speaker’s talk often emphasize information found in that talk. Seeing gestures has been found to improve com- prehension for listeners, particularly for bilinguals with low- proficiency in their second language (Sueyoshi & Hardison, 2005) or for young children (McNeil, Alibali & Evans, 2000). Seeing gestures also has been found to improve listeners’ mental imagery, particularly with respect to spatial topics (Driskell & Radtke, 2003). In a meta-analysis of gesture com- prehension studies, messages with gesture were shown to have a moderate, but significant, comprehension advantage for the listener compared with messages without gesture (Hostetter, 2011). But gestures also can provide nonredundant information not found in the speaker’s talk (Church, Garber & Rogalski 2007; Goldin-Meadow 2003; Kelly, 2001; Kelly,

Barr, Church & Lynch, 1999; McNeill, 1992), and listeners are able to take advantage of information conveyed uniquely in gesture (Goldin-Meadow & Sandhofer, 1999). For exam- ple, listeners are more likely to infer the meaning of an indirect request (e.g., BI’m getting cold^) if that speech is accompanied by a gesture (point to an open window) than if it is produced without the gesture (Kelly et al., 1999). Gesture serves a func- tion not only for speakers but also for listeners.

Moreover, the effects of perceiving gesture are not the same as the effects of perceiving instrumental action. For example, although adults can seamlessly and easily integrate informa- tion conveyed in speech with gesture, they often fail to inte- grate that information with instrumental action. For example, adults can easily ignore actions that are incongruent with the speech with which they are produced, but they have difficulty ignoring gestures that are incongruent with the speech they accompany, suggesting a difference in the relative strength of speech-gesture integration versus speech-action integration (Kelly, Healy, Özyürek, & Holler, 2014). Thus, gesture has a different relationship to speech than instrumental action does and, in turn, has a different effect on listeners than instrumen- tal action.

Gesture versus action in problem solving

Gesture not only has an impact on communication, but it also plays a role in more complex cognitive processes, such as conceptualization and problem-solving. Again, we find that gesture and instrumental action do not influence problem- solving in the same way.

Producing gesture in problem-solving Viewing gestures as representational action acknowledges that gesture has its base in action. Indeed, gestures often faithfully reflect our action experiences on objects in the world. Take, for example, the Tower of Hanoi task (Newell & Simon, 1972). In this task, participants are asked to move a number of disks, stacked from largest to smallest, from one peg to another peg; the goal is to recreate the stacked arrangement without ever placing a larger disk on top of a smaller disk by moving only one disk at a time. Solving the task involves actions (i.e., moving the disks) and the gestures that participants use to later explain their solution represent elements of the actions that they pro- duced while solving the task in the first place. More specifi- cally, participants who solved the problem using a physical tower produce more grasping gestures and curved trajectories than participants who solved the problem using a computer program in which disk icons could be dragged across the screen using a mouse curser (Cook & Tanenhaus, 2009). Gestures thus reflect a speaker’s action experiences in the world by re-presenting traces of those actions.

As noted earlier, gesturing about an action accomplishes nothing tangible––gesturing about moving disks does not

658 Psychon Bull Rev (2017) 24:652–665 Gesture as representational action: A paper about function7

actually move the disks. Even though gesture does not accom- plish anything physical, it can change our cognition in ways that action does not. Using the Tower of Hanoi task again as an example, we see that individuals who gesture about how they moved the disks encode the problem differently from individuals who do not gesture. In one study using this para- digm, after explaining how they solved the task and gesturing while doing so, participants were surreptitiously given a new stack of disks that looked like the original stack but differed in weight––the largest disk was now the lightest, the smallest disk became the heaviest and could no longer be lifted with one hand (Goldin-Meadow & Beilock, 2010). Participants who had initially produced one-handed gestures when de- scribing how to move the smallest disk were adversely affect- ed by the switch in weights––the more these participants ges- tured about the small disk with one hand, the slower their time to solve the problem after the disk weights had been switched (recall that the small disk could now not be moved with one hand). By gesturing about the smallest disk with one hand, participants set themselves up to think of the disk as light––the unanticipated switch in disk weights violated this expectation, leading to relatively poor performance after the switch. Importantly, if participants are not asked to provide explana- tions before the switch––and thus do not gesture––the switch effect disappears (Beilock & Goldin-Meadow, 2010). Moreover, participants who are asked to act on the objects and actually move the disks while explaining their solution (instead of gesturing) also do not show the switch effect (Trofatter, Kontra, Beilock & Goldin-Meadow, 2014). Gesture can have an effect (in this case, a detrimental effect) on thinking, and it can have a more powerful effect on think- ing than action does.

Finally, although gestures contain many components of the actions to which they refer, they also drop out components. Gestures are not, and cannot be, exact replicas of the actions to which they refer. Using the Tower of Hanoi task again as a case study, we see that one cannot veridically represent, in a single gesture, both the force needed to lift a heavy disk and the speed at which the disk is lifted. Incorporating into gesture the actual force needed to lift the disk (while lifting nothing) will necessarily result in a much faster movement than was made when the disk was actually lifted. Conversely, incorpo- rating into gesture the speed at which the disk actually moved (while moving nothing) would not require the same force as is necessary with an object in hand. Thus, gestures are not just smaller versions of actions; they have fundamentally different features from actions and, perhaps as a result, have different functional effects on cognitive processes.

Perceiving gesture in problem-solving The Tower of Hanoi task also exemplifies the impact that perceiving gesture has on the listener’s conceptualizations. As mentioned in the last sec- tion, participants gesture differently as a reflection of how they

solved the Tower of Hanoi task, producing smaller arches to represent the movement of the disks if they had solved the task on a computer than if they had solved the task with actual disks (Cook & Tanenhaus, 2009). Participants who saw those gestured explanations, but did not act on the Tower them- selves, were influenced by the gestures they saw when they were later asked to solve the problem themselves on a com- puter. Participants who watched someone explain how to solve the Tower of Hanoi task using gestures with high arches were more likely to produce higher arching movements them- selves on the computer (even though it is not necessary to arch the movement at all on the computer) than participants who saw someone use gestures with smaller arches––in fact, the bigger the gestured arcs, the bigger the participant’s move- ments on the computer screen. The gestures we see can influ- ence our own actions.

Gesture versus action in learning

Gesture also can lead learners to new ideas or concepts, both when learners see gesture in instruction and when they pro- duce gesture themselves. Learners are more likely to profit from a lesson in which the teacher gestures than from a lesson in which the teacher does not gesture (Cook, Duffy & Fenn, 2013; Ping & Goldin-Meadow 2008; Singer & Goldin- Meadow, 2005; Valenzeno, Alibali & Klatzky, 2003). When children gesture themselves, they are particularly likely to discover new ideas (Goldin-Meadow, Cook & Mitchell, 2009), retain those ideas (Cook, Mitchell & Goldin- Meadow, 2008), and generalize the ideas to novel problem types (Novack, Congdon, Hemani-Lopez & Goldin- Meadow, 2014). We argue that gesture can play this type of role in learning, because it is an action and thus engages the motor system but also because it represents information.

Learning from producing gesture Producing one’s own ac- tions has been found to support learning from infancy through adulthood (see Kontra, Goldin-Meadow, & Beilock, 2012, for a review). For example, 3-month-olds given experience wear- ing Velcro mittens that helped them grab the objects they reached for, come to interpret successfully other’s goal- directed reaches in a subsequent habituation test. In contrast, infants given experience simply watching someone else obtain objects while wearing the mittens do not come to understand other’s reaches (Gerson & Woodward, 2014; Sommerville, Woodward & Needham, 2005). Even college-aged students benefit from active experience in learning contexts. When physics students are given the chance to feel the properties of angular momentum first-hand (by holding a system of two bicycle wheels spinning around an axel), they score higher on a test of their understanding of force than their counterparts who simply had access to a visible depiction of the angular momentum (i.e., watching the deflection of a laser

Gesture as representational action: A paper about function8 659 Psychon Bull Rev (2017) 24:652–665

pointer connected to the bicycle system) (Kontra, Lyons, Fischer, & Beilock, 2015). Finally, neuroimaging data suggest that active experience manipulating objects leaves a lasting neural signature that is found when learners later view the objects without manipulating them (James, 2010; James & Swain, 2011; Longcamp et al., 2003; Prinz, 1997). For exam- ple, children given active experience writing letters later show greater activation in motor regions when just passively looking at letters in the scanner compared with children who were given practice looking at letters without writing them (James, 2010). Given that gestures are a type of action and that action affects learning, we might expect learning from gesture to resemble learning from action.

In fact, recent work suggests that learning via producing gesture engages a similar motor network as learning via pro- ducing action. When children were taught how to solve math- ematical equivalence problems while producing gesture strat- egies, they later showed greater activation in motor regions when passively solving the types of problems they had learned about compared with children who learned without gesture (Wakefield, et al., 2016). The same motor regions have been implicated in studies looking at the effect of producing action on learning (James 2010; James & Atwood, 2009; James & Swain, 2011), suggesting that gesture and action are similar in the effect they have on the brain.

But gestures differ from actions in a number of ways, and these differences might influence the impact that producing gesture has on learning. First, as mentioned earlier, actions are produced on objects; gestures are not. To compare the effects of learning via gesture versus learning via action, Novack and colleagues (2014) taught third-graders to produce actions on objects or gestures off objects during a math lesson. Children were shown movable number tiles placed over numbers in problems, such as 4 + 7 + 2 =__ + 2. Children in the Action condition were taught to pick up the first two number tiles (4 and 7) and then hold them in the blank. Children in the Concrete Gesture condition were taught to move their hands as if they were picking up the tiles and holding them in the blank but without actually moving them. Finally, children in the Abstract Gesture condition were taught to produce a V- point gesture to the first two numbers and then a point to the blank. In all three conditions, children were using their hands to represent a strategy for solving the problem––the grouping strategy in which the two numbers on the left side of the equation that are not found on the right are added and the sum is put in the blank. But the conditions differed in whether the hands actually moved objects. Although children in all three conditions learned how to solve the types of problems on which they had been trained, only children in the gesture conditions were able to transfer what they had learned to problems with a different format (near-transfer problems, e.g., 4 + 7 + 2 = 4 + __; far-transfer problems, e.g., 4 + 7 + 2 = __+ 6). Children in the Action condition seemed to have

gotten Bstuck^ in the concrete nature of the movements, learn- ing how to solve the problem at a shallow level that did not lead to transfer. Even more surprising, children in the concrete gesture condition were less successful on far-transfer prob- lems than children in the abstract gesture condition, suggest- ing that the closer a gesture’s form is to action, the closer the gesture comes to behaving like action.

Understanding how learners are affected by gesture com- pared to object-directed action is particularly important given the widespread use of manipulatives in educational settings (see Mix, 2010, for review). Manipulatives, or external sym- bols, are thought to help learners off-load some of the cogni- tive burden involved in maintaining abstract ideas in mind. Children can use concrete external symbols as a reference to be revisited, freeing up cognitive resources for other process- ing tasks. Importantly, external symbols can be moved and acted on, allowing for the integration of physical, motor pro- cesses with abstract conceptual ideas. Despite these potential benefits of learning through action, and consistent with findings on learning through gesture, research from the education liter- ature casts doubt on manipulative-based learning. Interacting with a manipulative can encourage learners to focus on the object itself rather than its symbolic meaning (Uttal, Scudder & DeLoache, 1997). The perceptual features of objects can be distracting (McNeil, Uttall, Jarvin & Sternberg, 2009), and young children in particular may lose track of the fact that the manipulatives not only are objects but also stand for something else (DeLoache, 1995). Gesture has the potential to distance learners from the concrete details of a manipulative, thus en- couraging them to approach the concept at a deeper level.

Learning from perceiving gesture The gestures that children see in instruction also have beneficial effects on learning (Cook, et al., 2013; Ping & Goldin-Meadow 2008; Singer & Goldin-Meadow, 2005; Valenzeno, et al., 2003). Some have suggested that seeing gestures can help learners connect ab- stract ideas, often presented in speech, to the concrete physical environment (Valenzeno et al., 2003). Seeing gesture also might support learning through the same mechanisms as pro- ducing gesture, that is, by engaging the motor system. Listeners recruit their own motor systems when listening to speakers who gesture (Ping, Goldin-Meadow, & Beilock, 2014), and neuro- imaging research suggests that recruiting the motor system may be key in learning. Adults learn more foreign words if they are taught those words while seeing someone produce meaningful iconic gestures compared with seeing someone produce mean- ingless movements (Macedonia, Muller, & Friederici, 2011). Those adults then activate areas of their premotor cortex when later recognizing words initially learned while seeing gesture, implicating the motor cortex in learning from seeing gesture.

Another way that perceiving gesture might have an impact on learning is through its ability to integrate with speech. Children are more likely to learn from a math lesson if the

660 Psychon Bull Rev (2017) 24:652–665 Gesture as representational action: A paper about function9

teacher provides one problem-solving strategy in speech si- multaneously with a different, complementary strategy in ges- ture (S1+G2) than if the teacher provides the same two strat- egies in speech (S1→S2), which, of course, must be produced sequentially (Singer & Goldin-Meadow, 2005). Moreover, it is gesture’s ability to be produced simultaneously with speech that appears to promote learning. Children are more likely to learn from the math lesson if the gesture strategy and the speech strategy occur at the same time (S1+G2) than if the speech strategy occurs first, followed by the gesture strategy (S1→G2). In other words, the benefit of simultaneous speech+gesture instruction disappears when the two strategies are presented sequentially rather than simultaneously in time (Congdon et al, 2016). A question for future work is whether learning through action will also be affected by timing––that is, will learning differ when an action problem-solving strate- gy is presented simultaneously with speech, compared to when the same action strategy is presented sequentially with speech? We suspect that this is yet another area where learning via gesture will differ from learning via action.

Part 4. Open questions and areas for future research

We have shown that, although gesture may be an effective learning tool, at least in part, because it is a type of action, it is the fact that gesture is abstracted action, or representational action, that likely gives rise to its far-reaching learning out- comes. Viewing gesture as representational action explains many of the benefits gesture confers in instruction and also may explain cases where using gesture in instruction is sub- optimal. For example, gesture instruction is less useful than action instruction for 2-year-olds (Novack et al., 2015), likely because, at this young age, children are only beginning to be able to decode representational forms. Gesture instruction also has been shown to be less useful than action instruction in children with a rudimentary understanding of a concept (Congdon & Levine, 2016), raising the possibility that a learner’s initial understanding of a task affects that learner’s ability to profit from a lesson on the task containing represen- tational action. In the final section, we explore open questions of this sort and discuss how their answers can inform the proposed framework.

One major topic that we have touched on in this paper, but that would benefit from additional research, is the relative effect of producing gesture versus perceiving gesture. We have provided evidence suggesting that gesture’s functions arise from its status as representational action both for the producer of gesture and for the perceiver of gesture. Thus, we believe that our framework can be applied to both situa- tions. However, the magnitude of gesture’s effects may not be identical for doing versus seeing gesture (Goldin-Meadow et al., 2012). Moreover, there might be effects on thinking

and learning that depend on whether a person is perceiving or producing a gesture. For example, gesture’s ability to sup- port learning, retention, and generalization may depend on whether the gesture is produced or perceived. When children are shown a gesture that follows speech and is thus produced on its own, they do no better after instruction than when the same information is displayed entirely in speech (Congdon et al., 2016). In other words, learning from a seen gesture may depend on its being produced simultaneously with speech. In contrast, when children are told to produce a ges- ture, they profit from that instruction (Brooks & Goldin- Meadow, 2015) and retain what they have learned (Cook, et al., 2008) even when the gesture is produced on its own without speech. Learning from a produced gesture does not seem to depend on its being produced along with speech. Producing versus perceiving gesture might then function through distinct mechanisms, although we suggest that ges- ture’s status as a representational form is still essential to both. Additional studies that directly compare learning from seeing versus doing gesture are needed to determine whether the mechanisms that underlie these two processes are the same or different, and whether seeing versus doing gesture interacts with interpreting gesture as representational action. For exam- ple, it may be easier to think of a gesture as representational when producing it (even if it’s a novel action) than when seeing someone else produce the gesture.

A related open question is whether movement is categorized as gesture in the same way for perceiving versus producing movement. We reviewed evidence about when perceivers of a movement see the movement as representational (Novack et al., 2016). However, it is unclear whether the same features lead producers of a movement to see the movement as representa- tional. This question is particularly relevant in tasks where learners are taught to produce movements during a lesson (Goldin-Meadow et al., 2009). These movements are meaning- less to the learner at the beginning of the lesson. The question is whether the movements become meaningful, and therefore rep- resentational, during the lesson and, if so, when? Do children think of these hand movements as Bgesture^ when they are initially taught them, or do they think of them first as Bmovement-for-its-own sake^ and only gradually come to see the movements as Bgesture^ as their conceptual understanding of the lesson shifts? If the process is gradual, might there be markers or features within the movement itself that an observer could use to determine when a rote movement has become a true gesture?

This possibility brings into focus whether learners need to be aware of the representational status of a gesture in order to benefit from that gesture during instruction. Although gesture- training studies find that, on average, instruction with gesture supports learning better than instruction without gesture (see Novack & Goldin-Meadow, 2015, for review), there is always variability in learner outcomes. Might a learner’s ability to profit from a gestural movement be related to that learner’s

Gesture as representational action: A paper about function10 661 Psychon Bull Rev (2017) 24:652–665

ability to categorize that movement as meaningful? Perhaps only learners who see a movement as gesture will benefit from incorporating that movement into instruction. Alternatively, learners may be able to benefit from gesture in instruction without being explicitly aware of its representational proper- ties. Thomas and Lleras (2009) found that adults asked to produce arm movements that were consistent with the solution to an unrelated problem were more likely to subsequently solve the problem, compared to adults asked to produce arm movements that were inconsistent with the solution to the problem (see also Brooks & Goldin-Meadow, 2015, for sim- ilar evidence in children). Importantly, these adults were not aware of the link between the arm movements and the problem-solving task (they were told that the arm movements were for Bexercise breaks^). Thus, at least in some cases, learners do not need to consciously see a movement as mean- ingful in order to learn from it.

Another open question related to the issue of categorizing movement as gesture or instrumental action is whether there are in-between cases. Clark (1996) has identified a class of movements called demonstrations––actions produced with the intention of showing something to someone. For example, if a mother were to show her child how to open a jar, she could hold the jar out in front of the child, twist open the lid in an exaggerated manner, and then put the lid back on the jar, handing the jar to the child to try the action himself. This object-focused movement has elements of an instrumental ac- tion––the mother’s hands directly interact with the object and cause a physical change. However, the movement also has obvious elements of representational actions––in the end, the jar is not open and the movement is clearly performed for communicative (as opposed to purely instrumental) purposes. As another example, consider Bhold-ups^––gestures in which someone holds up an object to display it to someone else (e.g., a child holds up her bottle to draw it to her mother’s attention). Hold-ups have some aspects of gesture––they are intended to communicate and are like deictic pointing gestures in that they indicate a particular object. They also have aspects of instru- mental actions; they are produced directly on objects. Developmentally, hold-ups tend to emerge before pointing gestures (Bates Camaioni & Volterra, 1975), lending credence to the idea that hold-ups may not be as representational as empty-handed gestures. The important question from our point of view is whether hold-ups function like gestures for the child. It turns out that they do, in at least in one sense; they predict the onset of various aspects of spoken language. For example, hold-ups have been counted as deictic gestures in studies finding that early gesture predicts the size of a child’s subsequent spoken vocabulary (Rowe & Goldin-Meadow, 2009), the introduction of particular lexical items into a child’s spoken vocabulary (Iverson & Goldin-Meadow, 2005), and the developmental onset of noun phrases (Cartmill, Hunsicker & Goldin-Meadow, 2014).

In terms of developmental questions, although we have reviewed evidence exploring the features that encourage adults to view a hand movement as a representational action (Novack et al., 2016), it is an open question as to what infants think about the movements they see. Infants have a special ability to process actions on objects (Woodward, 1998), but how do infants process actions off objects, that is, gestures? One possibility is that infants think of gestures as movements for their own sake––seeing them as mere hand waving (Schachner & Carey, 2013). Another possibility is that, de- spite the fact that infants may not be able to correctly interpret the meaning of a gesture, they can nonetheless categorize the gesture as a representational act. Just as infants seem to know that speech can communicate even if they cannot understand that speech (Vouloumanos, Onishi & Pogue, 2012), infants might know that gestures are meant to represent without being able to understand what they represent. Knowing what infants think about gesture (and whether they categorize it as a unique form) would contribute to our understanding of the develop- ment of gesture processing.

Finally, with respect to how gestures affect learning, addi- tional research is needed to determine when in the learning process, and for which content domains, gesture instruction is particularly helpful. Gesture’s status as representational action might mean that it is most useful for some content domains, and not others. For example, gesture has been shown to sup- port generalization and retention in math instruction. But math is a relatively abstract subject. Gesture may be less useful in domains grounded in physical experience, such as physics, where direct action on objects has been found to support learn- ing (Kontra et al., 2015; generalization has yet to be studied in these domains). There are, however, domains that are ground- ed in physical experience, such as dance, where practicing gesture-like movements has been found to promote learning better than practicing the actual dance movements (Kirsh, 2010, 2011). Dancers often Bmark^ their movements, a form of practice in dance that involves producing attenuated ver- sions of dance moves. Marking is comparable to gesturing in that the movements are produced to represent other move- ments, rather than to have a direct effect on the world (i.e., they represent movements that will, in the end, be seen by an audience, see Kirsh, 2011). Dancers use marking when prac- ticing on their own, as well as when communicating with other dancers (Kirsh, 2010). This marking seems to function like gesture in that it promotes learning, even though dance is grounded in physical action.

Conclusions

In this paper, we present a framework for understanding gesture’s function. We propose that gesture has unique effects on thinking and learning because of its status as

662 Psychon Bull Rev (2017) 24:652–665 Gesture as representational action: A paper about function11

representational action. More specifically, the fact that gesture is representational action, and not instrumental action, is critical to its capacity to support generalization beyond the specific and to support retention over a period of time. Our proposal is agnostic about whether gesture’s role in learning depends on its being embodied, and about whether the Gesture-as-Simulated-action framework can account for how gesture is produced, its mechanism. The proposal is designed to account for why gesture is pro- duced, that is, for the functions it serves, particularly in a learning context. Our proposal is thus not inconsistent with the mechanistic account of gesture production pro- posed in the GSA framework (Hostetter & Alibali, 2008). But it does offer another perspective—a functional per- spective that highlights the differences between gestures and other types of actions.

Although in some cases mechanism and function are critically related, in other cases they are not. For example, consider an alligator’s nightly sojourn into the Mississippi River. The functional explanation for this phenomenon is that the alligator is cold-blooded, and in the evening the river water is warmer than the air; entering the water at night serves the function of helping the alligator maintain its body temperature during the overnight hours. However, the mechanism by which this behavior comes about has nothing to do with temperature and depends instead on changes in sunlight. The alligator heads for the water in response to fading light, a relationship that was discovered by experimentally dissociating tempera- ture from light. Alligators approach the water as the light fades whether or not the temperature changes and do not approach the water as the temperature drops unless the light fades (Lang, 1976). Thus, the temperature- regulation function of the behavior (going into water to regulate temperature) is different from its light-sensitive mechanism (going into the water in response to changes in light). We therefore cannot assume that the function of a phenomenon is the complement of its mechanism and must explore function in its own right. Our hope is that by expanding the investigation of gesture to include a frame- work built around its functions, we will come to a more complete understanding of how and why we move our hands when we talk.

Acknowledgments This work was supported by NIH grant number R01-HD047450 and NSF grant number BCS-0925595 to Goldin- Meadow, NSF grant number SBE-0541957 (Spatial Intelligence and Learning Center, Goldin-Meadow is a co-PI), and a grant from the Institute of Education Sciences (R305 B090025) to S. Raudenbush in support of Novack. SGM thanks Bill Wimsatt and Martha McClintock for introducing her to the distinction between mechanism and function, and for convincing her of its importance in understanding scientific ex- planation back in 1978 when we taught our first Mind course together in the Social Sciences Division at the University of Chicago.

References

Acredolo, L. P., & Goodwyn, S. W. (1985). Symbolic gesturing in lan- guage development: A case study. Human Development, 28, 40–49. doi:10.1159/000272934

Acredolo, L., & Goodwyn, S. (1988). Symbolic gesturing in normal infants. Child Development, 59, 450–466. doi:10.2307/1130324

Ambrosini, E., Reddy, V., de Looper, A., Costantini, M., Lopez, B., & Sinigaglia, C. (2013). Looking ahead: Anticipatory gaze and motor ability in infancy. PLoS One, 8, e67916. doi:10.1371/journal. pone.0067916

Baldwin, D. A., & Baird, J. A. (2001). Discerning intentions in dynamic human action. Trends in Cognitive Sciences, 5, 171–178. doi:10.1016/S1364-6613(00)01615-6

Bates, E. (1976). Language and context: The acquisition of pragmatics (Vol. 13). New York: Academic Press.

Bates, E., Camaioni, L., & Volterra, V. (1975). The acquisition of perfor- matives prior to speech. Merrill-qalmer Quarterly of Behavior and Development, 21, 205–226.

Behne, T., Carpenter, M., & Tomasello, M. (2014). Young children create iconic gestures to inform others. Developmental Psychology, 50, 2049–2060. doi:10.1037/a0037224

Behne, T., Liszkowski, U., Carpenter, M., & Tomasello, M. (2012). Twelve-month-olds’ comprehension and production of pointing. British Journal of Developmental Psychology, 30, 359–375. doi:10.1111/j.2044-835X.2011.02043.x

Beilock, S. L., & Goldin-Meadow, S. (2010). Gesture changes thought by grounding it in action. Psychological Science, 21, 1605–1610. doi:10.1177/0956797610385353

Beilock, S. L., Lyons, I. M., Mattarella-Micke, A., Nusbaum, H. C., & Small, S. L. (2008). Sports experience changes the neural processing of action language. Proceedings of the National Academy of Sciences of the United States of America, 105, 13269–13273. doi:10.1073/pnas.0803424105

Bernardis, P., & Gentilucci, M. (2006). Speech and gesture share the same communication system. Neuropsychologia, 44, 178–190. doi:10.1016/j.neuropsychologia.2005.05.007

Bower, G. H., & Rinck, M. (1999). Goals as generators of activation in narrative understanding. In S. R. Goldman, A. C. Graesser, & P. van den Broek (Eds.), Narrative comprehension, causality, and coher- ence: Essays in honor of Tom Trabasso (pp. 111–134). Mahwah: Erlbaum.

Brooks, N., & Goldin-Meadow, S. (2015). Moving to learn: How guiding the hands can set the stage for learning. Cognitive Science. doi:10.1111/cogs.12292

Cartmill, E. A., Hunsicker, D., & Goldin-Meadow, S. (2014). Pointing and naming are not redundant: Children use gesture to modify nouns before they modify nouns in speech. Developmental Psychology, 50, 1660–1666. doi:10.1037/a0036003

Casile, A., & Giese, M. A. (2006). Nonvisual motor training influences biological motion perception. Current Biology, 16, 69–74. doi:10.1016/j.cub.2005.10.071

Church, R. B., Garber, P., & Rogalski, K. (2007). The role of gesture in memory and social communication. Gesture, 7, 137–158. doi:10.1075/gest.7.2.02bre

Church, R. B., Kelly, S., & Holcombe, D. (2014). Temporal synchrony between speech, action and gesture during language production. Language, Cognition and Neuroscience, 29, 345–354. doi:10.1080 /01690965.2013.857783

Clark, H. H. (1996). Using language. Cambridge, GB: Cambridge Univ. Press.

Congdon, E. L., & Levine, S. C. (2016) Learning to measure through action and gesture: Children’s starting state matters. Manuscript submitted for publication.

Gesture as representational action: A paper about function12 663 Psychon Bull Rev (2017) 24:652–665

Congdon, E. L., Novack, M. A., Brooks, N. B., Hemani-Lopez, N., O’Keefe, L. & Goldin-Meadow, S. (2016). Better together: Simultaneous presentation of speech and gesture in math instruction supports generalization and retention. Manuscript submitted for publication.

Cook, S. W., Duffy, R. G., & Fenn, K. M. (2013). Consolidation and transfer of learning after observing hand gesture. Child Development, 84, 1863–1871. doi:10.1111/cdev.12097

Cook, S. W., Mitchell, Z., & Goldin-Meadow, S. (2008). Gesturing makes learning last. Cognition, 106, 1047–1058. doi:10.1016/j. cognition.2007.04.010

Cook, S. W., & Tanenhaus, M. K. (2009). Embodied communication: Speaker’s gestures affect listeners’ actions. Cognition, 113, 98– 104. doi:10.1016/j.cognition.2009.06.006.Embodied

Cook, S. W., Yip, T. K., & Goldin-Meadow, S. (2012). Gestures, but not meaningless movements, lighten working memory load when explaining math. Language and Cognitive Processes, 27, 594– 610. doi:10.1080/01690965.2011.567074

DeLoache, J. S. (1995). Early understanding and use of symbols: The model model. Current Directions in Psychological Science, 4, 109– 113. doi:10.1111/1467-8721.ep1077240

Driskell, J. E., & Radtke, P. H. (2003). The effect of gesture on speech production and comprehension. Human Factors: The Journal of the Human Factors and Ergonomics Society, 45, 445–454. doi:10.1518 /hfes.45.3.445.27258

Filippi, C., & Woodward, A. L. (2016). Action experience changes atten- tion to kinematic cues. Frontiers in Psychology, 7, 19. doi:10.3389 /fpsyg.2016.00019

Gerson, S. A., & Woodward, A. L. (2014). Learning from their own actions: The unique effect of producing actions on infants’ action understanding. Child Development, 85, 264–277. doi:10.1111 /cdev.12115

Gibson, E., Piantadosi, S. T., Brink, K., Bergen, L., Lim, E., & Saxe, R. (2013). A noisy-channel account of crosslinguistic word order var- iation. Psychological Science, 24, 1079–1088. doi:10.1177 /0956797612463705

Goldin-Meadow, S. (2003). Hearing gesture: How our hands help us think. Cambridge, MA: Harvard University Press.

Goldin-Meadow, S. (2014). How gesture helps children learn language. In I. Arnon, M. Tice, C. Kurumada, & B. Estigarribia (Eds.), Language in interaction: Studies in honor of Eve V. Clark (pp. 157–171). Amsterdam: John Benjamins.

Goldin-Meadow, S., & Beilock, S. L. (2010). Action’s influence on thought: The case of gesture. Perspectives on Psychological Science, 5, 664–674. doi:10.1177/1745691610388764

Goldin-Meadow, S., & Brentari, D. (in press). Gesture, sign and lan- guage: The coming of age of sign language and gesture studies. Behavioral and Brain Sciences. online first. doi:10.1017 /S0140525X15001247

Goldin-Meadow, S., & Butcher, C. (2003). Pointing toward two-word speech in young children. In S. Kita (Ed.), Pointing: Where lan- guage, culture, and cognition meet (pp. 85–107). Mahwah, NJ: Earlbaum Associates.

Goldin-Meadow, S., Cook, S. W., & Mitchell, Z. A. (2009). Gesturing gives children new ideas about math. Psychological Science, 20, 267–272. doi:10.1111/j.1467-9280.2009.02297

Goldin-Meadow, S., Levine, S. L., Zinchenko, E., Yip, T. K.-Y., Hemani, N., & Factor, L. (2012). Doing gesture promotes learning a mental transformation task better than seeing gesture. Developmental Science, 15, 876–884. doi:10.1111/j.1467-7687.2012.01185.x

Goldin-Meadow, S., McNeill, D., & Singleton, J. (1996). Silence is lib- erating: Removing the handcuffs on grammatical expression in the manual modality. Psychological Review, 103, 34–55.

Goldin-Meadow, S., Nusbaum, H., Kelly, S. D., & Wagner, S. (2001). Explaining math: gesturing lightens the load. Psychological Science, 12, 516–522. doi:10.1111/1467-9280.00395

Goldin-Meadow, S., & Sandhofer, C. M. (1999). Gesture conveys sub- stantive information about a child’s thoughts to ordinary listeners. Developmental Science, 2, 67–74. doi:10.1037/0033-295 X.103.1.34

Goldin-Meadow, S., So, W. C., Ozyurek, A., & Mylander, C. (2008). The natural order of events: How speakers of different languages repre- sent events nonverbally. Proceedings of the National Academy of Sciences, 105, 9163–9168. doi:10.1073/pnas.0710060105

Goodrich, W., & Hudson Kam, C. L. (2009). Co-speech gesture as input in verb learning. Developmental Science, 12, 81–87. doi:10.1111 /j.1467-7687.2008.00735.x

Graham, J. A., & Heywood, S. (1975). The effects of elimination of hand gestures and of verbal codability on speech performance. European Journal of Social Psychology, 5, 189–195. doi:10.1002 /ejsp.2420050204

Hall, M. L., Ferreira, V. S., & Mayberry, R. I. (2013). Investigating con- stituent order change with elicited pantomime: A functional account of SVO emergence. Cognitive Science, 38, 943–972. doi:10.1111 /cogs.12105

Hodges, L. E., Özçalışkan, Ş., & Williamson, R. A. (2015). How early do children understand different types of iconicity in gesture? Proceedings of the 39th Boston University conference on language development. Somerville, MA: Cascadilla Press.

Hostetter, A. B. (2011). When do gestures communicate? A meta-analy- sis. Psychological Bulletin, 137, 297–315. doi:10.1037/a0022128

Hostetter, A. B., & Alibali, M. W. (2008). Visible embodiment: Gestures as simulated action. Psychonomic Bulletin & Review, 15, 495–514. doi:10.3758/PBR.15.3.495

Iverson, J. M., Capirci, O., & Caselli, M. C. (1994). From communication to language in two modalities. Cognitive Development, 9, 23–43. doi:10.1016/0885-2014(94)90018-3

Iverson, J. M., & Goldin-Meadow, S. (2005). Gesture paves the way for language development. Psychological Science, 16, 368–371. doi:10.1111/j.0956-7976.2005.01542.x

James, K. H. (2010). Sensori-motor experience leads to changes in visual processing in the developing brain. Developmental Science, 13, 279–288. doi:10.1111/j.1467-7687.2009.00883.x

James, K. H., & Atwood, T. P. (2009). The role of sensorimotor learning in the perception of letter-like forms: Tracking the causes of neural specialization for letters. Cognitive Neuropsychology, 26, 91–110. doi:10.1080/02643290802425914

James, K. H., & Swain, S. N. (2011). Only self-generated actions create sensori-motor systems in the developing brain. Developmental Science, 14, 673–678. doi:10.1111/j.1467-7687.2010.01011.x

Kelly, S. D. (2001). Broadening the units of analysis in communication: Speech and nonverbal behaviours in pragmatic comprehension. Journal of Child Language, 28, 325–349. doi:10.1017 /S0305000901004664

Kelly, S. D., Barr, D. J., Church, R. B., & Lynch, K. (1999). Offering a hand to pragmatic understanding: The role of speech and gesture in comprehension and memory. Journal of memory and Language, 40, 577–592. doi:10.1006/jmla.1999.2634

Kelly, S. D., Özyürek, A., & Maris, E. (2010). Two sides of the same coin: Speech and gesture manually interact to enhance comprehension. Psycho log i ca l S c i ence , 2 1 , 260– 267 . d o i : 10 . 1177 /0956797609357327

Kelly, S. D., Healy, M., Özyürek, A., & Holler, J. (2014). The processing of speech, gesture, and action during language comprehension. Psychonomic Bulletin & Review, 22, 517–523. doi:10.3758 /s13423-014-0681-7

Kendon, A. (1980). Gesticulation and speech: Two aspects of the process of utterance. In M. R. Key (Ed.), The relationship of verbal and nonverbal communication (pp. 207–227). The Hague, the Netherlands: Mouton.

Kendon, A. (2004). Gesture: Visible action as utterance. Cambridge: Cambridge University Press.

664 Psychon Bull Rev (2017) 24:652–665 Gesture as representational action: A paper about function13

Kimura, D. (1973). Manual activity during speaking–I. Righthanders. Neuropsychologia, 11, 45–50. doi:10.1016/0028-3932(73)90063-8

Kirsh, D. (2010). Thinking with the body. In S. Ohlsson & R. Catrambone (Eds.), Proceedings of the 32nd annual conference of the cognitive science society (pp. 2864–2869). Austin, TX: Cognitive Science Society. doi:10.1145/2037296.2037303

Kirsh, D. (2011). How marking in dance constitutes thinking with the body. Versus: Quaderni di Studi Semiotici, 113–115, 179–210.

Kita, S. (2000). How representational gestures help speaking. In D. McNeill (Ed.), Language and gesture (pp. 162–185). Cambridge, UK: Cambridge University Press.

Kita, S. (2009). Cross-cultural variation of speech-accompanying gesture: A review. Language and Cognitive Processes, 24(2), 145–167. doi:10.1080/01690960802586188

Kita, S., & Özyürek, A. (2003). What does cross-linguistic variation in semantic coordination of speech and gesture reveal?: Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and Language, 48, 16–32. doi:10.1016/S0749-596X(02)00505-3

Kontra, C., Goldin-Meadow, S., & Beilock, S. L. (2012). Embodied learning across the life span. Topics in Cognitive Science, 4, 731– 739. doi:10.1111/j.1756-8765.2012.01221.x

Kontra, C., Lyons, D. J., Fischer, S. M., & Beilock, S. L. (2015). Physical experience enhances science learning. Psychological Science, 26, 737–749. doi:10.1177/0956797615569355

Krehm, M., Onishi, K. H., & Vouloumanos, A. (2014). I see your point: Infants under 12 months understand that pointing is communicative. Journal of Cognition and Development, 15, 527–538. doi:10.1080 /15248372.2012.736112

Lang, J. W. (1976). Amphibious behavior of Alligator mississippiensis: Roles of a circadian rhythm and light. Science, 191, 575–577. doi:10.1126/science.1251194

LeBarton, E. S., Goldin-Meadow, S., & Raudenbush, S. (2015). Experimentally-induced increases in early gesture lead to increases in spoken vocabulary. Journal of Cognition and Development, 16, 199–220. doi:10.1080/15248372.2013.858041

Loehr, D. (2007). Aspects of rhythm in gesture and speech. Gesture, 7, 179–214.

Longcamp, M., Anton, J. L., Roth, M., & Velay, J. L. (2003). Visual presentation of single letters activates a premotor area involved in writing. NeuroImage, 19, 1492–1500. doi:10.1016/S1053- 8119(03)00088-0

Lucca, K. R., & Wilborn, P. M. (2016). Communicating to learn: Infants’ pointing gestures facilitate fast mapping. Manuscript submitted for publication.

Macedonia, M., Muller, K., & Friederici, A. D. (2011). The impact of iconic gestures on foreign language word learning and its neural substrate. Human Brain Mapping, 32, 982–998. doi:10.1002 /hbm.21084

Marentette, P., & Nicoladis, E. (2011). Preschoolers’ interpretations of gesture: Label or action associate? Cognition, 121, 386–399. doi:10.1016/j.cognition.2011.08.012

McNeil, N., Alibali, M., & Evans, J. L. (2000). The role of gesture in children's language comprehension: Now they need it, now they don't. Journal of Nonverbal Behavior, 24, 131–150. doi:10.1006 /jmla.2000.2752

McNeil, N. M., Uttal, D. H., Jarvin, L., & Sternberg, R. J. (2009). Should you show me the money? Concrete objects both hurt and help per- formance on mathematics problems. Learning and Instruction, 19, 171–184. doi:10.1016/j.learninstruc.2008.03.005

McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.

Mix, K. S. (2010). Spatial tools for mathematical thought. In K. S. Mix, L. B. Smith, & M. Gasser (Eds.), Space and language. New York: Oxford University Press.

Namy, L. L. (2008). Recognition of iconicity doesn’t come for free. Developmental Science, 11, 841–846. doi:10.1111/j.1467- 7687.2008.00732.x

Namy, L. L., Campbell, A. L., & Tomasello, M. (2004). The changing role of iconicity in non-verbal symbol learning: A U-shaped trajec- tory in the acquisition of arbitrary gestures. Journal of Cognition and Development, 5, 37–57. doi:10.1207/s15327647jcd0501_3

Newell, A., & Simon, H. A. (1972). Human problem-solving. EnglewoodCliffs, NJ: Prentice-Hall.

Nicoladis, E., Mayberry, R. I., & Genesee, F. (1999). Gesture and early bilingual development. Developmental Psychology, 35, 514. doi:10.1037/0012-1649.35.2.514

Novack, M. A., Congdon, E., Hemani-Lopez, N., & Goldin-Meadow, S. (2014). From action to abstraction: Using the hands to learn math. Psycho log i ca l S c i ence , 2 5 , 903–910 . d o i : 10 . 1177 /0956797613518351

Novack, M. A., Filippi, C., Goldin-Meadow, S., & Woodward, A. (2016). Actions speak louder than gestures when you are 2-years-old. Mansucript submitted for publication.

Novack, M., & Goldin-Meadow, S. (2015). Learning from gesture: How our hands change our minds. Educational Psychology Review, 27, 205–412. doi:10.1007/s10648-015-9325-3

Novack, M. A., Goldin-Meadow, S., & Woodward, A. (2015). Learning from gesture: How early does it happen? Cognition, 142, 138–147. doi:10.1016/j.cognition.2015.05.018

Novack, M. A., Wakefield, E. M., & Goldin-meadow, S. (2016). What makes a movement a gesture? Cognition, 146, 339–348. doi:10.1016/j.cognition.2015.10.014

Özçalışkan, Ş., & Goldin-Meadow, S. (2005). Do parents lead their chil- dren by the hand? Journal of Child Language, 32, 481–505. doi:10.1017/S0305000905007002

Özçalışkan, Ş., & Goldin-Meadow, S. (2011). Is there an iconic gesture spurt at 26 months. Integrating gestures: The interdisciplinary na- ture of gesture. Amsterdam, NL: John Benjamins.

Özçalışkan, S., Lucero, C., & Goldin-Meadow, S. (2016). Does language shape silent gesture? Cognition, 148, 10–18. doi:10.1016/j. cognition.2015.12.001

Ping, R. M., & Goldin-Meadow, S. (2008). Hands in the air: Using ungrounded iconic gestures to teach children conservation of quan- tity. Developmental Psychology, 44, 1277–1287. doi:10.1037/0012- 1649.44.5.1277

Ping, R., Goldin-Meadow, S., & Beilock, S. (2014). Understanding ges- ture: Is the listener’s motor system involved. Journal of Experimental Psychology: General, 143, 195–204. doi:10.1037 /a0032246

Prinz, W. (1997). Perception and action planning. European Journal of Cognitive Psychology, 9, 129–154. doi:10.1080/713752551

Rauscher, F., Krauss, R. M., & Chen, Y. (1996). Gesture, speech, and lexical access: The role of lexical movements in speech production. Psychological Science, 7, 226–231. doi:10.1111/j.1467-9280.1996. tb00364.x

Rowe, M. L., & Goldin-Meadow, S. (2009). Differences in early gesture explain SES disparities in child vocabulary size at school entry. Science, 323, 951–953. doi:10.1126/science.1167025

Schachner, A., & Carey, S. (2013). Reasoning about Birrational^ actions: When intentional movements cannot be explained, the movements themselves are seen as the goal. Cognition, 129, 309–327. doi:10.1016/j.cognition.2013.07.006

Searle, J. R. (1980). The intentionality of intention and action. Inquiry, 22, 253–280. doi:10.1080/00201747908601876

Singer, M. A., & Goldin-Meadow, S. (2005). Children learn when their teacher’s gestures and speech differ. Psychological Science, 16, 85– 89. doi:10.1111/j.0956-7976.2005.00786.x

Sommerville, J. A., Woodward, A. L., & Needham, A. (2005). Action experience alters 3-month-old infants’ perception of others’ actions. Cognition, 96, B1–B11. doi:10.1016/j.cognition.2004.07.004

Gesture as representational action: A paper about function14 665 Psychon Bull Rev (2017) 24:652–665

Sueyoshi, A., & Hardison, D. M. (2005). The role of gestures and facial cues in second language listening comprehension. Language Learning, 55, 661–699. doi:10.1111/j.0023-8333.2005.00320.x

Thomas, L. E., & Lleras, A. (2009). Swinging into thought: Directed movement guides insight in problem solving. Psychonomic Bulletin & Review, 16, 719–723. doi:10.3758/PBR.16.4.719

Tolar, T. D., Lederberg, A. R., Gokhale, S., & Tomasello, M. (2008). The development of the ability to recognize the meaning of iconic signs. Journal of Deaf Studies and Deaf Education, 13, 225–240. doi:10.1093/deafed/enm045

Tomasello, M., Carpenter, M., & Liszkowski, U. (2007). A new look at infant pointing. Child Development, 78, 705–722. doi:10.1111 /j.1467-8624.2007.01025.x

Trabasso, T., & Nickels, M. (1992). The development of goal plans of action in the narration of a picture story. Discourse Processes, 15, 249–275. doi:10.1080/01638539209544812

Trofatter, C., Kontra, C., Beilock, S., & Goldin-Meadow, S. (2014). Gesturing has a larger impact on problem-solving than action, even when action is accompanied by words. Language, Cognition and Neuroscience, 30, 251–260. doi:10.1080/23273798.2014.905692

Uttal, D., Scudder, K., & DeLoache, J. (1997). Manipulatives as symbols: A new perspective on the use of concrete objects to teach mathe- matics. Journal of Applied Developmental Psychology, 18, 37–54. doi:10.1016/S0193-3973(97)90013-7

Valenzeno, L., Alibali, M. W., & Klatzky, R. (2003). Teachers’ gestures facilitate students’ learning: A lesson in symmetry. Contemporary Educational Psychology, 28, 187–204. doi:10.1016/S0361-476 X(02)00007-3

Vouloumanos, A., Onishi, K. H., & Pogue, A. (2012). Twelve-month-old infants recognize that speech can communicate unobservable inten- tions. Proceedings of the National Academy of Sciences of the United States of America, 109, 12933–12937. doi:10.1073 /pnas.112105710

Wagner, S., Nusbaum, H., & Goldin-Meadow, S. (2004). Probing the mental representation of gesture: Is handwaving spatial? Journal of Memory and Language, 50, 395–407. doi:10.1016/j. jml.2004.01.002

Wakefield, E. M., Congdon, E. L, Novack, M. A., Goldin-Meadow, S., & James, K. (2016). Learning math by hand: The neural effects of gesture-based instruction in 8-year-old children. Manuscript sub- mitted for publication.

Wilson, M. (2002). Six views of embodied cognition. Psychonomic Bulletin & Review, 9, 625–636. doi:10.3758/BF03196322

Woodward, A. L. (1998). Infants selectively encode the goal object of an actor’s reach. Cognition, 69, 1–34. doi:10.1016/S0010- 0277(98)00058-4

Woodward, A. L., & Guajardo, J. J. (2002). Infants’ understanding of the point gesture as an object-directed action. Cognitive Development, 17, 1061–1084. doi:10.1016/S0885-2014(02)00074-6

Yoon, J. M., Johnson, M. H., & Csibra, G. (2008). Communication- induced memory biases in preverbal infants. Proceedings of the National Academy of Sciences, 105, 13690–13695. doi:10.1073 /pnas.0804388105

Zacks, J. M., Tversky, B., & Iyer, G. (2001). Perceiving, remembering, and communicating structure in events. Journal of Experimental Psychology: General, 130, 29–58. doi:10.1037/0096-3445.130.1.29