Flexible Model: Challenging Iconic Memory and Visual Working Memory Division, Exercises of Psychology

Download Flexible Model: Challenging Iconic Memory and Visual Working Memory Division and more Exercises Psychology in PDF only on Docsity! IS ICONIC MEMORY ICONIC? Jake Quilty-Dunn Faculty of Philosophy & Brasenose College, University of Oxford Department of Philosophy & PNP Program, WUSTL Abstract: Short-term memory in vision is typically thought to divide into at least two memory stores: a short, fragile, high-capacity store known as iconic memory, and a longer, durable, capacity-limited store known as visual working memory (VWM). This paper argues that iconic memory stores icons, i.e., image-like perceptual representations. The iconicity of iconic memory has significant consequences for understanding consciousness, nonconceptual content, and the perception–cognition border. Steven Gross and Jonathan Flombaum have recently challenged the division between iconic memory and VWM by arguing against the idea of capacity limits in favor of a flexible resource-based model of short-term memory. I argue that, while VWM capacity is probably governed by flexible resources rather than a sharp limit, the two memory stores should still be distinguished by their representational formats. Iconic memory stores icons, while VWM stores discursive (i.e., language-like) representations. I conclude by arguing that this format-based distinction between memory stores entails that prominent views about consciousness and the perception–cognition border will likely have to be revised. 2 “There are two kinds of visual memory: one when you skillfully recreate an image in the laboratory of your mind, with your eyes open (and then I see Annabel in such general terms as: ‘honey-colored skin,’ ‘thin arms,’ ‘brown bobbed hair,’ ‘long lashes,’ ‘big bright mouth’); and the other when you instantly evoke, with shut eyes, on the dark innerside of your eyelids, the objective, absolutely optical replica of a beloved face, a little ghost in natural colors[.]” —Vladimir Nabokov §1. Introduction As Nabokov points out, there is more than one kind of memory. ere is a difference between short-term memory, where you store a new phone number long enough to dial it a moment later, and long-term memory, where you store your own phone number such that you can produce it whenever asked (Miller 1956). Within long-term memory, there is a difference between semantic memory, where you store your knowledge that e Rolling Stones are a rock band, and episodic memory, where you may store your memory of hearing Exile on Main Street for the first time (Tulving 1983). ere is also more than one kind of short-term memory. Sticking to vision, theorists generally accept the existence of visual working memory (VWM), which allows a relatively small number of items to be stored for a relatively short amount of time (Miller 1956; Baddeley 1986; Luck & Vogel 1997; 2013; Cowan 2001; Block 2011a; Prinz 2012; Suchow et al. 2014; Cohen et al. 2016). e majority (though by no means universal) opinion is that VWM doesn’t exhaust visual short-term memory.1 ere appears to be an earlier, sensory aspect to visual short-term memory that has a higher capacity and shorter duration than VWM—this store is typically known as “iconic memory” (Sperling 1960; Neisser 1967; Block 2011a; Phillips 2011). Some argue for a third, “fragile” visual short-term memory store that is intermediate between iconic memory and VWM in processing order as well as capacity and duration (Sligte et al. 2008; Block 2011a; Pinto et al. 2013). Any adequate characterization of a memory store must (minimally) specify its capacity, duration, architecture, and format. Capacity is the amount of information a store can hold, and duration is the length of time it can hold it. e architecture of a memory store is its basic, invariant structure (Pylyshyn 1984). For example, some have argued that VWM contains at most four “slots,” such that storing an object and its various properties takes up one slot, and 1 e term “visual short-term memory” or “VSTM” for short is often used by vision scientists simply to refer to VWM. However, I use it here as a generic term meant to cover all short-term memory stores available to visual processing. 5 Figure 1 obeys PARTS; parts of the image correspond to parts of the cube. For example, a part of Figure 1 corresponds to the back top right corner. Parts of (1), on the other hand, don’t correspond to parts of the cube. Figure 1 also obeys HOLISM; any part of the image that you point at represents a color and a location simultaneously, and perhaps an aspect of the texture (e.g., a dot or two) as well. A single part of (1), however, represents the individual object (viz., ‘is’), while another part represents a color (‘blue’), another represents a texture (‘dotted’), and another represents a shape (‘cube’). (1) acquires its accuracy conditions from the mode of combination of these separate constituents, while Figure 1 acquires its accuracy conditions from the way features are holistically bound in each part, together with the spatial arrangement of those parts (Kosslyn et al. 2006; Fodor 2007). ese two principles are not arbitrarily related. Given an isomorphic relation (i.e., one- to-one correspondence) between parts of a representation and parts of the scene, any part of the representation that encodes a feature such as color will also correspond to a part of the scene. In that case, that part of the representation will not merely represent the color (as a single word or concept might) but will also minimally represent a location in the scene. Any other features instantiated at that part of the scene, insofar as they are represented in the icon at all, will also be represented by the same part that represents the color. It is possible that there may be representations that satisfy PARTS but fail to have a one-to-one correspondence between parts of the representation and parts of the scene (i.e., fail to be isomorphic to what they represent). ese non-isomorphic representations may satisfy PARTS without satisfying HOLISM. But paradigm cases like Figure 1 seem to have such a one-to-one correspondence and, not coincidentally, are accurately described by both PARTS and HOLISM. I will restrict use of the term ‘icon’ to these sorts of isomorphic representations. roughout cognitive science iconic mental representations that satisfy PARTS and HOLISM are posited to explain a variety of phenomena, such as core cognition (Carey 2009, 452; 459) and indeed iconic memory (Dretske 1981, 149; Fodor 2007, 112ff). Perhaps the most influential example, however, is mental imagery, where phenomena like mental rotation and the “scanning” of mental images are explained by appeal to iconic mental images (Kosslyn 1980; 1994; Kosslyn et al. 2006; Pearson et al. 2015). While the use of icons to explain imagery phenomena is controversial (Pylyshyn 2003), so-called “iconophilic” explanations typically appeal to both PARTS and HOLISM. In Shepard and Metzler’s (1971) famous mental rotation experiments, for example, participants were presented with a pair of objects and indicated whether they were two differently shaped objects or the same object at two different orientations (Figure 2). e authors found a nearly linear correlation between degree of rotation and reaction time—that is, the amount of time it took participants to correctly identify a match between two instances 6 of the same object (as in Figure 2) increased as a function of the degree of difference in orientation. Shepard and Metzler proposed that participants engaged in “mental rotation,” a manipulation of a mental representation of the object with stages corresponding to the stages of the physical rotation of the object. is description of the phenomenon posits a “second- order isomorphism” (Shepard 1978, 131) whereby relations among elements of the computational process correspond to relations among elements of the physical process. Figure 2—Example of a matching pair of stimuli from Shepard & Metzler (1971) One can offer an iconophilic explanation of this second-order isomorphism. If parts of a representation correspond to parts of the object and features are bound holistically in the representation, then the shape of the object is not represented separately from other features such as its orientation. e shape of the whole object is encoded by means of parts of the representation that correspond to parts of the object; parts of the representation encode parts of the shape at particular locations. e arrangement of these parts (corresponding to the arrangement of spatial locations) that encodes the entire shape thus cannot fail to encode its orientation. e resulting holistic binding of shape and orientation entails that a participant cannot access the object’s shape independently of its orientation, which hampers the ability to recognize a match in shape across different orientations. Instead the participant must perform some operation to transform the represented orientation to match that of the other object in the pair, at which point the participant can identify a match (or not) between orientation- bound shapes. is can be accomplished by performing an operation functionally analogous to physically rotating the object, wherein the image runs through intermediate orientation values until reaching an upright orientation that matches that of the other object.2 Since 2 One might wonder why the system would need to run through intermediate orientation values instead of simply changing the orientation immediately to a specific value. Since the orientation of the object is holistically bound up with other properties like shape, the capacity to iconically represent some object at an arbitrary orientation presupposes the capacity to represent its shape at that orientation—but the very point of changing the orientation is to acquire the capacity to represent its shape at the desired orientation. Without this capacity the system could instead implement a stepwise algorithm for representing an object at adjacent degrees of orientation (i.e., a mental-rotation algorithm), thereby moving in a stepwise fashion from the initial orientation to the desired orientation. e capacity to implement this algorithm wouldn’t require the ability to represent objects at arbitrary orientations but only adjacent ones, which would be much more computationally feasible—especially assuming 7 running through a greater number of orientation values requires more computational steps and therefore more time, one would predict that reaction time would be a function of degree of rotation. Appealing to icons that satisfy PARTS and HOLISM thus supplies a principled explanation of mental rotation. e debate about whether iconic format should be invoked to explain imagery phenomena has been long and hard fought (see, e.g., Pylyshyn 2002). Despite its controversial nature, the mental imagery debate provides a useful case study of iconicity as it figures in scientific explanation. I’ll assume in what follows that an icon-based explanation of iconic memory should follow the same principles, and is therefore committed to both PARTS and HOLISM. §3. Iconic memory e original experiment that motivated the existence of iconic memory was due to George Sperling (1960). Sperling presented participants with rows of letters (e.g., three rows of three letters each) and asked them to report as many letters as possible. Participants stored an average of about four letters, suggesting that the report was constrained by VWM. In another condition, after the presentation of the letters Sperling cued an individual row for participants to report. In this “partial-report” condition subjects could report nearly all the letters in the cued row (this advantage is sometimes called “partial-report superiority”). Since the cue occurred after the presentation of the letters, so the classic explanation goes, subjects must have stored all or nearly all the letters such that a particular subset of them could be attended in response to the cue and thereafter represented in VWM. In his influential textbook Cognitive Psychology, Neisser (1967) dubbed the earlier, informationally rich representation an “icon” and referred to the high-capacity visual memory store in which it is housed as “iconic memory.” Iconic memory has a shorter duration than VWM—Sperling found that partial-report superiority went away after about a quarter-second, whereas information can be stored in VWM for several seconds or even longer.3 e basic effect a high degree of featural similarity between representing an object at one orientation and an adjacent one. See Quilty-Dunn & Mandelbaum forthcoming for discussion. 3 e difference in duration between iconic memory and VWM is a complicated issue. More recently, Landman et al. (2003) found that retro-cuing a particular item in a display and asking subjects whether that item changed its orientation remained effective (i.e., showed a higher capacity than “post-cuing”, or introducing a cue after the probe display appears and thereby tapping into VWM) for 1.5 seconds. Sligte et al. (2008) found that even retro- cues that appeared four seconds after the display remained more effective than post-cues. Sligte et al. posit an intermediate “fragile visual short-term memory” store. I will generally talk about iconic memory in a way that is 10 their various features such as color and shape “spontaneously and free of cost” (Bronfman et al. 2014, 1398). ough there is significant controversy about whether this holistically encoded information for each item is consciously experienced (Block 2014; cf. Phillips 2016; 2018; Ward et al. 2016), my argument here is simply that it is encoded and stored and this fact is best explained by appeal to iconic format. It is possible that partial-report superiority could be explained by massive parallel processing that delivers discursive representations into a very large memory store. is explanation is inelegant in that there is no independent reason to think that discursive representations can be stored in such high capacity, particularly given the fact that there is a comparatively small item limit on discursive representations in visual and verbal working memory systems (e.g., Cowan 2001).4 It would also be hard pressed to explain why properties of uncued items (such as color in the Bronfman et al. 2014 experiments) seem to be encoded automatically. One could add the hypothesis that parallel processes automatically output discursive representations of properties of uncued items, but this addition seems more like an ad hoc epicycle rather than an independently motivated hypothesis. Moreover, icons whose structure mirrors the structure of the scene can implicitly encode many more features; e.g., an array of pixels that are ordered spatially can also thereby encode edges and therefore contours of shapes without adding additional symbols. A discursive model of iconic memory would have to posit that these properties are all represented by means of discrete symbols, thus positing a psychologically implausible explosion of discursive symbols.5 ere may also be independent neurobiological evidence in favor of the thesis that iconic memory stores perceptual icons. e sensory “informational persistence” (Coltheart 1980) that underlies iconic memory utilizes early, retinotopically mapped visual cortical areas of the brain (Duysens et al. 1985; Irwin & omas 2008). ese areas of the brain overlap with the loci of visual images (Pearson & Kosslyn 2015). Since mental images seem to be iconic, there is therefore some independent reason to think that iconic memory stores perceptual icons. is inference is not simply that since two representations are activated in overlapping brain areas, therefore they have the same format. In retinotopically mapped visual cortex, the parts of the cortex (such as columns of cells in V1) that instantiate representations of features 4 Note that this point does not require endorsing a “slot-based” over a “resource-based” model of working memory as long as the resources available to VWM limit its capacity relative to iconic memory (see Suchow et al. 2014). 5 anks to Sam Clarke for suggesting this last point. 11 (such as edge orientations) correspond to particular parts of the retina; thus the neural instantiations of representations of features coincide with neural instantiations of representations of particular locations in the scene that reflect light to particular parts of the retina. is sort of neural architecture seems to be exploitable for instantiating iconic representations that satisfy PARTS and HOLISM—e.g., that represent edges at particular orientations and particular locations in visual field.6 §4. The challenge from flexible resources Gross and Flombaum (2017) have recently provided a compelling defense of an alternative model of visual short-term memory. ey argue against the idea of any principled capacity limit on VWM, raising the possibility that iconic memory and VWM may not really be distinct. eir argument begins from the rejection of a classic picture of VWM, according to which its architecture consists of 3–4 discrete slots that can be “filled” with representations of one object each (Luck & Vogel 1997; Zhang et al. 2008; Adam et al. 2017; Xu et al. 2018). Gross and Flombaum argue that instead that VWM is governed by a continuous resource that can be differentially allocated to represented objects and features, thereby improving storage of those objects and features.7 Moreover, they argue that objects and features are represented by means of a probability density. Allocation of resources may improve storage by increasing the probability that a particular object is present or that it has some feature. Gross and Flombaum use this picture to argue that iconic memory may not in fact have a higher capacity than VWM. Instead, iconic memory might involve a relatively flat probability density (i.e., low probabilities assigned to the presence of any particular feature). e role of the cue after cessation of the stimulus may be to allocate resources to a particular 6 Contra Clark (2009) and Kosslyn (1994), the exploitability of retinotopy for instantiating iconic format does not suggest that iconic representations should be reductively characterized in neural terms. As Clark (2009) points out, the neural properties of V1 don’t perfectly satisfy PARTS. But while Clark takes this to undermine that V1 instantiates iconic representations, we should instead hold that iconicity is a functional, psychological-level notion rather than a neural one. 7 e metaphysics of memory resources is murky. For present purposes, what matters is that, in interpreting a number of highly noisy signals, one can reduce the noise on some signal and store the result only at the cost of failing to reduce noise for other signals—one cannot reduce noise on all incoming signals at once and store all of them independently. e fact that noise reduction somewhere precludes noise reduction somewhere else captures the essence of the idea of a flexible memory resource; it may arise due to constraints on the normalization of firing rates in neural populations (Bays 2015) or some other neural-level limitation. 12 subset of the information represented, which would shift the probability density to be high for cued features and much lower for uncued ones.8 e basic assumption of this model is well-supported: VWM is not simply an array of non-competing object slots, but rather constitutes a flexible resource that can be allocated across different objects (Bays & Husain 2008; Ma et al. 2014; Suchow et al. 2014; Bays 2015; Schneegans & Bays 2016; Park et al. 2017). Crucially, the flexible resource that underwrites VWM storage modulates the precision of stored items. For example, the difference between storing one item and storing four is not simply a matter of filling object-specific slots that encode features like color and orientation. Instead, storing more items causes a decrease in the precision of represented features; subjects’ responses on a continuous color wheel, for instance, will still be roughly accurate but will be noisier (Bays et al. 2009). Taking up VWM resources by adding another item causes a decrease in the resources allocated to the other items, resulting in a decrease in precision for stored features across objects. e “flexibility” of VWM resources consists in the fact that a common resource base is allocated across different objects and feature domains. Gross and Flombaum construe the modulation of resources in terms of probability densities. Allocating more memory resources to an item increases the represented probability (and reduces the estimated standard deviation) that the item has some feature. Gross and Flombaum’s probabilistic model, and the resources-not-slots picture it exploits, raise two salient problems for an iconic model of iconic memory. First, they argue that iconic memory may not in fact have a higher capacity than VWM, which challenges a fundamental motivation for distinguishing the two stores and positing iconic format in iconic memory. Second, they suggest that iconic memory may not even store actual representations at all. Instead, they suggest, partial report superiority may simply arise by enhancing the earliest, pre- representational stages of perceptual processing rather than cuing already-stored full-blown perceptual representations. Gross and Flombaum cite evidence (e.g., Bays & Husain 2008) showing that increasing the number of items to be remembered in VWM causes a decline in precision. is suggests that the limitation on storage in VWM is not a sharp, item-based capacity limit but is rather due to the allocation of a flexible resource shared across items. If VWM does not in fact have a sharp, item-based capacity limit, then the argument that iconic memory exceeds VWM in 8 Gross and Flombaum also argue that Sperling’s (1960) results might be explained in terms of hierarchical processing from explicitly represented shape features to full-blown letter representations. I assume that iconic memory represents analog magnitudes and grant for the sake of argument that letter identities are not represented as such prior to VWM. 15 eight objects—fits uncomfortably with the idea that the number of objects imposes any sharp limit on VWM capacity.9 It is still possible that iconic memory can store even more items. If Sperling’s (1960) and Phillips’ (1974) results are taken at face value, then iconic memory may store dozens of items. VWM may not be able to store that many items even with extremely low precision. Even if there is no sharp item limit on VWM, there may be a qualitative difference in the number of items explicitly stored with some non-zero probability estimates. It is not clear that results showing storage of dozens of items in iconic memory should be taken at face value, however. is brings us to Gross and Flombaum’s second challenge, viz., that the effects of early retro-cuing may simply modulate the pre-representational registration of sensory information. According to this challenge, the earliest stages of post- retinal processing do not yet involve genuine mental representation, but instead constitute a non-representational registration of proximal stimulation that is at most a mere elaboration of retinal transduction (Burge 2010, 315ff). In that case, the cue in Sperling-type experiments affects “what gets represented (consciously or unconsciously) in the first place” rather than “the selective transfer of representations one already has” (Gross & Flombaum 2017, 365–366). While there may be a great deal of information registered at this early stage, it does not involve genuine representation, and therefore does not actually require positing a memory store that houses representations (iconic or otherwise). According to this challenge, results purporting to show iconic storage of dozens of items (e.g., Phillips 1974) don’t actually show anything about storage capacity. Instead, they show that the transition from rich sensory registration to 9 Adam et al. (2017) showed subjects six items and had them report color or orientation for all six objects, one by one. ey found that subjects remembered three items and gave answers for the remaining three that were uniformly distributed among possible answers, strongly suggesting guessing rather than merely imprecise storage. is result suggests a hard cap of three items. It’s not immediately obvious how to render this evidence consistent with the evidence from Schneegans and Bays that subjects store as many as eight items with variable precision. One salient difference is that Adam et al. asked subjects to report the color of each square on a color wheel using a mouse, while Schneegans and Bays simply had subjects use their finger to move colored test items (e.g., a red item appearing in the middle of the test display) to corresponding locations (e.g., where the red stimulus had appeared in the original memory display). e relative ease of Schneegans and Bays’ task may be responsible for the difference. is explanation is admittedly hand-wavey, however, and perhaps hard to square with Adam et al.’s finding that the result is not due to subjects’ simply reporting the three best remembered items first and subsequently losing less precise memories of the remaining items (see their Experiment 2, in which the order of report is randomly determined by the computer). Perhaps the best model of working memory resources will entail (i) that resources are doled out in object-specific as well as feature-specific ways, and (ii) that doling out an ordinary amount of resources to three items exhausts remaining resources, creating a virtual limit that can only be overcome through drastically lowering precision on each item and using an extremely easy behavioral measure like Schneegans and Bays’ (2016). Exploring this sort of hybrid model would require a separate paper. 16 genuine perception—and, only thereafter, storage in short-term memory—can be affected by a cue. How can we determine whether a piece of information is actually encoded in a genuine mental representation as opposed to the registration of the stimulus? Representations are the elements of computational operations; the transition from registration to genuine representation involves the transformation of stimulus information into a form that can be computed over by mental processes. erefore, showing that a piece of information is computed over in perception would provide some reason to think that it is genuinely represented rather than merely registered. Another method would be to show that the information does not merely concern the low-level physical energies that are transduced (Pylyshyn 1984, Ch. 6), but instead pertains to higher-level perceptible kinds such as faces, objects, or causal relations. Since higher-level properties aren’t available in the proximal stimulus, it is plausible that such properties must be represented rather than merely registered. We can therefore look for an answer to Gross and Flombaum’s challenge by seeing (i) whether information in early visual processes concerns large number of items in respect of higher-level properties and (ii) whether that information functions directly as an input to later visual processes. If evidence for this sort of early representation exists, then we thereby have independent evidence that early retro-cuing can modulate pre-existing representations rather than merely modulating the initial formation of perceptual representations, thus answering Gross and Flombaum’s second challenge. And if the number of items encoded by such representations is significantly higher than eight—the highest number of items Schneegans & Bays (2016) found could be stored in VWM at very low degrees of precision—then we have prima facie evidence that these early icons have a higher representational capacity than VWM, thus answering Gross and Flombaum’s first challenge. e experimental literature on ensemble perception provides relevant evidence. Ensemble perception involves extraction of statistical properties of collections of items, such as the average size of a group of circles (Ariely 2001) or the variation of different sizes (Solomon et al. 2011). In one common paradigm, subjects are shown a set of items (e.g., circles of varying sizes) and then shown a single probe item and asked whether or how it differs from the average of the set (e.g., whether it is bigger or smaller than the average). Subjects succeed at such tasks even when they are at chance in indicating whether the probe item was the same size as any individual item in the set. Notably, ensemble perception works on very large set sizes, such as 16 (Ariely 2001). Some have argued that ensembles are computed by sampling three or four presented items 17 (e.g., Myczek & Simons 2008). However, if ensembles were computed by sampling three or four items rather than computing over all items in parallel, then increasing the heterogeneity of sets should reduce accuracy. For example, if a set of 16 circles contains eight circles of one size and eight circles of another size, then sampling four circles and averaging them would likely produce a result close to the average of the set. If instead all 16 circles are distinct sizes, then it becomes more likely that an arbitrary sample will be skewed and thus that the average will be less accurate. Instead, Utochkin and Tiurina (2014) found that heterogeneity does not affect accuracy, suggesting that all items are computed over. Ensemble perception also works even when subjects are under working memory load (Epstein & Emmanouil 2017). ese results suggest that large numbers of items are explicitly represented in early stages of visual perception prior to storage in VWM. Moreover, while Gross and Flombaum’s resource-based model allows that there may be no limit to the number of objects in VWM, it still predicts that performance on tasks that make use of stored objects should decline as the number of objects increases. Adding more items requires more VWM resources, lowering precision across items and thereby damaging performance. Robitaille & Harris (2011), however, found that increasing the number of items actually increases accuracy and speeds up reaction time in estimating average size and orientation. e fact that ensemble perception improves with more items suggests that, unlike later representation in VWM, there is virtually no cost to adding more items for the early representations that function as inputs to ensemble computations. Indeed, Dakin’s (2001) work on averaging of orientation involves displays with as many as 1,024 items. e ensemble perception evidence cited so far pertains to size and orientation. Size in particular may not be the sort of property that is transduced as such and thus may require genuine representations. Indeed, Whitney and Yamanashi Leib (2018) take ensemble perception of size to be notable given its status as a “mid-level” property. Nonetheless, one might still insist that ensemble computations of these properties could operate over mere sensory registrations. However, there is a wealth of evidence showing that ensemble perception operates not only on low-level features but also on higher-level properties. For example, the same sorts of paradigms discussed above have been used to show ensemble perception of the average happiness or sadness in a set of facial expressions (Haberman & Whitney 2007; 2009). is capacity is even exhibited by subjects with “face blindness” who have impaired processing of faces (Yamanashi Leib et al. 2012). In order to compute the average of a set of faces, it is not 20 back against Gross and Flombaum’s suggestion that iconic memory and VWM may not differ in capacity. It is still logically possible that VWM can store just as many items as iconic memory, albeit with sufficiently low precision that performance fails to be accurate. For Gross and Flombaum, this raises the possibility that we should simply do away with the idea of multiple short-term memory stores in vision: [W]e have challenged the claim of successive stores of declining capacity. One can reject this claim by rejecting only the claim of declining capacity, leaving in place the claim of successive stores. But why do so, rather than just posit transitions from noisy signals to probabilistic representations, without any transition from a first such store to a second? e latter view is simpler and requires fewer resources. (Gross & Flombaum 2017, 382) I’ve argued that there is indirect evidence in favor of a genuine capacity decline from iconic memory to VWM. Putting capacity aside, however, focusing on representational format provides an independent way to distinguish iconic memory and VWM. Some of the evidence discussed above for early icons concerns high capacity (emphasizing PARTS), but some concerns the holistic representation of features (emphasizing HOLISM). Even if early icons fail to have a higher capacity than representations in VWM, they may still differ in format. One crucial question, then, is whether features are also represented holistically in VWM. If not, then we can draw a distinction between the early storage of holistic icons and the later storage of non- holistic discursive representations even if the former fail to encode more information than the latter. e evidence strongly suggests that representations in VWM do not encode features holistically (see Green and Quilty-Dunn forthcoming for a review). For example, Fougnie and Alvarez (2011) showed subjects colored triangles at various orientations. Subjects then indicated the color followed by the orientation (or vice versa) of a cued triangle after a delay period. In cases where subjects were very far away from the correct value on one dimension (i.e., when they lost information about that feature), they were nonetheless typically able to produce accurate responses on the other dimension. at is, storage of color in VWM doesn’t necessarily correlate with storage of orientation (nor vice versa). Very similar results were found by Bays et al. (2011). ese effects imply a lack of holistic binding in representations in VWM, and therefore suggest that representations in VWM have a discursive format and employ distinct symbols to represent distinct feature dimensions. Since these dimensions are represented by means of distinct symbols (i.e., not in an icon that satisfies HOLISM), the fact that one is lost should not be expected to tell you whether the other is lost as well. 21 One might object that an icon could encode features separately, as in a line drawing of a triangle at some orientation that carries no chromatic information. In that case, the effect may be due to an encoding failure rather than separate feature dimensions being represented by means of separate symbols. is possibility would preserve the iconicity of VWM since it could still be true that, when multiple features like color and orientation are successfully encoded in a representation of an object, they are holistically bound into an icon. However, Fougnie and Alvarez varied encoding times and found no difference in the separability of color and orientation. e effect is not due to encoding failure. Rather, even when both features are encoded into a representation in VWM, they are not holistically bound and can easily be lost separately. A discursive model of representations in VWM makes a further prediction, namely that separate feature dimensions might have separate memory stores. is prediction seems to be true. Wang et al. (2017) found that increasing the variation in color of an array of triangles diminished storage of the colors of the triangles but not storage of their orientations; and likewise, increasing the variation in the orientations of the triangles diminished storage for orientation but not for color (see also Wheeler & Treisman 2002; Olson & Jiang 2002; Markov et al. 2019). Green and Quilty-Dunn (forthcoming) appeal to this sort of evidence to argue that VWM consists of discursive object files that represent separate feature dimensions by means of separate representations. On their “multiple-slots” model, representations of features in the same dimension compete for storage in a dimension-specific “slot”. While resources can be allocated flexibly across objects and feature slots, resource competition is higher within feature- specific slots than across them. is model and the evidence that supports it require a discursive model of the constituents of VWM. e foregoing discussion points toward a format-based distinction between iconic memory (and its close relative, fragile visual short-term memory) on the one hand and VWM on the other by appeal to the iconic format of representations stored in the former and the discursive format of representations stored in the latter. One might argue that the relation between iconic memory and VWM involves a smooth change in probability density, wherein precision is increased for some subset of items and decreased for the rest. e smoothness of this transition may push against the idea of a sharp difference in format between two separate memory stores and instead suggest a picture closer to Gross and Flombaum’s. However, Pratte (2018) showed that information in iconic memory is either completely present or completely absent. Instead of smoothly decaying or transitioning into VWM, representations in iconic memory “die a sudden death” (Pratte 2018, 77; see also 22 Zhang & Luck 2009). A format-based distinction between iconic memory and VWM predicts just this sort of sharp discontinuity between information stored in the two memory stores. A representation in VWM is not a modulated version of a representation from iconic memory. It is a distinct token vehicle with a distinct representational format. is format-based distinction between iconic memory and VWM fits well the mainstream capacity-based distinction between these memory stores, but it does not logically require it. e holistic character of feature binding in iconic memory and the non-holistic character of feature binding in VWM suggest a distinction that cuts across the capacity-based distinction targeted by Gross and Flombaum. And ensemble perception provides evidence for high-capacity early representation that escapes Gross and Flombaum’s methodological concerns, suggesting that the PARTS principle is true of the elements of iconic memory but not for VWM. ere is very good reason to posit a distinction between iconic memory and VWM and to hold that iconic memory stores high-capacity icons. In other words, iconic memory is iconic. §5. Conclusion: Consciousness and Short-term Memory I noted at the outset that the question whether iconic memory stores perceptual icons has significant upshots for debates about consciousness and the perception–cognition border. As we’ve seen, the evidence strongly pushes in favor of an affirmative answer to this question. is provides a solid foundation for arguments such as Block’s (2011a) and Lamme’s (2003) for the thesis that phenomenal consciousness overflows cognitive access, since iconic memory does indeed store unconceptualized icons that store more information than discursive representations in VWM. It is still possible that consciousness only enters the picture in VWM. is position interacts in interesting ways with the perception–cognition border, however, given the hypothesis that iconic memory and VWM differ in their format. On a popular approach to the perception–cognition border pursued by Block (unpublished) and others (Dretske 1981; Carey 2009; Kulvicki 2015), the distinction between perception and cognition tracks the distinction between iconic and discursive formats. If this approach is correct, and if only representations in VWM are conscious, it follows that visual representations are never conscious.11 11 An anonymous referee for this journal suggests an alternate description: visual representations are only conscious when conceptualized. is re-description sounds less unacceptable than the formulation in the text. But it seems to me inapt. One could think of a representation’s being conceptualized as its being accompanied by a concept, in which case saying a conceptualized representation is conscious might mean that the representation, together with its accompanying concept, is conscious. But the circumstance described here is different. Supposing a format-based distinction between iconic memory and VWM, it follows that iconic 25 Burns, B., & Hopkins, A. (1987). Integral and separable stimulus structure in iconic memory: A replication and extension of Burns (1987). Perceptual and Motor Skills 64, 263–270. Byrne, A. (2005). Perception and conceptual content. In E. Sosa and M. Steup (eds.), Contemporary Debates in Epistemology (London: Basil Blackwell), 231–250. Carey, S. (2009). e Origin of Concepts. Oxford: OUP. Clark, Austen. (2009). Location, location, location. In D. Dedrick and L. Trick (eds.), Computation, Cognition, and Pylyshyn (Cambridge, MA: MIT Press), 281–302. Cohen, M.A., Dennett, D.C., & Kanwisher, N. (2016). What is the bandwidth of perceptual experience? Trends in Cognitive Sciences 20(5), 324–335. Coltheart, M. (1980). Iconic memory and visible persistence. Perception & Psychophysics 27(3),183–228. Cowan, N. (2001). e magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences 24, 87–185. Dakin, S.C. (2001). Information limit on the spatial integration of local orientation signals. Journal of the Optical Society of America A 18(5), 1016–1026. Dehaene, S. & Changeux, J-P. (2005). Ongoing spontaneous activity controls access to consciousness A neuronal model for inattentional blindness. PLoS Biology 3(5), 910– 927. Demeyere N., Rzeskiewicz A., Humphreys K.A., & Humphreys, G.W. (2008). Automatic statistical processing of visual properties in simultanagnosia. Neuropsychologia 46(11), 2861–2864. Dretske, F. (1981). Knowledge and the Flow of Information. Cambridge, MA: MIT Press. Duysens, J., Orban, G.A., Cremieux, J., & Maes, H. (1985) Visual cortical correlates of visible persistence. Vision Research 25(2), 171–178. Epstein, M.L., & Emmanouil, T.A. (2017). Ensemble coding remains accurate under object and spatial working memory load. Attention, Perception, & Psychophysics 79(7), 2088– 2097. Evans, G. (1982). e Varieties of Reference. Oxford: OUP. Fodor, J. (2007). e revenge of the given. In B. McLaughlin and J. Cohen (eds.), Contemporary Debates in Philosophy of Mind (Oxford: Blackwell), 105–116. Fougnie, D., & Alvarez, G.A. (2011). Object features fail independently in visual working memory: Evidence for a probabilistic feature–store model. Journal of Vision 11(12), 1–12. 26 Fougnie, D., Asplund, C.L., & Marois, R. (2010). What are the units of storage in visual working memory? Journal of Vision 10(12), 1–11. Goodman, N. (1976). Languages of Art: An Approach to a eory of Symbols. 2nd edition. Indianapolis: Hackett. Green, E.J., & Quilty-Dunn, J. (Forthcoming). What is an object file? British Journal for the Philosophy of Science. Gross, S., & Flombaum, J. (2017). Does perceptual consciousness overflow cognitive access? e challenge from probabilistic, hierarchical processes. Mind & Language 32(3), 358– 391. Haberman, J., & Whitney, D. (2007). Rapid extraction of mean emotion and gender from sets of faces. Current Biology 17(17), R751–R753. Haberman, J., & Whitney, D. (2009). Seeing the mean: Ensemble coding for sets of faces. Journal of Experimental Psychology: Human Perception & Performance 35(3), 718–734. Heck, R. (2000). Nonconceptual content and the “space of reasons.” e Philosophical Review 109(4), 483–523. Hubert-Wallander, B., & Boynton, G.M. (2015). Not all summary statistics are made equal: Evidence from extracting summary statistics across time. Journal of Vision 15(4), 1– 12. Irwin, D.E., & omas, L.E. (2008). Visual sensory memory. In S.J. Luck & A. Hollingworth (eds.), Visual Memory (Oxford: OUP), 9–41. Kosslyn, S.M. (1980). Image and Mind. Cambridge, MA: Harvard University Press. Kosslyn, S.M. (1994). Image and Brain. Cambridge, MA: MIT Press. Kosslyn, S.M., ompson, W.L., & Ganis, G. (2006). e Case for Mental Imagery. Oxford: OUP. Kulvicki, J. (2015). Analog representation and the parts principle. Review of Philosophy and Psychology 6(1), 165–180. Lamme, V.A.F. (2003). Why visual attention and awareness are different. Trends in Cognitive Sciences 7(1), 12–18. Landman, R., Spekreijse, H., & Lamme, V.A.F. (2003). Large storage capacity of integrated objects before change blindness. Vision Research 43, 149–164. Luck, S.J., & Vogel, E.K. (1997). e capacity of visual working memory for features and conjunctions. Nature 309, 279–281. Luck, S.J., & Vogel, E.K. (2013). Visual working memory capacity: From psychophysics to neurobiology to individual differences. Trends in Cognitive Sciences 17(8), 391–400. 27 Ma, W.J., Husain, M., & Bays, P.M. (2014). Changing concepts of working memory. Nature Neuroscience 17(3), 347–356. Mandelbaum, E. (Forthcoming). Seeing and conceptualizing: Modularity and the shallow contents of perception. Philosophy and Phenomenological Research. Markov, Y.A., Tiurina, N.A., & Utochkin, I.S. (2019). Different features are stored independently in visual working memory but mediated by object-based representations. Acta Psychologica 197, 52–63. McDowell, J. (1994). Mind and World. Cambridge, MA: Harvard University Press. McKelvie, S.J. (1995). e VVIQ as a psychometric test of individual differences in visual imagery vividness: A critical quantitative review and plea for direction. Journal of Mental Imagery 19(3-4), 1–106. Miller, G.A. (1956). e magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review 101(2), 343–352. Myczek, K., & Simons, D.J. (2008). Better than average: Alternatives to statistical summary representations for rapid judgments of average size. Perception & Psychophysics 70(5), 772–788. Neisser, U. (1967). Cognitive Psychology. Englewood Cliffs, NJ: Prentice-Hall. Olson, I.R., & Jiang, Y. (2002). Is visual short-term memory object based? Rejection of the “strong–object” hypothesis. Perception & Psychophysics 64, 1055–1067. Oriet, C., & Brand, J. (2013). Size averaging of irrelevant stimuli cannot be prevented. Vision Research 79, 8–16. Park, Y.E., Sy, J.L., Hong, S.W., & Tong, F. (2017). Reprioritization of features of multidimensional objects stored in visual working memory. Psychological Science 28(12), 1773–1785. Pearson, J., & Kosslyn, S.M. (2015). e heterogeneity of mental representation: Ending the imagery debate. PNAS 112(33), 10089–10092. Pearson, J., Naselaris, T., Holmes, E.A., & Kosslyn, S.M. (2015). Mental imagery: functional mechanisms and clinical applications. Trends in Cognitive Sciences 19(10), 590–602. Phillips, W.A. (1974). On the distinction between sensory storage and short-term visual memory. Perception & Psychophysics 16(2), 283–290. Phillips, I. (2011). Attention and iconic memory. In C. Mole, D. Smithies, & W. Wu (eds.), Attention: Philosophical and Psychological Essays (Oxford: OUP), 202–225.