Haptics and perception
After providing an introduction to the terminology and the theory of perception used here, I would like to explain what constitutes the haptic qualities of color and how this manifestation of light evokes very real tactile sensations of surface textures, spatial forms, temperatures, orientations, firmness, and weight. The term “perception” itself refers to the haptic act of taking in and evaluating, of grasping and comprehending, of understanding, reflecting, and imagining. The Latin root of the term, “percipere,” also implies the act of taking in and receiving. Haptic perception refers to the sensory perceptions that, based on the skin’s surface and depth sensitivity as well as the kinesthetic sensitivity of joints, muscles, and tendons, enable us to locate and evaluate information about our own bodies and their environment.
Color describes the multitude of optical sensory impressions that arise when light falls on the retina of the eye. It encompasses the entire visible spectrum of a specific wavelength or a mixture of wavelengths and thus becomes the medium for all possible visual information. Each color causes us to see in a certain way, attunes our body to the atmospheric quality we experience and thus brings about a certain modification of the phenomenal body, as Merleau-Ponty calls it. The being of an object, he continues, can only be identified by passing through the open series of possible experiences of it. The human being as a perceiver is connected to the sensually perceived object and so color is revealed through our ability to see it, form through our ability to touch it. “…my gaze pairs with the color, my hand with the hard and soft, and in this exchange between the subject of sensation and the sensual there is no question of the one having an effect, the other suffering, the one giving the other its meaning. Without my inquiring gaze, my groping hand and before my body synchronizes with it, the sensual is merely a vague excitement.” (Merleau-Ponty, Maurice, “Phénoménologie de la Perception” 1945, source: “Phänomenologie der Wahrnehmung”, de Gruyter Studienbuch Berlin, 1966)
Correlations between haptic and visual perception
We obtain the visually perceptible information from the light around us through the exploratory movements of our eyes. Gibson explicitly speaks of a correspondence between the scanning of a surface with the hands and the visual scanning of an optical situation for its concise characteristics. (Gibson, James J.; “Die Sinne und der Prozess der Wahrnehmung” (1966), Verlag Hans Huber Bern,
1973 and Gibson, James J.; “Die Wahrnehmung der visuellen Welt” (1950), Beltz Verlag Weinheim und Basel
1973) If we touch an object in the dark, we perceive a sequence of touch stimuli and not a simultaneous pattern, as is the case in the visual situation. Only the constancy within the tactile experience gives us an impression of the simultaneous nature of the object. The sense of touch is not immediately apparent, as with sight, but is only revealed when we have sufficient information about the object. Nevertheless, the simultaneous perception of an object leads to an equivalence with its successive perception, if we can fully encompass a body. This means that the same information about the shape is recorded when we feel an apple, for example, by running our fingers over it or simply clasping it in our hand. However, most of the objects in our environment are larger than our hand space, so that we can only scan them successively or cannot reach them at all. This illustrates the power of the visual sense, which allows us to grasp haptic information that goes far beyond the tactile space, from the micro realm of molecular biology to the macro realm of exploring the universe. According to the philosopher Berkeley, seeing is touching with foresight. (Berkeley, George “Principles of Human Knowledge”, (first published in 1710) Oxford University Press 1999)
To this end, I would like to convey the physical, tactile sensations of a blind twelve-year-old boy regarding the nature of sight, which bring us even closer to understanding how the sense of touch is expanded by the visual sense: “Those who can see were connected to me through an unknown sense that, even from a distance, completely enveloped me, followed me, permeated me, and, in a sense, dominated me from the moment I woke up until I went to sleep.” (Senden, M. von, “Perception of Space and Form in People Born Blind Who Underwent Surgery, Before and After the Operation,” published by J. A. Barth, Leipzig, 1932)
The influence of the haptic system on visual orientation in space
Since our body is not a rigid body, but a system of moving parts held together by muscles, vision, cartilage and skin, we have a spatially differentiated experience of gravity. The receptors in all joints are crucial for the perception of the position of all our body parts at every moment of their movement. They transmit information to the brain about the angles of the limbs in their relative displacement to each other and thus enable perfect orientation of our position in space. The color perception of the environment correlates with the position of our body in space. We do not interpret a hidden meaning in the manifestations of color, but rather the meanings of our bodily being in the world are revealed to us through the medium of color. The orientation of our body always takes place anew after our awakening from sleep; we come to ourselves by orienting ourselves, perceiving our environment, experiencing our body. When lying down, our body is passive to a state comparable to paralysis, in which all limbs withdraw from perception, we doze, dream, sleep. If we direct our action towards lying down, this state of our body is activated, we perceive the ground, the heaviness of the limbs lying on it, the haptic properties of the surfaces such as the warmth or the textures, our body can ache, lie softly or feel hardness, we perceive the forms of the ground through the experience of the deformation of our body. The color makes the feeling of lying down visible in the experience of horizontality.
Every movement of our body is a movement towards or away from something, an orientation movement. By straightening up, we bring our body into balance, which strives upwards from lying down until we are kneeling, squatting, sitting and standing. We feel the heaviness of our body, the straightening of which requires a force that we perceive as an effort, the overcoming of the passivity of lying down into the activity of being upright in the world. The experience of standing, of finding our state of equilibrium, is visualized through the color in the experience of verticality. We stand within ourselves, building up an active but restrained tension from the heel to the crown of the head. In contrast, our locomotion is a continuous falling and rising movement that begins and ends when we are standing still, which gives rise to the visual experience of rising and falling diagonals. Standing vertically is associated with a standpoint from which we perceive our mostly horizontally spread out environment, from which it becomes our surroundings. The haptic character of our bodily perceptions (proprioception) briefly outlined here is always present in the process of visual orientation in the environment (exteroception).
The constancy of the perceptual impression
Even though the visual image on the retina is constantly changing, we perceive a stable environment that remains physically constant. Gibson attributes this to the fact that, through the stimulated receptors of the retina, we perceive only relationships in the arrangements of things, and these relationships remain constant even when our body or our eyes move. Therefore, objects do not spin when we tilt our heads; they do not drift away when we let our eyes wander across the panorama of our environment; they are not confined by the boundaries of our field of view, they do not blur even when we do not constantly focus on them, and they do not disappear when we briefly close our eyes. From a physiological standpoint, a constant image of the visual environment exists neither on the retina nor in the brain (Newton’s theory), so Gibson assumes that fixations do not need to be stored in short-term memory either, but rather that the constancy of the perceived image is generated exclusively through the invariants of the perceptual situation—through a structure of image relationships. (Newton, Isaac, *Opticks* (orig. 1704), Optics—Springer, Berlin 2001) These image relationships are synesthetically experienced properties of the perceived space, such that tactile experiences confirm, falsify, or correct visual experiences, just as the other senses interact in this way.
The bipolarity of haptic perception
Our receptors are stimulated either by direct or indirect excitation, so that the various perceptual systems receive information directly or indirectly. According to this view, the sense of touch is a near sense that helps us to experience bipolar perceptual sensations in direct contact with the object of stimulation. We can touch an object, recognize its shape and its position in space or concentrate on the sensations of our body and register the movements of our hands or the temperature changes of our skin. The visual sense is a remote sense in which the object viewed must be at a minimum distance from the sensory organ in order to be perceived. Light is the mediator between the environment and the sensory organ, so that the physical existence of our environment cannot be verified by the eye. The perception of the inner states of our body is also reserved for the haptic sense.
Surface scanning and surface colors
In contrast to the monotony of tactile matter stands the polymorphism of the tactile world, which, through its richness of forms and surfaces, also points to the haptic qualities of color. Pressure, vibration, and temperature stimuli can be distinguished by their intensity; when they interact, they produce specific sensory impressions, much like the color mixtures of light, and they provide a picture of a tactile field that is comparable in key respects to that of the visual field. Optical and haptic fields always appear continuous to us, even though we rarely fully explore the boundaries or surfaces of an object with our fingers or the fovea. Katz also draws parallels between surface colors and the haptic impressions of surfaces—that is, the tactile exploration of surfaces. Even when perceiving a surface color, form recedes into the background. In most cases, the color impression fills the entire optical field, creating a two-dimensional appearance. When we walk, the ground does not appear to us as a solid body, but rather as a modulated topographical layer—like a skin that delimits the mass beneath it. Like surface color, surface touch can only provide insight into the qualities of the material, but cannot grasp the thing as an object. Surface touch and color offer the eye and the body the resistance of a skin, so that our perception cannot extend beyond them. The boundary layer retains its surface character up to the degree of yielding that still leaves it with the fundamental property of impenetrability.
Spatial color and tactile perceptions
If we penetrate below the surface of the object with the organ of touch, a spatial perception of touch occurs, such as that which occurs when feeling through the bone through the skin and tissue. We can speak of spatial colors when we can see something through them, as is possible with transparent or translucent color appearances. Just as the tissue is superimposed on the bone, the spatial qualities of the colors also make the superimposition of objects and materials visible in the optical field. As in the haptic experience, there is also an area in the color experience in which perception becomes spatial. If a fog is only very faint, it hardly obstructs the view of the layers of space it envelops or obscures; if it becomes stronger, it reaches the point where its physical spatial effect becomes most visible; it stretches into the depths like a viscous mass. If it becomes even more impenetrable to the eye, the impression of its spatiality disappears until it reveals a closed, two-dimensional character as a homogeneous layer of color. The haptic experience of fog does not lead to the impenetrability of the spatial layer, but rather the course of the acoustic experience. However, the movement of our body in space involves a variety of spatial tactile experiences, such as walking through water and mud or eating food of different consistencies. Spatial tactile and color perceptions become all the more intense the more physical the objects that lie behind the penetrated surface become.
The visual and tactile perception of surface structures
The perception of the tactile surface structure of an object corresponds to the relationship of its material properties to our body. All of its sensually perceptible qualities do not correspond to absolute values, but to relative values that are dependent on the sensitivity and materiality of our body. Touching it gives rise to ideas about its physical existence, which are consolidated by the perceived material properties. For example, we recognize the weight, thickness, roughness, elasticity, warmth or hardness of the object by moving our tactile organ towards the object with a specific pressure force adapted to its tactile properties. The combination of many properties in varying degrees of intensity results in an overall picture, as exemplified by the soft, warm and at the same time dry, supple smoothness of the silk fabric.
The visual surface structure can be seen in the modulations of the surface color and the body color, which I will come to later. Thus, the visual color impression of a gently falling and folded silk fabric creates a more vivid image of the haptic qualities and thus also of the fabric’s essence than a pure surface modulation would achieve. According to Katz, the quantity of possible surface modulations of a series of surfaces, which runs from smooth to rough, does not fall short of the grayscale modulations of a series from black to white. However, we lack suitable terminology for expressing the multitude of different surface structures in both the haptic and visual areas.
Visual surfaces achieve their tactile effect not only through colored pigmentation, but also through the modulation of the background, which is why we often experience their surface structures as skin-like textures. Physiologists consider the visual sense’s ability to recognize surface structures to be equivalent to shape recognition. The color space is constructed equally from the boundaries of colored patches and their surface color and structure. The contours mainly provide information about the spatial form and thus the meaning of the object, whereas the surface structures reveal essential information about the materiality of the phenomena.
In 1938, Binns showed in an experiment in which test subjects were asked to judge the degree of softness of fiber bundles that the results of the haptic and visual perception of fabrics can be comparably good. Textures are not perceived as individual shapes, but as surface structures and thus as material qualities. Our eyes cannot find a way to fixate a concise single form in the color field. We find it difficult to fix a point in the color field, because our gaze almost tirelessly sweeps over the material, creating the impression of different surface qualities, which have a specific roughness, softness, graininess or fibrousness. (Katz, David “Der Aufbau der Tastwelt”, Verlag J. A. Barth Leipzig 1925)
The influence of temperature on the perception of materials
In addition to the information from compressive strength, roughness and vibration, we always perceive the temperature of a material in relation to our body temperature when we touch it. This results in a temperature flow, either from the material to the skin or vice versa, which we perceive as warmth or cold. Every material known to us is associated with a specific memory temperature, which we also transfer to other materials in the process of perception if we can assume connections in the type of materiality on the basis of their other properties. Metals, glass and stone appear cool to us, whereas wood, fabrics and foams appear warm. If we recognize a material as wood, the temperature maintenance is also set to sense heat, even if we have never touched this type before. If test subjects are blindfolded, they can reliably recognize the properties of the materials they are familiar with by touch. Both temperature sensations and pressure sensations are crucial for verifying the tactile impression, in addition to touching the surface structures. Materials whose specific temperature was artificially altered could no longer be clearly identified. It is also interesting to note that heated metallic surfaces are perceived as warmer than wooden surfaces with the same measured temperature.
Color is visible to us in a spectrum that ranges from short-wave cold blue-violet light to long-wave warm orange-red light. Experiments on the perceived temperature have shown the temperature effect of color, for example, people in a room painted with blue paint compared to a room painted with red paint only perceived the same room temperature generated by heating at a difference of four degrees. If we change the color temperature of a material, the temperature impression or our ability to assign it to a memory color also changes, which impairs material recognition.
The influence of imagination in haptic and visual perception
Colors often remain associated with the perceptual situation beyond the perceptual experience; they are present in our memory as memory colors, just as we also have tactile ideas as memory tactile sensations. Memory stimuli and colors can have such a strong effect that they temporarily or even permanently alter our perceptual abilities. For example, the haptic effect of the color of a stone changes when we learn that its appearance is based on painted paper, we associate the tactile idea of paper and irretrievably lose the idea of the material-specific object qualities, such as coolness, hardness and heaviness.
The reproduction of a memory touch usually implies the touching hand, or at least a touchable part of our body. If we are to describe a material quality from memory, this becomes particularly clear in the tactile contact presented. The reproduction of a color impression leads, as Hering already made clear, to the memory of the color impression of a situation, which we look at and experience again in our memory. (Hering, Ewald “Zur Lehre vom Lichtsinne”, original 1878, reprint: VDM, Müller, Saarbrücken 2007)
Stereognosis and movement in visual and haptic perception
The haptic sense generates perceptions in the movement of our body, in which the kinematic form of the stimulus is transformed into the static effect of an object. In the tactile movement of our hand, we generate specific stimulus sequences that do not lead to the perception of a moving object, but rather reveal to us its existence in time, matter and space. The perception of color also only occurs through the scanning movement of the fovea centralis, the retinal area of our sharpest vision, although the constant movement of the eyes does not become part of the color appearance. In principle, static tactile and color perception is also possible, but in this case noticeably fewer object properties are perceived than in kinematic perception. The successive recording of object properties leads to a simultaneous spatial impression in which time is experienced as the expansion of the object in space, whether in surface or depth. Since the perception of a surface always includes the space that aligns it with the viewer, every tactile and color experience carried out in movement contains our awareness of being together in and with space.
The experience of moving objects unfolds differently: here, the visual and tactile elements elude the gaze or the touch, transforming before us and reaching out toward us. In these visual experiences, our tactile experience is always present. When a tree bends in the wind, we sense the experience of elasticity and the tension in the trunk even through visual observation; as a branch bends, the color gradients signal an impending break, and almost every form of compressive or tensile stress that affects the object’s shape—that is, causing a change in its spatial and surface color—is experienced by us as a sensation of tension. Experiences of tension can be traced back to our experience of gravity through our haptic system. Thus, our muscular system—in conjunction with the skeletal and joint systems as well as the skin—registers every deformation of the body and every internal tension in terms of direction and intensity.
Since the early twentieth century, there have been increasing attempts to research the influence of the viewer’s movement on spatial perception. Cornelius and Wulff derived requirements for architecture and sculpture from this. The spatial form of objects is revealed by walking around them and not looking at them from a perspective or even frontally, but by capturing them in a constant change of cinematic perspectives. Here too, stereognosis from the field of haptic spatial perception is helpful for observing the spatial effect of color through movement. Stereognosis describes the ability of our sensory organs to recognize spatial shapes and determine their position and weight. When our fingers encircle an object, it becomes tangible in its spatial form. We do not perceive the position of our fingers or the stimuli at the points of contact, but the position of the object in relation to the room, its weight and its physical spatial form. Similarly, when we move around an object, we have the visual experience of its spatial form, we understand its orientation and have an experience of weight, even if these visual achievements can only be an indirect form of cognition. Just as we distinguish the color body from the color ground in color space, we can speak of the tactile body and the tactile ground in tactile space.
The importance of vibration perception for the recognition of materiality
We scan the object by moving our eyes, whereby the visual organ constantly adapts to the light conditions. In the phases of movement we have an unstructured impression of color, which condenses in the phases of rest to a design impression of edges and surface structures. During the tactile process, we perceive while our tactile organs are moving, whereas in the resting phase we only receive a greatly reduced tactile impression. This fact can be recognized from the importance of vibration perception, which only arises during movement and plays a decisive role in tactile perception. In addition to the pressure-sensitive receptors, the receptive fields sensitive to vibration play a decisive role in tactile perception, as Katz’s experiments show. The vibrations caused by the roughness of the tactile object can be felt through an intermediary medium. From my practical work as a stonemason and stone sculptor, I still have my own experiences of the interaction of hand and eye. In this way, the material properties of the stone communicate themselves to the hand at rhythmic intervals via the vibrations of the tool. Layering, irregularities or cracks in the depth of the stone become noticeable long before they become visible to the eye. The roughness of the stone surface is so clearly present at all times via the tip of the tool that the gaze can already determine the course of the action in advance, as it is only the gaze that makes the modulations of the tactile surface present as a whole. It is worth mentioning that the haptic perception of the vibrations corresponds to the accompanying acoustic perception, although I cannot go into this in detail here. The experience of the vibration of the color impression becomes extreme when viewing interfering structures, which are referred to as beats in the acoustic field. There are also comparable experiences of resonances in our body, especially in the chest area, which are based on specific acoustic vibrations.
The transposability of color and key impressions
The differences in tactile surfaces are transposed by the introduction of intermediate media, such as thin gloves or cloths, which means they change in quality but retain their relationships to each other, resulting in an almost identical tactile image. Just as the transposition of harmonic music maintains the structure within the keys, the transposition of tactile qualities through intermediate media can also be understood as tactile constancy. Here, too, there is a comparable behavior in the area of visual perception, so the impression of color perception as a whole shifts when the sun is obscured by a cloud, for example, without all objects changing the relationship of their colorfulness to each other. We can also speak here of a change of mood or color constancy. The transposability ends when we give up the unity of the spatial impression by comparing colored surfaces in the interior with colored surfaces in the exterior or by touching a form with one hand through a glove and making skin contact with the other hand.
Closing words
Color contains only indirect information about materiality, but has also become the medium for communicating practical and aesthetic meanings, which are illustrated in images and writing. It allows us to extend the haptic experience of our body and our surroundings into the entire visible space. The bipolar character of tactile perception makes it the only form of direct cognition and experience of the world. Therefore, the question of how we would see without the ability to perceive haptically can only lead to meditation, to seeing without the body, to the creation of an idea of the pure spirit such as Descartes’ “Cogito ergo sum – I think, therefore I am”. Herder, on the other hand, brought the basis of all knowledge back into existence with his reputation: “I feel! I am!”. (Herder, J.G. “Vom Erkennen und Empfinden der menschlichen Seele”, in ders. Werke in 10 Bänden” hgg. J. Brummack, FFM 1998 B.4)
George Berkeley sums up the nature of our cognitive possibilities when he says, “What can be felt exists.” But what is the nature of feelings which, as Hermann Schmitz puts it, can often be physically sensed in directional-spatial atmospheres? In this context, the human body-space extends beyond the boundaries of the skin. Blind people who have regained their sight after cataract surgery are often initially uncertain and confused by the colorful sights; they frequently fall into postoperative depression and darken their homes in order to orient themselves in the familiar way, which is reminiscent of Plato’s Allegory of the Cave. (Schmitz, Hermann, *Begriffene Erfahrung*, Ingo Koch Verlag, Rostock, 2002) This tentative vision into the distance through the colors of light holds the potential to expand our capacity for understanding beyond the limits of our bodies and endows visual phenomena with their spatial, temporal, and material significance for our existence.
Further reading “The development of spatial-visual competence”
