Haptics and perception
After an introduction to the concepts and the theory of perception used, I would like to describe the haptic qualities of color and how this manifestation of light evokes very real tactile sensations of surface structures, spatial forms, temperatures, orientations, solidities and weights. The term perception itself refers to the haptic action of taking and evaluating, grasping and comprehending, grasping, considering and imagining. The Latin origin of the term “percipere” also contains the action of taking and receiving. Haptic perception refers to the sensory perceptions that enable us to localize and evaluate information about our own body and its environment based on the surface and depth sensitivity of the skin and the kinaesthetic sensitivity of joints, muscles and tendons.
Color describes the multitude of optical sensory impressions that arise when light falls on the retina of the eye. It encompasses the entire visible spectrum of a specific wavelength or a mixture of wavelengths and thus becomes the medium for all possible visual information. Each color causes us to see in a certain way, attunes our body to the atmospheric quality we experience and thus brings about a certain modification of the phenomenal body, as Merleau-Ponty calls it. The being of an object, he continues, can only be identified by passing through the open series of possible experiences of it. The human being as a perceiver is connected to the sensually perceived object and so color is revealed through our ability to see it, form through our ability to touch it. “…my gaze pairs with the color, my hand with the hard and soft, and in this exchange between the subject of sensation and the sensual there is no question of the one having an effect, the other suffering, the one giving the other its meaning. Without my inquiring gaze, my groping hand and before my body synchronizes with it, the sensual is merely a vague excitement.” (Merleau-Ponty, Maurice, “Phénoménologie de la Perception” 1945, source: “Phänomenologie der Wahrnehmung”, de Gruyter Studienbuch Berlin, 1966)
Correlations between haptic and visual perception
We obtain the visually perceptible information from the light around us through the exploratory movements of our eyes. Gibson explicitly speaks of a correspondence between the scanning of a surface with the hands and the visual scanning of an optical situation for its concise characteristics. (Gibson, James J.; “Die Sinne und der Prozess der Wahrnehmung” (1966), Verlag Hans Huber Bern,
1973 and Gibson, James J.; “Die Wahrnehmung der visuellen Welt” (1950), Beltz Verlag Weinheim und Basel
1973) If we touch an object in the dark, we perceive a sequence of touch stimuli and not a simultaneous pattern, as is the case in the visual situation. Only the constancy within the tactile experience gives us an impression of the simultaneous nature of the object. The sense of touch is not immediately apparent, as with sight, but is only revealed when we have sufficient information about the object. Nevertheless, the simultaneous perception of an object leads to an equivalence with its successive perception, if we can fully encompass a body. This means that the same information about the shape is recorded when we feel an apple, for example, by running our fingers over it or simply clasping it in our hand. However, most of the objects in our environment are larger than our hand space, so that we can only scan them successively or cannot reach them at all. This illustrates the power of the visual sense, which allows us to grasp haptic information that goes far beyond the tactile space, from the micro realm of molecular biology to the macro realm of exploring the universe. According to the philosopher Berkeley, seeing is touching with foresight. (Berkeley, George “Principles of Human Knowledge”, (first published in 1710) Oxford University Press 1999)
To this end, I would like to reproduce the physical, haptic sensations of a blind twelve-year-old boy about the nature of vision, which bring us even closer to understanding the expansion of tactile space through the visual sense: “Those who can see were in contact with me through an unknown sense that completely embraced me from a distance, followed me, penetrated me and to a certain extent dominated me from the moment I got up until I went to bed.” (Senden, M. von “Raum- und Gestaltauffassung bei operierten Blindgeborenen vor und nach der Operation”, Verlag von J. A. Barth Leipzig 1932)
The influence of the haptic system on visual orientation in space
Since our body is not a rigid body, but a system of moving parts held together by muscles, vision, cartilage and skin, we have a spatially differentiated experience of gravity. The receptors in all joints are crucial for the perception of the position of all our body parts at every moment of their movement. They transmit information to the brain about the angles of the limbs in their relative displacement to each other and thus enable perfect orientation of our position in space. The color perception of the environment correlates with the position of our body in space. We do not interpret a hidden meaning in the manifestations of color, but rather the meanings of our bodily being in the world are revealed to us through the medium of color. The orientation of our body always takes place anew after our awakening from sleep; we come to ourselves by orienting ourselves, perceiving our environment, experiencing our body. When lying down, our body is passive to a state comparable to paralysis, in which all limbs withdraw from perception, we doze, dream, sleep. If we direct our action towards lying down, this state of our body is activated, we perceive the ground, the heaviness of the limbs lying on it, the haptic properties of the surfaces such as the warmth or the textures, our body can ache, lie softly or feel hardness, we perceive the forms of the ground through the experience of the deformation of our body. The color makes the feeling of lying down visible in the experience of horizontality.
Every movement of our body is a movement towards or away from something, an orientation movement. By straightening up, we bring our body into balance, which strives upwards from lying down until we are kneeling, squatting, sitting and standing. We feel the heaviness of our body, the straightening of which requires a force that we perceive as an effort, the overcoming of the passivity of lying down into the activity of being upright in the world. The experience of standing, of finding our state of equilibrium, is visualized through the color in the experience of verticality. We stand within ourselves, building up an active but restrained tension from the heel to the crown of the head. In contrast, our locomotion is a continuous falling and rising movement that begins and ends when we are standing still, which gives rise to the visual experience of rising and falling diagonals. Standing vertically is associated with a standpoint from which we perceive our mostly horizontally spread out environment, from which it becomes our surroundings. The haptic character of our bodily perceptions (proprioception) briefly outlined here is always present in the process of visual orientation in the environment (exteroception).
The constancy of the perceptual impression
Although the optical image of the retina is constantly renewed, we perceive a stable and materially constant environment. Gibson sees the reason for this in the fact that we only perceive relations in the arrangement of things through the excited receptors of the retina and these remain constant, even if our body or our eyes move. Things therefore do not turn when we tilt our head, they do not slide away when we let our eyes wander over the panorama of the environment, they are not framed by the boundaries of our field of vision, they do not blur even if we do not constantly focus on them and they do not disappear when we briefly close our eyes. A constant image of the visual environment does not exist, physiologically speaking, either on the retina or in the brain (Newton’s theory), so that Gibson assumes that fixations do not have to be stored in short-term memory either, but that the constancy of the perceived image is generated exclusively via the invariants of the perceptual situation, via a structure of image relationships. (Newton, Isaac “Opticks” (Orig. 1704), Optik – Springer, Berlin 2001) These image relationships are synaesthetically experienced properties of the experienced space, so that the tactile experiences confirm, falsify or correct the visual experiences, just as the other senses interact in this way.
The bipolarity of haptic perception
Our receptors are stimulated either by direct or indirect excitation, so that the various perceptual systems receive information directly or indirectly. According to this view, the sense of touch is a near sense that helps us to experience bipolar perceptual sensations in direct contact with the object of stimulation. We can touch an object, recognize its shape and its position in space or concentrate on the sensations of our body and register the movements of our hands or the temperature changes of our skin. The visual sense is a remote sense in which the object viewed must be at a minimum distance from the sensory organ in order to be perceived. Light is the mediator between the environment and the sensory organ, so that the physical existence of our environment cannot be verified by the eye. The perception of the inner states of our body is also reserved for the haptic sense.
Surface scanning and surface colors
In contrast to the monotony of tactile matter is the polymorphism of the tactile world, which also refers to the haptic qualities of color through its richness of forms and surfaces. Pressure, vibration and temperature stimuli can be differentiated according to their intensity, they interact to produce qualified sensory impressions, as do the color mixtures of light, and they provide an image of a tactile field that is comparable in essential points to the visual field. Optical and haptic fields always appear continuous to us, even though we rarely fully scan the interfaces or surfaces of an object with our fingers or the fovea. Katz also draws parallels between the surface colors and the haptic impressions of surfaces, the surface touches. The shape also fades into the background when perceiving a surface color. In most cases, the color impression fills the entire optical field, creating a two-dimensional appearance. When we walk, the ground does not appear corporeal to us, but as a modulated topographical layer, as a skin that delimits the mass below. Like the surface color, the surface feel can only provide information about the material qualities, but cannot capture the object as an object. Surface touches and colors offer the gaze and the body the resistance of a skin, so that our perception cannot go beyond it. The boundary layer retains its surface character up to the degree of yielding, which still leaves it with the basic property of impenetrability.
Spatial color and tactile perceptions
If we penetrate below the surface of the object with the organ of touch, a spatial perception of touch occurs, such as that which occurs when feeling through the bone through the skin and tissue. We can speak of spatial colors when we can see something through them, as is possible with transparent or translucent color appearances. Just as the tissue is superimposed on the bone, the spatial qualities of the colors also make the superimposition of objects and materials visible in the optical field. As in the haptic experience, there is also an area in the color experience in which perception becomes spatial. If a fog is only very faint, it hardly obstructs the view of the layers of space it envelops or obscures; if it becomes stronger, it reaches the point where its physical spatial effect becomes most visible; it stretches into the depths like a viscous mass. If it becomes even more impenetrable to the eye, the impression of its spatiality disappears until it reveals a closed, two-dimensional character as a homogeneous layer of color. The haptic experience of fog does not lead to the impenetrability of the spatial layer, but rather the course of the acoustic experience. However, the movement of our body in space involves a variety of spatial tactile experiences, such as walking through water and mud or eating food of different consistencies. Spatial tactile and color perceptions become all the more intense the more physical the objects that lie behind the penetrated surface become.
The visual and tactile perception of surface structures
The perception of the tactile surface structure of an object corresponds to the relationship of its material properties to our body. All of its sensually perceptible qualities do not correspond to absolute values, but to relative values that are dependent on the sensitivity and materiality of our body. Touching it gives rise to ideas about its physical existence, which are consolidated by the perceived material properties. For example, we recognize the weight, thickness, roughness, elasticity, warmth or hardness of the object by moving our tactile organ towards the object with a specific pressure force adapted to its tactile properties. The combination of many properties in varying degrees of intensity results in an overall picture, as exemplified by the soft, warm and at the same time dry, supple smoothness of the silk fabric.
The visual surface structure can be seen in the modulations of the surface color and the body color, which I will come to later. Thus, the visual color impression of a gently falling and folded silk fabric creates a more vivid image of the haptic qualities and thus also of the fabric’s essence than a pure surface modulation would achieve. According to Katz, the quantity of possible surface modulations of a series of surfaces, which runs from smooth to rough, does not fall short of the grayscale modulations of a series from black to white. However, we lack suitable terminology for expressing the multitude of different surface structures in both the haptic and visual areas.
Visual surfaces achieve their tactile effect not only through colored pigmentation, but also through the modulation of the background, which is why we often experience their surface structures as skin-like textures. Physiologists consider the visual sense’s ability to recognize surface structures to be equivalent to shape recognition. The color space is constructed equally from the boundaries of colored patches and their surface color and structure. The contours mainly provide information about the spatial form and thus the meaning of the object, whereas the surface structures reveal essential information about the materiality of the phenomena.
In 1938, Binns showed in an experiment in which test subjects were asked to judge the degree of softness of fiber bundles that the results of the haptic and visual perception of fabrics can be comparably good. Textures are not perceived as individual shapes, but as surface structures and thus as material qualities. Our eyes cannot find a way to fixate a concise single form in the color field. We find it difficult to fix a point in the color field, because our gaze almost tirelessly sweeps over the material, creating the impression of different surface qualities, which have a specific roughness, softness, graininess or fibrousness. (Katz, David “Der Aufbau der Tastwelt”, Verlag J. A. Barth Leipzig 1925)
The influence of temperature on the perception of materials
In addition to the information from compressive strength, roughness and vibration, we always perceive the temperature of a material in relation to our body temperature when we touch it. This results in a temperature flow, either from the material to the skin or vice versa, which we perceive as warmth or cold. Every material known to us is associated with a specific memory temperature, which we also transfer to other materials in the process of perception if we can assume connections in the type of materiality on the basis of their other properties. Metals, glass and stone appear cool to us, whereas wood, fabrics and foams appear warm. If we recognize a material as wood, the temperature maintenance is also set to sense heat, even if we have never touched this type before. If test subjects are blindfolded, they can reliably recognize the properties of the materials they are familiar with by touch. Both temperature sensations and pressure sensations are crucial for verifying the tactile impression, in addition to touching the surface structures. Materials whose specific temperature was artificially altered could no longer be clearly identified. It is also interesting to note that heated metallic surfaces are perceived as warmer than wooden surfaces with the same measured temperature.
Color is visible to us in a spectrum that ranges from short-wave cold blue-violet light to long-wave warm orange-red light. Experiments on the perceived temperature have shown the temperature effect of color, for example, people in a room painted with blue paint compared to a room painted with red paint only perceived the same room temperature generated by heating at a difference of four degrees. If we change the color temperature of a material, the temperature impression or our ability to assign it to a memory color also changes, which impairs material recognition.
The influence of imagination in haptic and visual perception
Colors often remain associated with the perceptual situation beyond the perceptual experience; they are present in our memory as memory colors, just as we also have tactile ideas as memory tactile sensations. Memory stimuli and colors can have such a strong effect that they temporarily or even permanently alter our perceptual abilities. For example, the haptic effect of the color of a stone changes when we learn that its appearance is based on painted paper, we associate the tactile idea of paper and irretrievably lose the idea of the material-specific object qualities, such as coolness, hardness and heaviness.
The reproduction of a memory touch usually implies the touching hand, or at least a touchable part of our body. If we are to describe a material quality from memory, this becomes particularly clear in the tactile contact presented. The reproduction of a color impression leads, as Hering already made clear, to the memory of the color impression of a situation, which we look at and experience again in our memory. (Hering, Ewald “Zur Lehre vom Lichtsinne”, original 1878, reprint: VDM, Müller, Saarbrücken 2007)
Stereognosis and movement in visual and haptic perception
The haptic sense generates perceptions in the movement of our body, in which the kinematic form of the stimulus is transformed into the static effect of an object. In the tactile movement of our hand, we generate specific stimulus sequences that do not lead to the perception of a moving object, but rather reveal to us its existence in time, matter and space. The perception of color also only occurs through the scanning movement of the fovea centralis, the retinal area of our sharpest vision, although the constant movement of the eyes does not become part of the color appearance. In principle, static tactile and color perception is also possible, but in this case noticeably fewer object properties are perceived than in kinematic perception. The successive recording of object properties leads to a simultaneous spatial impression in which time is experienced as the expansion of the object in space, whether in surface or depth. Since the perception of a surface always includes the space that aligns it with the viewer, every tactile and color experience carried out in movement contains our awareness of being together in and with space.
The experience of moving objects is different, where the color and tactile media elude the gaze or the hand, change in front of it and strive towards it. Our haptic experience is always present in our visual experiences. When a tree bends in the wind, we also experience the elasticity and tension in the trunk when we look at it visually; when a branch bends, the color gradients tell us that it is about to break; almost every form of pressure and tensile load that affects the shape of the object, i.e. changes its spatial and surface color, can be experienced as an experience of tension. Experiences of tension can be traced back to the experience of gravity through our haptic system. In this way, our muscular system, in interaction with the bone and joint system and the skin system, registers every deformation of the body and every internal tension in terms of direction and strength.
Since the early twentieth century, there have been increasing attempts to research the influence of the viewer’s movement on spatial perception. Cornelius and Wulff derived requirements for architecture and sculpture from this. The spatial form of objects is revealed by walking around them and not looking at them from a perspective or even frontally, but by capturing them in a constant change of cinematic perspectives. Here too, stereognosis from the field of haptic spatial perception is helpful for observing the spatial effect of color through movement. Stereognosis describes the ability of our sensory organs to recognize spatial shapes and determine their position and weight. When our fingers encircle an object, it becomes tangible in its spatial form. We do not perceive the position of our fingers or the stimuli at the points of contact, but the position of the object in relation to the room, its weight and its physical spatial form. Similarly, when we move around an object, we have the visual experience of its spatial form, we understand its orientation and have an experience of weight, even if these visual achievements can only be an indirect form of cognition. Just as we distinguish the color body from the color ground in color space, we can speak of the tactile body and the tactile ground in tactile space.
The importance of vibration perception for the recognition of materiality
We scan the object by moving our eyes, whereby the visual organ constantly adapts to the light conditions. In the phases of movement we have an unstructured impression of color, which condenses in the phases of rest to a design impression of edges and surface structures. During the tactile process, we perceive while our tactile organs are moving, whereas in the resting phase we only receive a greatly reduced tactile impression. This fact can be recognized from the importance of vibration perception, which only arises during movement and plays a decisive role in tactile perception. In addition to the pressure-sensitive receptors, the receptive fields sensitive to vibration play a decisive role in tactile perception, as Katz’s experiments show. The vibrations caused by the roughness of the tactile object can be felt through an intermediary medium. From my practical work as a stonemason and stone sculptor, I still have my own experiences of the interaction of hand and eye. In this way, the material properties of the stone communicate themselves to the hand at rhythmic intervals via the vibrations of the tool. Layering, irregularities or cracks in the depth of the stone become noticeable long before they become visible to the eye. The roughness of the stone surface is so clearly present at all times via the tip of the tool that the gaze can already determine the course of the action in advance, as it is only the gaze that makes the modulations of the tactile surface present as a whole. It is worth mentioning that the haptic perception of the vibrations corresponds to the accompanying acoustic perception, although I cannot go into this in detail here. The experience of the vibration of the color impression becomes extreme when viewing interfering structures, which are referred to as beats in the acoustic field. There are also comparable experiences of resonances in our body, especially in the chest area, which are based on specific acoustic vibrations.
The transposability of color and key impressions
The differences in tactile surfaces are transposed by the introduction of intermediate media, such as thin gloves or cloths, which means they change in quality but retain their relationships to each other, resulting in an almost identical tactile image. Just as the transposition of harmonic music maintains the structure within the keys, the transposition of tactile qualities through intermediate media can also be understood as tactile constancy. Here, too, there is a comparable behavior in the area of visual perception, so the impression of color perception as a whole shifts when the sun is obscured by a cloud, for example, without all objects changing the relationship of their colorfulness to each other. We can also speak here of a change of mood or color constancy. The transposability ends when we give up the unity of the spatial impression by comparing colored surfaces in the interior with colored surfaces in the exterior or by touching a form with one hand through a glove and making skin contact with the other hand.
Closing words
Color contains only indirect information about materiality, but has also become the medium for communicating practical and aesthetic meanings, which are illustrated in images and writing. It allows us to extend the haptic experience of our body and our surroundings into the entire visible space. The bipolar character of tactile perception makes it the only form of direct cognition and experience of the world. Therefore, the question of how we would see without the ability to perceive haptically can only lead to meditation, to seeing without the body, to the creation of an idea of the pure spirit such as Descartes’ “Cogito ergo sum – I think, therefore I am”. Herder, on the other hand, brought the basis of all knowledge back into existence with his reputation: “I feel! I am!”. (Herder, J.G. “Vom Erkennen und Empfinden der menschlichen Seele”, in ders. Werke in 10 Bänden” hgg. J. Brummack, FFM 1998 B.4)
George Berkeley sums up the nature of our cognitive possibilities when he says: “What is palpable exists”. But what is the nature of feelings, which, as Hermann Schmitz puts it, can often be felt physically in directional spatial atmospheres? The human body space extends beyond the boundaries of the skin. Blind people who have regained their sight after cataract surgery are often unsure and confused about the color phenomena at first, often fall into a post-operative depression, darken their home in order to orient themselves in the usual way, which is reminiscent of Plato’s allegory of the cave. (Schmitz, Hermann “Begriffene Erfahrung”, Ingo Koch Verlag Rostock 2002) The groping vision into the distance through the colors of light holds an expansion of our cognitive ability beyond the limits of our body and gives the visual phenomena their spatial, temporal and material meaning for our existence.
Further reading “The development of spatial-visual competence”
