1. Introduction: the Phenomenon of Texture. In the visual realm, texture is color's sophisticated cousin. As such it has attracted investigation for many decades; see Raff, pp. 187 - 195 in Hendee and Wells (eds.) 1997, for a short review. Nevertheless discussion of this aspect of visual perception has remained confused, perhaps because sophisticated theories have been offered without any sufficiently clear formulation of the question that they address. Raff's opening remarks reflect this confusion:
'Texture is what Gibson would call the characteristic with respect to which a figure appears to be homogeneous...According to Haralick, texture can be thought of as an "Organized area phenomenon", which, when decomposable, has two basic components: one dimension is used to describe "primitives' out of which image texture is made, and the second dimension is utilized to describe the spatial distribution between these primitives in texture. Also described by Resnikoff, texture is not itself a microscopic property of surfaces; rather it is a statistical property of surface features....The present .. understanding of texture is mainly due to ... Julesz, who developed algorithms to construct synthetic textures with established properties and introduced parameters to describe the conspicuous features of texture. In 1981, Julesz wrote "Research with texture pairs having identical second-order statistics has revealed that the preattentive texture discrimination system cannot globally process third and higher order statistics, and that discrimination is the result of a few local conspicuous features, called textons. It seems that only the first order statistics of these textons have perceptual significance, and that the relative phase between textons cannot be perceived without detailed scrutiny by focal attention.'
A few more words on this literature are in order. The American Heritage Dictionary defines 'texture' as 'The appearance of a fabric resulting from the woven appearance of its yarns or fibers...a grainy, fibrous, rough, or dimensional quality as opposed to a uniformly flat, smooth aspect...the representation of the structure of a surface as distinct from color or form...distinctive or identifying character.' Attempts at more formal notions of texture are bound up with the development of computer vision systems that use texture-finding as a means of image analysis. Gibson (1952) indicates the primitive state of the discussion of texture prior to the development of strong computer influences. However, as seen in Haralick (1979), texture detection in engineered vision systems requires no consistent formal view of what texture is: in any application texture is whatever is detected by the texture-finding heuristic used in the application. So work in this engineering tradition has remained content to use the term 'texture' in common sense fashion. Julesz' famous work, extensively reviewed and elaborated in Resnikoff (1989) reflects deeper psychophysical concerns. However, the technique used, study of rotated figure fragments in regular arrays of isolated blobs, differs fundamentally from the approach used in this paper. It can also be said that since Julesz proposes a theory of texture detection prior to any formal consideration of what is to be detected, he remains over-much under the influence of the engineering tradition that Haralick reflects. In a late version of Julesz' evolving view, texture perception rests on detection of the statistics of quasi-colinearity, corners, end-stops, closed elements, granularity, and edge orientation in an image. After appropriate preparation we comment on this view in Section 4.
We begin by proposing a definition, analogous to that of perceptual color, of a major subclass of what can be called textures, namely the homogeneous textures. An area R of uniform color has the following property: (I) copy a subregion S of R, of any predetermined shape, e.g. a polygon, oval, or spline-bounded region. Move it elsewhere within R. You will observe that the part of R covered by the moved copy of S cannot be discriminated, i.e. is visually indistinguishable from the rest of R. This is the property on which the present paper focuses.
We pursue our investigation by presenting a series of interactive figures or 'Laboratories', starting with Interactive Laboratory 1 - What is texture? The interactive figure seen shows four areas each containing a texture, as more formally defined below. Begin by clicking on the "Remove Cover' button, which will remove the 'cover rectangle' overlying the large textured rectangle in the middle of the figure. Now drag one of the smaller rectangles of like texture into the large rectangle and release it. (All of the three smaller textured areas seen are draggable.) You will note that the dragged area sinks invisibly into the background, leaving no clue as to where it has been positioned. The crucial fact is that the area that has been disturbed in this way fails completely to reveal itself, even though painstakingly detailed comparison of the two figures (or binocular comparison between the 'before' and 'after' images of the large rectangle in a stereoscope) can eventually find the difference. This is the basic fact around which we develop our theory of texture. The present paper reviews and gives theoretical expression to the phenomenon which this experiment demonstrates. You can repeat it with each of the three smaller textured areas seen in the figure. The small textured figures remain present and draggable even after they have been moved into the large textured area and become invisible. You can find them by groping with the cursor, and then drag them to another position, or even out of the large rectangle. You will note that as the smaller rectangle and circles are being dragged through the large rectangle they remain visible, but as soon as motion ceases they disappear.
The interactive figure contains a bar near its bottom made up of thirteen small textured squares. By clicking on any one of them you will be able to repeat the experiment just described with a different texture. Evidently the visual property noted is general. We now describe it in more abstract terms.
Uniformly colored areas A have one other property besides (I) which completes their definition, namely (II): A contains no perceptible visual detail. Plainly, any area A having visual properties (I) and (II) must be uniformly colored, and vice-versa.
We call any area A having property (I) but (generally) not property (II) a uniformly textured area, and call the 'visual property' common to all its local subregions a homogeneous texture. This visual property is manifest in the fact, implicit in and equivalent to our definition, that if any two such subareas are shifted to make them overlap, no distinguishable boundary is formed.
A colored area A can have property (II; i.e. contain no detail) without having property (I). If so, it contains what is commonly called a (smooth) color gradient. If a subregion S of such an R is copied and then moved gradually, it will initially stand out only slightly from its background, but will gradually become more distinct as it moves. Textured areas can have the same property, i.e. any copied, gradually moving subregion may gradually become more distinct as it moves within the area from which it has been copied. A patterned area within which copied areas behave in this way is said to contain a texture gradient. Gradients of this kind are explored in later figures.
An area A which does not contain a texture gradient in this looser sense must contain elements which remain discernable even after small motions of subregions S containing them, whose perceived positions will therefore reveal the translation which has been applied to S. Such, for example, is the line or curve separating two regions of easily distinguishable fine texture.
Not all areas that would commonly be regarded as uniformly textured contain homogeneous textures in the strict sense of the preceding definition. For example, a uniform (or slightly randomized) rectangular mesh covering a rectangle R is not uniformly textured in our sense, since if we copy a circular subregion S of R and then shift it slightly the cut ends of the mesh reveal the shifted location of S. Patterns of this kind, whose subregions reveal displacement at their boundaries but not internally, can be called generalized homogeneous textures. They can be defined as areas which generate the same homogeneous texture irrespective of what small area is cut out of them and then distributed at random over a larger area to generate a homogeneous texture. Thus the texture we regard a tartan cloth as carrying is the texture that would be seen if a flat random pile of small swatches of the cloth were laid on a table-top. An interactive experiment illustrating this concept is given as Figure 3a below.
By clicking on one of the small textured circles appearing to the lower left of the interactive figure introduced above, you can change the texture of one of the draggable circles to the texture clicked on. Unless it is in an identically textured area, it will immediately stand out.This shows that textures can be the same (if they hide invisibly within each other) or different (if when one is placed within the other it remains clearly visible.)
By clicking on one of the small textured circles appearing to the upper left of our interactive figure,you can change the texture of all three the draggable circles to the texture clicked on. By dragging them you will see immediately that they stand out in any region of different texture.
The two multicolored 'color picker' areas seen on the right of our interactive figure serve to control the foreground and background colors of the large rectangle and two of the small draggable textured areas. Experimenting with these will show that the color of the geometric elements comprising a texture are aspects of the texture as we have defined it, since changing any of these colors will make an area stand out clearly within a background that has not undergone this change, whereas if two similarly textured areas undergo identical changes of element color together and one is initially hidden within the other it will not become visible.
Finally we note the four green buttons seen to the left of the figure. These control the appearance and action of the draggable 'cover' in the figure. This 'cover' is itself textured, and combines with the area beneath it on one of three ways (or disappears completely) depending on which button is clicked. It can combine either by appearing as a textured mask, or by blending, either as a 50-50 blend or as a 20-80 blend with the area below. In neither case does this make visible any texture difference that was previously invisible, or hide any difference that was previously visible. This shows that these blends and maskings respect the equivalence of images implicit in the notion of texture introduced above, i.e. can be regarded as algebraic operations in the space of textures. More will be said on this point below.
A small area A having the property that no discernable boundary results when a copy of A is overlapped with itself is called a local texture. Plainly any subarea of a region R of homogeneous texture is a local texture. Conversely, if we take any local texture and use it to tile a larger area R, this will fill R with a homogeneous texture, provided only that the tiling is not so regular as to create visible periodicities within R. If the local texture is that a subarea of an original homogeneously textured R', then R will have a texture indistinguishable from that of R' (or, rather, close to that of R', since the sampled subarea A of R' will have a visual character close to, but varying slightly in a random way from, those of R'). In this sense, every homogeneous texture is visually determined by any local area within itself.
Our next interactive figure allows you to verify the assertion made in the preceding paragraph by filling the area on the right by circles carrying the same homogeneous texture as the rectangular are seen on the left. Interactive Laboratory 2 - Tiling an area with patches of constant homogeneous texture. If you do this and compare the two resulting rectangles, you will see that they are not discernably different, even though you can be certain, having just composed the right-hand area, that the two figures are full of small details in which they differ. Compare two areas, the first containing a homogeneous texture generated by a random process of the type described in the next section, the other generated by manually (and casually) filling a square of the same size with copies of a circular region cut from Fig. 4. As seen, these two figures make almost identical visual impressions, even though they must obviously differ completely in fine detail.
The three small textured boxes at the bottom of this second interactive figure left let you repeat this experiment with any of the three sample textures shown.
2. Generation of Textures. Stationary random processes which act to fill two-dimensional areas commonly generate homogeneous textures. These can be processes which choose individual pixels according to some Markov rule, processes which place multiple copies of one or more small graphical elements at random positions in an enclosing rectangle, and processes which place such graphical elements at random while also blending them with their backgrounds. As it proceeds, random placement will often obscure the boundaries of each individual graphic used, and converge to a homogeneous appearance. If only one graphic, itself a local texture, is used in this process then convergence is immediate, i.e. the final texture emerges as soon as the rectangle in which the texture elements are being placed is covered. In this sense homogeneous textures are the invariant limit objects to which homogeneous random placement and growth process of many kinds will tend to converge.
Our third interactive figure allows you to verify the fact that random graphical overlay operations by random placement and blending of two image fragments. Interactive Laboratory 3 - Generating a homogeneous texture by scattering geometric elements randomly. Clicking on any of the texture elements on the right of the figure will scatter 50 copies of in within a rectangular area to the left. By doing this repeatedly, either using a single texture element repeatedly or varying between the texture elements shown in some predetermined way, you can see that sufficiently frequent repetition of more or less any process of random scatter will generate a homogeneous texture, in the sense defined above. You can use the 'Clear' button seen at the lower left to clear the texture formed by scattering and start again. You can use 'Copy', 'Blend 50%', and 'Blend 25%' buttons to vary the opacity of the texture elements scattered, thereby producing a range of textures varying in quality from 'hard' to 'soft'. You can use the buttons marked '50' and '500' to toggle the number of elements scattered between 50 (if you want to see how repeated scattering leads in the limit to the generation of an homogeneous texture) or 500 (if you want to move rapidly to the limit). Note that the three final geometric elements in the in the list shown at the right of our interactive figure are themselves textures, and that scattering them merely repeats the texture that they contain, but over a larger area.
It is the fact that random processes of scattering and growth ordinarily generate homogeneous textures in the limit that explains the ubiquity of such textures in nature. The growth of leaves, of hair, of grass, of mold, and the random positioning of tassels in shag rugs can all generate homogeneous textures. So can the random accumulation of sand grains of different colors, of pebbles of various sizes and colors on a riverbed, or reflections from chaotic ripples on a liquid surface, etc.
Our next interactive figure returns to a point discussed earlier, the derivation of homogeneous textures by random distribution of patches drawn arbitrarily from a generalized homogeneous textures. The column of selection squares at the extreme right of this figure show a variety of regular textures, and the column of squares just to the left of this shows tartans of the MacAlister, MacDonald, MacDougall, MacGregor, and MacLoughlin clans. By clicking on any of these squares you can distribute copies of it to random positions within a larger area, thereby forming an homogeneous texture from the regular texture originally given. Each of these patters is provided with a test swatch which you can drag (invisibly) into the area of texture generated to verify that it remains invisible no matter where it is dropped.
We now return to a second point discussed above, the notion of texture gradient, with which you can experiment using the following interactive figure. The figure offers a choice of two sample texture gradients, reached by clicking on one of the textured boxes seen at its lower left. The gradient in each of these runs from top to bottom and is represented by a progressive change of element size. Each texture gradient is accompanied by three draggable circular swatches drawn from sample subregions in it. By dragging these swatches to roughly the zone from which they came you will see that they disappear completely. Positioned elsewhere in the texture gradient they will differ enough from their surroundings to stand out recognizably.
Note that texture gradients can be created by smooth variation of any of a texture's determining parameters over the area that it occupies. Parameters for which this can be done include element size, color, degree of blurring, percent mix of various superposed elements, etc.
Textures an also be generated by setting the dots of a random field to black or white, or to one of several colors, in a random way according to some probabalistic rule. Our next interactive figure explores this possibility. Interactive Laboratory 4 - Generating a homogeneous texture by setting pixels on and off randomly. It allows generation of textures of two basic kinds. (a) fields of independent random dots, all of the same size, independently switched on/off with a stated probability p of being on. (b) fields of random dots, all of the same size, switched on/off by a Markov process which makes each dot in sequence identical to of different from its predecessor with a stated probability p. Once can also generate corresponding fields of colored dots. In textures of the first kind, dot colors are chosen independently at random. In textures of the second kind, a new color is chosen at random if and only the black/white Markov rule stated previously would change its color from black to white. Textures of the first kind are generated by clicking on the small black rectangle seen at the upper right of the figure; textures of the second kind are generated by clicking on the textured rectangle just beneath it. In both cases, dot size is set by clicking on one of the buttons at the lower left in the row labeled 'Dot Size',, and the color feature is toggled on and off by use of the 'Color' button. For textures of the first kind, the probability that a dot will have a non-background color can be set by clicking on one of the buttons at the lower left in the row labeled 'Prob'. For textures of the second kind, The buttons in the 'Prob' row control the probability that the next dot will have a color different from that of its predecessor, and four more values for this probability are available through the buttons at the upper right marked with numerical values.
3. Various Mathematical Properties of the Space of Textures.
3a. Algebra of textures; the affine structure induced by texture blending. Whenever two or more unrelated textures T1, T2 are combined in any fashion which acts uniformly over image points, the result will be a texture. This is because copying and repositioning of a subregion R of the combined texture T can be achieved by copying and repositioning the same subregion in each of the two component textures and then combining the two component textures as a final step. Since neither of the two component textures will take on a changed appearance as the result of the subregion motion applied to it, neither will combined texture T.
Examples of combinations to which this operation applies are: combine by choosing darkest or lightest pixel, or by choosing a pixel at random according to some Markov rule, by or by exclusive oring, addition, subtraction, or blending of pixel color intensities, or by selecting one color channel from T1 and another from T2. All these operations can therefor be thought of as applying to textures just as they do to images, allowing us to write expressions like darkest(T1,T2), lightest(T1,T2), c * T1 + (1 - c) * T2 (for the blend of two textures) etc. The same observation applies to pixelwise uniform operations f, like color mapping, which transform a single image, allowing us to write expressions like f(T1). These operations clearly define an 'algebra of textures'. Plainly any algebraic rule for such operations which applies to images also applies to textures. For example, the rule
d * (c*I1 + (1 - c)*I2) + (1 - d) * I3 = c * (d*I1 + (1 - d) I3) + (1 - c) * (d*I2 + (1 - d) I3)
for image blends carries over to texture blends.
Textures defined by special mathematical operations may have special properties in regard to certain of these mathematical operations and may form interesting subalgebras. Consider, for example, the texture R(p) formed by setting pixels randomly and independently black and white with probability p that a bit should be black. It is clear that if we combine pixels by taking the lighter of two coincident pixels the combination of R(p) and R(q) is R(p * q), whereas if the darker of two coincident pixels is taken the combination of R(p) and R(q) is 1 - R((1 - p) * (1 - q)). If instead we take the exclusive of two coincident pixels, the combination of R(p) and R(q) is R(p * (1 - q) + q * (1 - p)), since p * (1 - q) + q * (1 - p) is the probability that one, but not both, of the two pixels should be black. Thus, if we designate the exclusive or of two textures S and T by S @ T, we have R(1/2) @ R(1/2), that is, R(1/2) is an idempotent in the space of textures with this operation. Also R(p) @ R(p) = R(2p(1 - p)) = R(1/2 - 2(p - 1/2)*(p - 1/2)), so R(p) @ R(p) always has the form R(1/2 - q) for positive q, but converges rapidly to R(1/2). For example, starting at the texture R(1/10) we would see the successive exclusive-or textures R(0.18), R(0.30), R(0.42), and R(0.49). Similar considerations apply to the Markov textures M(p) obtained by reversing the next pixel in line with probability p (note that R(1/2) = M(1/2)). For much the same reason as above we have M(p) @ M(q) = M(p * (1 - q) + q * (1 - p)).
Our next interactive figure allows experimentation with algebraic texture combinations of the kind just described. Each of the pages selectable by the numbered buttons at the right of the figure contain four moveable textures and shows a designated algebraic combination of them in the area in which they overlap. Each figure contains various test swatches which can be dragged into the overlap area of the larger draggable textured rectangles. In the first page the topmost texture is R(1/2), which can be seen to reduce every black and white texture with which it combines by exclusive oring to itself. This is a consequence of the idempotency noted in the preceding paragraph. In the second page topmost texture is M(3/4), which though not very different from R(1/2) is different enough that removal of a few of the other textures from the mix makes a discernable difference, which does not happen with R(1/2) as you can see by experimentation. On the third page of this same figure the three lower textures are combined by selecting the darkest of overlapping textures, and on the fourth page by selecting the lightest pixels. This produces several quite different textures, as shown in the test swatches provided, but as usual combining the result texture with a topmost R(1/2) texture using exclusive or entirely obscures this difference. On the fifth page, three of the four textures shown are combined by blending, but the top R(1/2) texture still by exclusive or. Here the underlying textures are not as effectively hidden, since R(1/2) hides underlying black and white but not gray scale textures. On the sixth page all four textures ae combined by blending, and here the removal of the top R(1/2) texture has an easily noticeable effect.
Note however that combination of two closely related textured images in special positions can sometimes reveal detailed relationships not visible in either image alone. Consider, for example, combination of two identical black/white random-dot textures by choice of darkest pixel. In general, this will simply yield a darker, still uniform, random-dot texture. If, however, the images combined happen to be precise inverses of each other within some restricted region, the combined image will be uniformly black in this region, and therefore plainly not a homogeneous texture.
3b. Decomposition of a colored texture into its separate color planes. Since the operation which projects an image into its red (resp. green, blue) color plane is pixelwise and position invariant it maps homogeneous textures into homogeneous textures, and since the operation which recombines these three color planes into a single colored image is also pixelwise and position invariant it acts as a mapping of textures into textures. Thus colored textures can generally be regarded as being composed of three separate gray scale textures, one in each color plane. Our next interactive figure demonstrates this by allowing choice of any one of five homogeneous textures, each consisting of three color planes which can be dragged independently. Each of these sample textures is accompanied by a small rectangular swatch which can be dragged into the overlap area of the three planes to verify that the texture created by adding together the tree color generally planes depends only on their textures, not on their relative positioning. There is however one systematic exception to this principle. A few special color-plane arrangements generate textures differing from those seen when the color planes are positioned 'generically'. For example, a gray scale or a black-and-white texture will be without color only when its color planes are exactly superposed in their original position,and will be bichromatic rather than trichromatic only when two of these three planes are given precisely corresponding positions. You can verify this by positioning the color planes seen in the figure very carefully. One of the textures is provided with two subtly different test swatches, one taken for the texture seen when the three color planes are precisely aligned, the other in a generic position. You can verify that the former of these blends in most perfectly when the color planes have been realigned precisely, the other blending in best in all other situations.
From experiments of this type we can draw the conclusion that colored textures can always be regarded as superpositions of one, two, or three gray scale textures, precolored by multiplying their grey levels by a constant color coefficient.
It is worth noting that randomized colored textures change only slightly even if the number of colors used to compose them are changed drastically. This is shown by the following figure, which, reading left to right, shows two randomized colored textures composed using 256, 64, 16, and 8 colors. Note that no very distinct boundaries are formed.
3c. Texture blurring. A texture T is blurred by blending together multiple copies of T with small random offsets. The resulting texture is roughly independent of the offsets used if these are randomly selected and kept reasonably small. (One can even use all the offsets in a small range of, say 10 by 10 pixels; this range should be proportioned to the scale of the texture itself.) The following interactive figure allows you to see that the result of such a blend depends little on the precise choice of offsets involved. The figure contains thirteen copies a single 'Markov 1/10" random dot texture, which you can see by clicking on the 'Show 1' button at the bottom left of the figure. Each texture is provided with a long thin 'handle', ending in a dot which can be dragged to reposition the textured area. (These draggable dots are colored black-to-grey, and then dark green to light green, to indicate the front-to-back order of the blended copies that they control.) A draggable circular test swatch, showing the blend of all thirteen copies, is also provided. By positioning all the dots to make the rectangular textured areas roughly but not precisely coincident (move them into an irregular, reasonably tight vertical column) you will be able to see that the resulting blend depends little in the position of any of its component copies, and that the circular swatch is reasonably well hidden no matter where in the overlap area it is positioned. You can even drag a few of the blended copies completely out of overlap, to see that the blend produced depends little on the number of copies involved.
3d. Rotational and distortion-related properties. Although uniformly colored areas retain their color when rotated or expanded, textures plainly can change when they are rotated or reflected, and will always change under geometric expansion or contraction. A texture which does not change under reflection (resp. rotation) is said to be symmetric (resp. rotationally symmetric). The sixth and seventh textures (randomly placed circles) represented by the small squares in the first interactive laboratory introduced above are rotationally symmetric. Many of the others are symmetric under reflection about the horizontal and vertical axes, but the ninth texture (randomly placed right-hand semicircles) is plainly invariant under reflections about a horizontal axis, but not under left-to-right reflection.
Textures can be invariant under change of scale if they are either uniform colors or if their details are distributed fractally. But in this latter case they must contain arbitrarily large and small detail elements. Thus textures are generally not invariant under change of scale, and so each must be viewed at a specified scale. Our next interactive figure allows you to experiment with change of scale for each of the thirteen test textures introduced in our earlier figures.
4. Texture Detection. To use textures effectively in its edge-finding and object recognition operations, the visual system must detect the presence of homogeneous textures, discriminate between them so as to locate the boundaries along which one homogeneous texture gives way to another. and be able to identify objects by their characteristic textures. To find regions of homogeneous texture, one needs to take that part of an image lying within a moveable circle C of appropriate scale, and form some vector-valued functional F(I(C)) which is relatively independent of the position of C. Then the fact that F varies only slightly as C moves reveals the presence of a homogeneous texture, and the component values of F(I(C)) identify the texture.
The following considerations show how readily such functionals can be formed. Every homogeneous texture has a natural scale, which can be defined as the smallest radius such that any circular patch cut from the texture will contain a local texture from which an acceptable copy of the entire texture can be formed by tiling. The average color of such a patch will be close to invariant over the points of the texture, so the R, G, B intensities of this average color are a first set of three translationally invariant numerical texture parameters. Let RGB(T) denote the vector of these three parameters, as calculated for the texture T. All translationally invariant texture operations F, G,.. will map T into other textures F(T), G(T), .. for which we can calculate RGB(F(T)), RGB(G(T)), getting additional numerical invariants of T. We can also form RGB(F(F(T))), RGB(F(G(T))), RGB(G(F(T))),..., readily generating as many numerical texture invariants as we wish.
These abstract operations can readily be performed by populations of neurons. For this, we simply require a few retinotopically organized cell layers which respond differentially to intensity and color signals flowing in from retinal zones centered on their nominal position. Certain cells might then react to the least or most brightly illuminated input in their receptive field, others to center/surround differences, others slant, the presence of corners, motion, etc. Collectively these form transformed images like those that we have described abstractly as F(T), G(T), etc. An effective texture-recognition capability may only require that many textures be discriminated, not that textures be identified in any fixed way. Hence neural texture-discrimination mechanisms could readily evolve by random proliferation of populations of visual neurons which connect to pre-existing populations in uniform, but not necessarily specific, ways. The texture-discrimination system can then function as if it were a fourth, fifth,..color sense. Ultimately the whole texture-discrimination system must feed the elaborated ('super-colored') images it senses into edge-detection processes which prepare for shape-based object recognition.
Our next interactive figure illustrates these ideas. Its first panel shows 10 miscellaneous textures drawn for the collection used as examples in Figure 1. By clicking on any of the textured patches shown, you can produce blurred versions of the texture (the leftmost of the four transforms that are constructed) and of three local transforms of the original texture. These are the texture derived by combining the original texture with a two-pixels horizontally (resp. vertically) shifted version of itself by the lightest-pixels operation (the second, resp. last, transform from the left), and the exclusive or of the texture with itself after a shift by one pixel horizontally and one pixel vertically. These four transformed versions of the original image can be thought of as representing its averaged density, 'verticality', 'horizontally', and (reversed) 'blockiness' respectively. You can either run these image calculations by clicking on the patches shown, or move on to panel 2 of the interactive figure, which shows the outputs of each of the 10 possible runs. (Panel 3 shows this same data in pseudocolor, to give you another view of the degree to which our collection of four texture discriminators succeeds in discriminating between the collection of 10 textures shown.) Note that texture 8 is well discriminated from all the others by its high degree of 'blockiness', texture 3 from the others by its low 'horizontality', texture 2 among the first four (and from 10) by its high 'horizontality', and others by more particular combinations of attributes. However, since no edge detector is included among the three basic feature finders used in this experiment, nor are any of our detectors normalized to a standard average density of features, texture 2 and 10 ae not well discriminated, nor are 7 and 9 discriminated at all, aside from being seen as largely 'empty'.
By advancing to the fourth panel of the interactive figure, you can repeat the experiment described in the preceding paragraph for ten textured blocks containing horizontal boundaries between various of the textures already used. The results of all the 10 possible runs are summarized in the interactive figure's fifth panel. It will be seen that in most cases reasonably clear boundaries are detected in one or another of the processed images.
The discrimination technique just described can be improved very easily just by adding more basic discriminators to the set employed. One eay way of doing this is to process both an image and its reverse, since in some cases these will have significantly different properties. The seventh and final panel in the interactive figure shows the result of doing this for a few of the textures not well discriminated in the results shown in the second panel, e.g. the two 'circle' textures (textures 5 and 6 in the second panel). For example, texture 6 is not itself 'blocky', but its complement is more so.
4. Dimensionality of the space of textures. A complete psychophysical theory of texture, of the kind at which Bela Julesz' famous series of papers aims, would enumerate a complete and independent set of texture discriminators. Independence of discriminators can be demonstrated by exhibiting pairwise discriminable textures which realize every possible combination of discriminator values; completeness by the more difficult process of verifying experimentally that no two textures identical in all proposed discriminator values can be discriminated preattentively by the eye. The corresponding demonstration in case of color, i.e. experimental verification of color trichromacy, is greatly simplified by the physical fact that every perceptible form of light must be a linear mixture of wavelengths drawn from the visible spectrum. Knowing this, we can verify that the eye (a) can discriminate between any additive mix of three chosen primaries; (b) identifies every pure spectral color with some such mix; (c) identifies mixes of spectral colors with the corresponding mixes of their representations as mixes of primaries (at least after suitable allowance is made for 'response functions' presumably corresponding to the physical responses of the retina's three types of cones.
As noted above, Julesz has posited such a full set of discriminators, at last for the restricted case of black/white textures. His proposed discriminators are roughly: (i) average grey level ('first order statistic' of the image) (ii) local image autocorrelation ('second order statistics'); (iii) various less formally defined discriminators: 'edge orientation', 'quasi-collinearity', 'corners', 'end-stops', 'closed elements', and 'granularity'. To test this proposal we would first need to develop evaluation algorithms for the informally defined quantities which the proposal involves. By applying the discriminators to a variety of images we could then establish that they return approximately uniform values over the surface of many homogeneous textures. However, to test any proposed family of texture discriminators for completeness is much more difficult, since one must somehow establish that all efforts to find textures which the eye discriminates but the proposed family does not must fail. It is not clear that Julesz' texton family meets this challenge. For example, the semicircles texture S shown in Fig. is clearly distinguishable from its mirror image S2, but S and S2 have the same average grey level, density of local edges in all orientations, degree of 'quasi-collinearity' (whatever this may be), density of end-stops and corners, granularity, and density of closed elements. Moreover the autocorrelation of the image shown is not at all left-right asymmetric. These two textures are most easily distinguished by the average direction of their local element curvatures, a property which it seems hard to relate to Julesz' texton proposal.
In the same way, one can attempt to exploit the texture-combination operations described above to represent every discriminable homogeneous texture as a combination of suitably defined 'primaries'.
5. Motion Textures. Rapid reversal between black and white of all the pixels of a black and white texture produces a kind of 'chaotic motion texture' generating a perception that can be rather different from anything easily seen in the underlying static texture. The following animated figure shows this. It offers the choice of several frame rates for each of four animations: reversal of every other frame, blanking of every other frame, display every other frame of an unrelated image having the same texture, display every other frame of an image having a different texture. The eye discriminates surprisingly easily among all these percepts, even at frame rates of 30 fps. The 'reversal' percept seems the most active; the 'blanking every other frame' percept is seen as almost stationary; the 'display of an unrelated image' case has a much less intensive motion than the 'reversal' case.
These motion textures have been little study and may eventually reveal new facts about the visual system's motion handling subsystems.
Bibliography
Beck Perceptual grouping produced by line figures. Perception and Psychophysics 1967; 2:491-495
Bergen, T. and Julesz, B. Parallel versus serial processing in visual pattern discrimination. Nature 1983; 303:696-698.
Caelli, T. and Julesz, B. On perceptual analyzers underlying visual texture discrimination. Part I. Biological Cybernetics 1978; 28 (3):167-175.
Caelli, T. Julesz, B., and Gilbert, E. On perceptual analyzers underlying visual texture discrimination. Part II. Biological Cybernetics 1978; 29 (4):201-214.
Caelli, T. and Julesz, B. Psychophysical evidence for global feature processing in visual texture discrimination. Journal of the optical society of America 1979; 69 (5):675-678.
Hendee, W.R. and Wells, N. T. (eds.) The Perception of Visual Information (2nd. edn) 1997; Springer
Julesz, B., Gilbert, E., and Victor, J.D. Visual discrimination of textures with identical third-order statistics. Biological Cybernetics 1978; 31 (3):137-140.
Gibson,J.J. The Perception of the Visual World. Boston: Houghton Mifflin; 1952
Farrel, B. and Julesz, B. Perception of static directional flow fields. Perception 1989; 18 (2):155-172.
Fix, J. and Mahew, J. Texture discrimination and analysis of proximity. Perception 1979; 8:75-91.
Frisch, H.L. and Julesz, B. Figure-ground perception and random geometry. Perception and Psychophysics 1966; 1:389-398
Haralick, R. M. Statistical and structural approaches to texture. Proc. IEEE 1979; 67:768-804
Julesz, B. Visual pattern discrimination. IRE Transactions on Information Theory 1962; 8:84-92.
Julesz, B. Texture and visual perception. Scientific American 1965; 212(2): 38-49.
Julesz, B., Gilbert, E., Shepp, L. A. and Frisch, H. L. Inability of humans to discriminate between textures that agree in second-order statistics - revised. Perception 1973; 2: 391-405
Julesz, B. Experiments in the visual perception of texture. Scientific American 1975; 232(4): 33-43.
Julesz, B. Perceptual limits of texture discrimination and their implications to figure-ground separation. in Leeuwenberg and Buffart (eds.) Proceeedings of the Conference on Formal Theories of Visual Perception, Nijmegen, Holland. Wiley 1978.
Julesz, B. and Caelli, T. On the limits of Fourier decompositions in visual texture perception. Perception 1979; 8: 69-73.
Julesz, B. Spatial nonlinearities in the instantaneous perception of textures with identical power spectra. Phil. Trans. of the Royal Society 1980; B290: 83-94.
Julesz, B. A theory of preattentive texture discrimination based on first-order statistics of Textons. Biological Cybernetics 1980; 41 :131-138.
Julesz, B. Textons, the elements of texture perception, and their interactions. Nature 1981; 290: 91-97
Julesz, B. and Schumer, R. A. Early visual perception. Annual Reviews of Psychology 1981: 575-627.
Julesz, B. Figure and ground perception in briefly presented iso-dipole textures. in Kubovy and Pomerantz (eds.). Perceptual Organization 1982, Lawrence Erlbaum Publishers
Julesz, B. Psychologically determined receptive fields in monocular and binocular texture perception. in Szekely, Lobos, and Danjanovich (eds.) Neural Communication and Control 1981, Pergamon Press.
Julesz, B. Textons, the elements of texture perception, and their interactions. Nature 1981; 290: 91-97
Julesz, B. The role of terminators in preattentive perception of line textures. in Proceedings of the Conference on Recognition of Pattern and Form 1979. University of Texas at Austin, 1982.
Julesz, B. A brief outline of the texton theory of human vision. Trends in Neurosciences 1984; 7:41-45
Julesz, B. Adaptation in a peephole: a texton theory of preattentive vision. In Spillman and Wooten (eds.) Sensory Experience, Adaptation and Perception 1984; Erlbaum Pubs.
Julesz, B. Toward an axiomatic theory of preattentive vision. In Edelman, Gall, and Cowan Dynamic Aspects of Neocortical Function 1984; Joh Wiley Pubs.
Julesz, B. Texton gradients: the texton theory revisited. Biological Cybernetics 1986; 54:245-51
Julesz, B. Texture Perception. in G. Adelman (ed.) Encyclopedia of Neuroscience, 1987 . Birkhauser Publishers.
Julesz, B. Vision: the early warning system. In R. Gregory (ed.) The Oxford Companion to the Mind 1987: Oxford University Press.
Julesz, B. AI and early vision: Part 1. in Blake and Troscianko (eds.) AI and the Eye 1990; John Wiley Pubs.
Marr, D. Early processing of visual information. Phil. Trans. of the Royal Society 1976; B275: 483-524.
Nothdurft, H.C. Orientation sensitivity and texture segmentation in patterns with different Orientation. Vis. Res. 1985; 25: 551-560
Nothdurft, H.C. Sensitivity for structure gradient in texture discrimination tasks Vis. Res. 1985; 25: 1957-1965
Nothdurft, H.C. Texton segregation by associated differences in global and local luminance distribution. Proc. Royal Society London 1990 B239:295-320
Nothdurft, H.C. Texture segmentation and pop-out from orientation contrast. Vis. Res. 1991; 31: 1073-1078
Nothdurft, H.C. Feature analysis and the role of similarity in pre-attentive vision. Perception and Psychophysics 1992: 52:355-375
Nothdurft, H.C. Different approaches to the coding of visual segmentation, in Jenkin and Harris (eds.) Computational and Physical Mechanisms of Visual Coding. Cambridge University Press; 1997
Olson and Attanneave What variables produce similarity grouping? Am. J. Psych. 1970; 83:1-21
Resnikoff, H.L. The Illusion of Reality. New York: Springer; 1989
Sagi, D. and Julesz, B. Detection versus discrimination of visual orientation. Perception 1985; 14: 619-628
Victor, J.D. and Brodie, S.E. Discriminable textures with identical Buffon-needle statistics. Biological Cybernetics 1978; 31:231-34.
Williams, B. and Julesz, B. Perceptual asymmetry in texture perception. Proceedings of the Nationa Academy of Sciences 1992; 89:6531-6534.