Using Color to Separate Reflection Components, Study notes of Linear Algebra

Download Using Color to Separate Reflection Components and more Study notes Linear Algebra in PDF only on Docsity! ROCHESTER UNIV NY DEPT OF COMPUTER SCIENCE S A SHAFER FE92 APR 84 TR-136 N@@@14-82-K-@i93 UNCLSSIFIDi F/G 9/2 N EEERhh EE 999E~l".i 11111,.25 "- -- SI li-i ....... _. / III= -Ia+ + HI,,/ MICROCOPY RESOLUTION TEST CHART NATIONAL BUREAU OF STANDAROS-1963-A %F Sii °.tSal ° i'. 'e ; 5 . . . . . . . ..-.-.,.-.+. .. ... .. .. . ..- , *S. ..... . .... , ..'.-, ,-. ,., . • . - -JI . . ... ....f. , . .. . ....- + ... .+ . . . ... + . . . " .- ,, . ..5• . .' SECURITY CLASSIFICATION OF THIS PAGE (When D.e Enterd) REPORT DOCUM ENTATION PAGE READ INSTRUCTIONS BEFORE COMPLETING FORM I. REPORT NUMBER 2. GOVT ACCESSION NO. 3. RECIPIENT'S CATALOG NUMBER TR 136 [A ,__-.__ _ 4. TITLE (and Subile) i. TYPE OF REPORT & PERIOD COVERED Using Color to Separate Reflection Components technical report 6. PERFORMING ORG. REPORT NUMBER 7. AUTHOR(a) 8. CONTRACT OR GRANT NUMBER(s) Steven A. Shafer N00014-82-K-0193 9. PERFORMING ORGANIZATION NAME AND ADDRESS I. PROGRAM ELEMENT. PROJECT. TASK Computer Science Department AREA A WORK UNIT NUMBERS University of Rochester Rochester, NY 14627 "_ II. CONTROI.ING OFFICE NAME AND ADDRESS 12. REPORT DATE Defense Advanced Research Projects Agency April 2, 1984 1400 Wilson Blvd. 13. UMUER OF PAGES Arlington, VA 22209 -3 14. MONITORING AGENCY NAME 6 ADDRESS(it diltemrnt from CoamfllInd Office) IS. SECURITY CLASS. (of this report) Office of Naval Research n Information Systems • _unclassifled__ Arlington, VA 22217 Is. DECLASSIFICATION/DOWGRADINOG SCHEDU E 16. DISTRIBUTION STATEMENT (of thia Repori) Distribution of this document is unlimited. .1 17. DISTRIBUTION STATEMENT (of the abstract entered in block 20, it different from Report) IS. SUPPLEMENTARY NOTES none 19. KEY WORDS (Conllnue on reverssi It U neceear end ldenOtly by block number) 20. ABSTRACT (Continue on reveres aide It neceeea.y and Identify by block number) This paper presents an algorithm for analyzing a standard color image to determine intrinsic images of the amount of interface ('specular") and body ("diffuse") reflection at each pixel. The interface reflection represents the highlights from the original image, and the body reflection represents the original image with highlights removed. Such intrinsic images are of interest because the geometric properties of each type of reflection are simpler than the geometric properties of intensity in a black-and-white image. (over) FORM143 ",OND O ,SO,,.T"-" DD , 1473 EDITIONOFINOVSSISOBSOLETE unclassified SECURITY CLASSIFICATION OF THIS PAGE (Phon Data Entered) 0 - . . - ., . . , : . . ., . . . . , . - . - SECURITY CLASSIFICATION OF THIS PAOGtW"we, Data Entered) 20. Abstract (cont.) The algorithm is based upon a physical model of reflection which states tha two distinct types of reflection--interface and body reflection--occur, and tha each type can be decomposed into a relative spectral distribution and a geo- -. ) metric scale ractor. This model is far more general than typical models used in computer vision and computer graphics, and includes most such models as special cases. In addition, the model does not assume a point light source or - uniform illumination distribution over the scene. The properties of spectral projection into color space are used to derive a new model of pixel-value color distribution, and this model is exploited in an algorithm to derive the intrinsic images. Suggestions are provided for extending the model to deal with diffuse illumination and for analyzing the intrinsic images of reflection. SECURITY CLASSIFICATION Of THIS PAGE(Whes Date Ente4 * * . '* * * ** * * * 5 . °" .' - . i -: - Using Color to Separate Reflection Components Steven A. Shafer* Computer Science Department University of Rochester Rochester, New York 14627 2 April 1984 Abstract ) This paper presents an algorithm for analyzing a standard color image to determine intrinsic images of the amount of interface ('pecularland body ("iffuse I reflection at each pixel. The interface reflection represents the highlights from the original image, and the body reflection represents the original image with highlights removed. Such intrinsic images are of interest because the geometric properties of each type of reflection are simpler than the geometric properties of intensity in a black- and-white image. The algorithm is based upon a physical model of reflection which states that two distinct types of reflection -- interface and body reflection -- occur and that each type can be decomposed into a relative spectral distribution and a geometric scale factor. This model is far more general than typical - models used in computer vision and computer graphics, and includes most such models as special cases. In addition, the model does not assume a point light source or uniform illumination distribution over the scene. The properties of spectral projection into color space are used to derive a new model of pixel-value color distribution, and this model is exploited in an algorithm to derive the intrinsic images. Suggestions are provided for extending the model to deal with diffuse illumination and for analyzing " the intrinsic images of reflection. 6LCd'l r, - c ,,^ f!. -L I I (0) The author's permanent address is: Computer Science Department, Carnegie-Mellon University, Pittsburgh, Pennsylvania 15213. This research was sponsored by the Defense Advanced Research Projects Agency (DOD), monitored by the Office of Naval Research under Contract NOOO1 4-82-K-01 93. .-. -. The views and conclusions contained in this document are those of the author and should not be Interpreted as representing the official policies, either expressed or implied, of the Defense Advanced Research Projects Agency or the US Government. * . . ..... : - ................................ ......... . . 1. Introduction When we look around us, the surfaces we see are typically glossy. They may seem to be very shiny, fairly matte, or anywhere in between, but virtually all the surfaces around us exhibit highlights to varying degrees. These highlights are most pronounced when the surface normal bisects the angle between the direction of illumination and the direction of view, making the position and intensity of highlights very sensitive to viewing geometry. This causes problems with many low.level computer vision methods such as segmentation (which typically assumes uniform or smoothly varying intensity across a surface) or stereo and motion analysis (which attempt to match images taken from different viewpoints). Highlights are not the only source of intensity variation across a surface. Even with uniform illumination of a matte, uniformly colored surface, there will be smooth shading due to the angle of incidence of the incoming illumination relative to the surface normal. - - It would be very useful to be able to separate the effects of shading from highlights. This might result in intrinsic images telling, at each pixel, the intensity of the shading and the intensity of the highlight at that point. This separation was first suggested by Barrow and Tenenbaum (1], and would effectively produce an image of the highlights, and an image with the highlights removed. These would be useful for analysis since each of these phenomena is more simply related to the angles of illumination and viewing than is their sum (which is measured in a black-and-white image). In addition, the relative insensitivity of shading to viewpoint would make the shading intrinsic image an ideal candidate for stereo or motion analysis. It is frequently observed that highlights have a different color from the characteristic color of a surface (which is related to shading). In this paper, we show how a simple, rather general model of reflection, called the "Dichromatic Reflection Model", can be .used to determine intrinsic images of the two types of reflection from a standard color image (i.e. red-green-blue separation). The analysis uses the properties of spectral projection, the process whereby color pixel values are determined from the spectral power distribution (SPD) of incoming light. Combining the Dichromatic Reflection Model with spectral projection results in a new model of pixel value distribution in R-G-B color space. The model predicts that pixel values from pixels on a single surface will lie on a parallelogram in color space, and that the position of any pixel's color within that parallelogram yields the coefficients of the two types of reflection. I~ 2 A simple algorithm is presented which utilizes this model of pixel values to determine the desired intrinsic images. The model and the algorithm are then extended to deal with diffuse illumination and shadows. Additional work for the future includes implementation of the algorithm and verification of the model with real images. For additional background information, the reader interested in radiometry in general is referred to [9], [12], and [22]; while appearance measurement (gloss and color) is discussed in (10], [14], [16], and [35]. 1.1 Previous Work in Color Image Understanding Color image understanding in the past has not been based on general models of reflection. Most of the work has been the application to three-dimensional color space of algorithms originally developed and used for analyzing monochrome images. This includes primarily edge detection [21] and clustering [5,18, 20, 23] (etc.). In such work, image regions or edges are identified by distances between pixel values in color space, without appeal to any model of color generation in the scene. Color has also been used for object labelling based on known object colors or object colors measured in the image [19, 26,31,32,34, 36]. This approach, known as "spectral signature analysis" in remote sensing, uses the known reflection properties of materials of interest in a particular domain. It depends on having few different types of materials in the scene, and having prior knowledge of their spectral reflectance. The properties of color space transformations have also been studied [17, 24], although no such transformations have sufficed to "solve" the image segmentation or labelling problems. The only previous work in utilizing general properties of reflected color has been in the form of simple statements such as "far-away outdoor objects look bluish", "outdoor shadows are bluish", and "natural colors tend to be desaturated" [19, 31]. In the psychological field, Rubin and Richards proposed a method for using color to determine changes in material across an image [27]; however, while the method is quite interesting, their assumptions appear to be restrictive. The approach described in this paper is significant because it is applicable to many common types of materials, without prior knowledge about their colors, and because it is based on a very general model of reflection in the real world. Section 2.3 presents a discussion of previous work in reflectance models for computer graphics and image understanding. P. e Z e ~............................. 3 2. Reflection and the Dichromatic Model In this chapter, we present a brief account of the physics of reflection, followed by the Dichromatic Reflection Model which captures certain aspects of the reflection process. The validity and assumptions of the model are discussed, and it is compared with other reflection models used in computer graphics and image understanding. 2.1 Physical Properties of Reflection The methods of this paper deal with materials which are optically inhomogeneous, meaning that light interacts both with a medium that comprises the bulk of the surface matter, and with particles of a colorant that produce scatering and coloration. Many common materials can be described this way, including most paints, varnishes, paper, ceramics (including porcelain), plastics, etc. Materials that are homogeneous are not included in this discussion; thus metals and many crystals are not amenable to the analysis presented herein. We will also limit this discussion to opaque surfaces, I which transmit no light from one side to the other. macroscopic perfect specular direction interface reflection incident light body reflection AIR ' ~%~7~Nc- INTERFACE • * - MEDIUM 0 00 0 * COLORANT Figure 2-1: Reflection of Light from an Inhomogeneous Material Although we sometimes think of a visual surface as a plane, this is an approximation useful only at the macroscopic level. To understand reflection from inhomogeneous materials, we will instead view the surface as having a definite thickness (Figure 2-1) [14,16, 35]. When light strikes a surface, it must first pass through the interface between the air and the surface 6 2.1.1 An Aside on Terminology This paper adopts the terms "interface" and "body" reflection rather than the more common terms "specular" and "diffuse" reflection. While the latter have gained some popularity in the literature of appearance measurement and computer imaging (graphics and vision), they have some severe .:,- problems which render them controversial. "Specular reflaction" may mean any of three things: * interlace reflection .. the most common usage * interface reflection in perfect specular direction at macroscopic level .. the usage most common in gloss measurement e interface reflection where surface is locally optically smooth -. the usage in scattering theory and laser optics [2]. Similarly, the term "diffuse reflection" means "reflected light scattered over a large solid angle", and thus may apply equally well to body and to interface reflection. The author, having experienced numerous communication problems due to the use of the terms specular and diffuse, has ultimately adopted the terminology used in this paper, which is sometimes found in the more technical optics literature. • . 4'% e FIgure 2-4: Photometric Angles The terminology used throughout this paper for describing reflectance geometry is illustrated in figure 2.4, which defines the following angles [11]: * the angle of incidence, i .- the angle between the illumination direction I and the surface normal N * the angle of emittance, e -- the angle between N and the viewing direction V * the phase angle, g .. the angle between I and V :::: ..- ~.-.... . . . . . '., 7 the off-specular angle. s . the angle between V and the direction of (macroscopic) perfect specular reflection J.--" We also use the standard symbol X to refer to the wavelength of light. 2.2 The Dichromatic Reflection Model We now propose a simple mathematical model of reflectance, based on the above discussion, called the Dichromatic Reflection Model. The Dichromatic Reflection Model states: L(X,i,e,g) = Li(X,i,e,g) + Lb(X,i,e,g) (2-1) = mi (i, e, g) c, (X) + m b (i, e, g) Cb (,) (2-2) This model represents two statements about reflected light, as expressed by the two parts of the equation: 1. Equation (2-1) says that the total radiance L of the reflected light is the sum of two independent parts: the radiance Li of the light reflected at the interface and the radiance Lb of the light reflected from the surface body. 2. Equation (2-2) says that each of these components of the light can be decomposed into two parts: - a. composition -- a relative spectral power distribution c, or Cb which depends only on wavelength but is independent of geometry, and b. magnitude .- a geometric scale factor mi or mb which depends only on geometry and is independent of wavelength. Intuitively, the Dichromatic Reflectance Model says that there are two independent reflection processes, and that each has a characteristic color whose magnitude, but not spectral distribution, varies with the directions of illumination and view. In the remainder of this chapter we will address the scope of this model and its validity; in the next - chapter we will see how this model may be exploited to determine intrinsic images of mi and mb, the amount of interface and body reflection at each pixel. The Dichromatic Reflection Model assumes the following: o The surface Is an opaque, inhomogeneous medium with one significant interface. e The surface is not optically active, i.e. it has no fluorescence or thin-film properties, and it is uniformly colored, i.e. it has a uniform distribution of the colorant. Reflection from the surface is Isotropic with respect to rotation about the surface normal. .4.; ** 4~* . .~ 4 4 ' . . ' . . \ . . . ... * ..- * . '.-°;. . . . ** *q o.O" 4 . . °. ° . 7 7 7 - - - 8 * There is no inter-reflection among surfaces. * There is a single light source, i.e. no diffuse ("ambient") illumination, and the relative spectral power distribution S(A) of the illumination is constant across the scene. The assumptions about the surface are typical for reflectance models and not too unrealistic. The assumption of no inter-reflection is also typical, but unfortunately not realistic at all. Finally, the assumption of no ambient light is not at all realistic, but will be relaxed (in fact, eliminated) in section 4.1 below. Equally interesting is a list of assumptions not made by the model, which express the scope or generality of the model: . Imaging geometry -- the model makes no assumption that orthography or perspective is being used. Either projection satisfies the model. o Planar surface -- the model applies equally to curved and planar surfaces. It also applies to textured surfaces, i.e. surfaces with macroscopic roughness (but see the note below about analyzing intrinsic images). o Specific reflectance model -. the model does not assume specific functions mi, ci, N or Cb; in particular, there is no specific geometric model of highlights, no assumption that the highlights have the same color as the illumination, and no assumption that the body roflection is isotropic. o Point light source -- the model applies equally well to a point light source, an extended light source, or a light source infinitely far away. o Uniform distribution of illumination .- the model does not assume that the amount of illumination is the same everywhere in the scene; only that the SPD is the same (see above). This is important, since real (especially extended) light sources produce nonuniform amounts of illumination in different areas of the scene. It must be pointed out that any complexity of the forms mentioned above will cause great difficulty in analyzing the resulting intrinsic images of m i and mb. However, the Dichromatic Reflection Model will make it possible to compute these intrinsic images regardless of such complexity. In spite of the apparent simplicity of the Dichromatic Reflectance Model, it suffers from some flaws. In practice, they ought to have only a minimal impact on the usefulness of the model. * There is no obvious way to decide how to scale the magnitude functions m, and mb against the composition functions ci and cb. Therefore, the resulting intrinsic images may be only of relative, rather than absolute, reflection magnitudes. At least, it is reasonable to require that O < mi, m b 5 1 for all i, e, and g. e Interface reflection exhibits an interdependence between wavelength and geometry, as expressed by Fresnel's equations. As stated above, this is a small effect; the author's . .- *ba 3. Pixel Values in Color Space This chapter begins with a discussion of spectral projection, the relationship between an SPD of light and its color coordinates. When this process is applied to the D.chromaic Reflection Model, a new model for pixel values in color space is the result. A simple algorithm is presented for exploitlIng this model to determine the intrinsic images of m i and mb. 3.1 Spectral Projection Spectral projection is the process whereby pixel values are computed from the spectral power distribution (SPD) of the measured light. The process in a monochrome camera is quite simple, with the pixel value p being just a summation of the amount of light at each wavelength X(,), weighted by the responsivity of the camera to the various wavelengths, s(.): p = J X(X) slA) d, The interval of summation is determined by s(,), which is non-zero over a bounded interval of wavelengths X. In a color camera, color filters are interposed between the incoming illumination and the camera Each filter has a transmittance function 'r(A), specifying the fraction of light transmitted at each wavelength; thus, spectral projection with a filter is specified by the above integral, with s(,) relplaced by i(A) s(A). Typically, three filters (red, green, and blue) are used with transmittances tr' 1. and rb, resulting in a vector of three color values, C a [r, g, b]. If we let r(A) be the reeponavity of the camera combined with the red filter, 7(A) = r(X) s(k), (etc. for g and b), then the color value of any SPD X(X) is given by: x x 1 = [ x(A)i(A) dh, bx L X(X) b(A) dJ Spectral projection is a linear transformation, as shown in [281; in other words, C , bY * a +x " bCy where a and b are scalars and X(\) and Y(X) are SPD's. To see this, consider first the red component rax + by; it is easily seen that- reX+by = f[aX) + bY(,)] F(X)dA = a f X(,\) T(X) dX + b f Y(A) T(A) d. N a + b ry With similar equations for green and blue, we have the complete result. The linearity property of "- -. 'E : ... . . -. -.z. - _-. .'l 12 spectral projection is important because it says that a mixture of two SPD's of light results in a sum of the corresponding pixel values, taken in the same proportion. 3.2 The Dichromatic Model in Color Space When the linear property of spectral projection is combi.ed with the Dichromatic Reflection Model, a powerful new model of pixel color values results. Fils. consider a specific point on a surface. At that point, the geometry (angles i, e, and g) is demned, and the magnitudes mi and mb may be considered as scalars. So, the Dichromatic Reflectance Model may be rewritten at a specific point as: L (A) - m, c,(.\) + mb cb(\) Thim defines the SPD of the light reflected from the surface. Now, applying the linearity of spectral Prgcbnil . we have: CL - ,, ci ,,b Cb where C;L is e color (pixel value) measured, mi and mb are the magnitudes of reflection at the point in quisllon, and Ci and Cb are the colors of the interface and body reflection of the material. Cosder the colors C. of a set of points on the same uniformly colored surface. Because the geometry is different at each point, the scale factors mi and mb vary from point to point. However, the colors Q, and Cb of the interface and body reflection are the same at all points on the same surface, becase they are simply the spectral projections of ci(A) and cb() which do not vary with geometry. In other words, the pixel values are a linear c6mbination of C and Cb, with the coefficients dby m, and mb at each point. pixgl cogorm nClr'p B C : Figure 3-1: Pixel Values on a Surface Lie on a Parallelogram in Color Space .,.: " .r 13 Cb mb miC! I Figure 3-2: Position Within Parallogram Determines Magnitudes Recalling that we can assume 0 < m i, mb <: 1 without loss of generality, we see that the pixel values CL for a set of points on a single surface must lie within a parallelogram in color space, bounded by the colors Ci and Cb of the interface and body reflection of the surface (Figure 3-1). One corner of this parallelogram will be located at the origin, [r,g,b] = [0,0,0]. Further, within this parallelogram, the position of any color is determined by the values of mi, and m b at the corresponding point (Figure 3-2). The Dichromatic Model in Color Space makes the following assumptions: • Validity of the Dichromatic Reflection Model, i.e. all the assumptions made therein. .Prior segmentation of the image into groups of pixels known (or believed) to lie on a single surface. * Pixel values returned by the camera are linearly related to the irradiance on the sensor plane, i.e. the camera is photometrically calibrated. If the camera is calibrated for monochrome response, it is not necessary to re-calibrate it with each color filter separately since the relative SPD of a filter's transmittance is constant with respect to total intensity of illumination. The model of pixel values does not assume: * Specific C -. it is not assumed that Ci is achromatic (i.e. r, = g, b,), nor that it Is the same for all surfaces in the image. o Specific color responses.. no assumption is made about the color filters rr(X,), , (A), and 7rb(A), except that they are linearly independent, nor is any assumption made about the camera's spectral responsivity s(A), except that the relative spectral responsivity Is constant with respect to the total amount of irradiance and constant across the sensor plane. L'.," 16 4. Applying the Dichromatic Analysis Algorithm In this chapter we discuss the extension of the Dichromatic Reflection Model for dealing with diffuse illumination, and some possible methods of analyzing the intrinsic images of m i and mW 4.1 Extending the Model for Diffuse Illumination The Dichromatic Reflection Model as presented above depends on the assumption that the illumination at any point comes from a single (point or extended) light source. It is more realistic to model the illumination as consisting of a light source, plus "ambient" or "diffuse" light, of lower intensity, coming from all directions in equal amounts, and possibly with a different SPD than the small source. This model is a better approximation of daylight, which contains light from a point source (the yellowish sun) and light from a hemisphere (the bluish sky), and of light in a room, which comes from light fixtures and from inter- reflections off walls and other objects. All the previously mentioned models of reflection in computer graphics presume some sort of diffuse illumination, although the assumption made here (diffuse illumination from all directions equally) is admittedly highly idealized. pixel colors G B Cb _ color of pixels in shadow.- Figure 4-1: Color Space Parallelogram With Diffuse Illumination The light reflected by diffuse illumination contains a part due to interface reflection and a part due to %-' body reflection. By assuming that this light is incident from and reflected into all directions equally, it can be modelled by adding a single term, L,(X), to the Dichromatic Reflection Model: L~~~ (A, °°, ) L (?,,,e,g) V m(I, e,g) c 1 ) + m b (i, e, g) C () + Lal') In color space, the model becomes: 17 = mC + m b C + Ca where Ca is the color of the light reflected from diffuse illumination La(X). Since La(A) does not vary with geometry, the effect of this change is to translate the parallelogram of pixel colors for a single - surface away from the origin by the vector Ca. as seen in figure 4.1. The Dichromatic Analysis Algorithm can still be applied with this change, but note that the plane. fitting and parallelogram-fitting operations specified in the algorithm will be less constrained and therefore more error-prone. If the relative SPD of the diffuse illumination is the same as that of the small source, then C. is a linear combination of C and Cb [6]; in such a case the plane containing the points should pass through the color space origin although the parallelogram will not. It is interestig to note that a point whose color lies at exactly CL = Ca has m i = m b = 0, and might therefore be suspected of lying within a shadow. This is a far more precise description of colors within shadows than statements of the form "shadows tend to be bluish", which have been seen in L previous work in image understanding. In fact, since for different surfaces C. is very likely to differ, it may be pcssible to associate shaded parts of surfaces with illuminated portions, by constructing Ca for each adjacent illuminated area and finding which one matches best with the color of the pixels in the shaded area. Unfortunately, since pixel values in shaded areas of an image tend to be poorly measured by current digitizing cameras, this kind of analysis may prove to be unreliable. The description presented here is intended to model diffuse illumination; however, it will be a poor substitute for a detailed model of inter- reflection when surfaces are close to each other. The author continues to call the above equation a "Dichromatic Reflection Model" (rather than a trichromatic model) because the essence of the model is that reflection occurs from two places in a surface: the interface and the surface body. 4.2 Analysis of the Intrinsic Reflection Images The potential utility of the intrinsic images of m i and mb stems from two facts: * Both m i and M b have simpler geometric properties than does intensity in a monochrome image, which represents a weighted sum of the two. -mb, representing diffusely distributed light, is relatively insensitive to changes In viewpoint from one image to the next. Here are some possible methods for exploiting these properties of the intrinsic images of m, and mb: 1. In stereo or motion analysis, Image matching using m b might be more reliable because of , -", 18 the elimination of interface reflection, whose position in the image is highly viewpoint- dependent. 2. If a specific reflectance model such as Phong's is assumed, then the two values m i and mb provide two constraints on the surface normal at any point. For example, using Phong's model, the angle s can be calculated from mi and the angle i can be calculated from m b.With such constraints, unique surface normals can be established from a color image. (In practice, this kind of analysis would be limited by the applicability of such reflectance models and by the distribution of points on the parallelogram in color space.) 3. If, as above, a specific reflectance model is assumed but the parameters (t, n in Phong's model) are not known, it may be possible to determine or estimate these from the image. At a point with a known orientation, the coefficients m i and m b can be determined by color analysis; using these, the surface parameters may be calculated. This might form the basis for a technique of looking at parts of a surface (say, the spots with pronounced highlights) to estimate the nature of the reflectance properties of the surface, then using this estimated reflectance model to analyze the rest of the surface in detail. 4. Using differentials, since the photometric angles i, e, and g are functions of x and y in the image, we can relate the differentials of the intrinsic image of m i to the geometric properties of m i and the imaging geometry by: (ami/ax amilay] = [amilai ami/ae amilag] aila) aelayWe x /a . ag/ax aglay This equation tells how the intrinsic image and a reflectance model yield constraints on the imaging geometry using first derivatives. • "'j ° ,2 21 6. Bibliography [1] Barrow, H. G. and Tenenbaum, J. M. Recovering Intrinsic Scene Characteristics from Images. In Hanson, A. R. and Riseman, E. M. (editor), Computer Vision Systems, pages 3-26. Academic Press, New York, 1978. [21 Beckmann, P. and Spizzichino, A. The Scattering of Electromagnetic Waves from Rough Surfaces. MacMillan, 1963. Pages 1-33,70-98. S[3] Billmeyer, Fred W. Jr. Private communication. [4] Blinn, J. F. Models of Light Reflection for Computer Synthesized Pictures. Computer Graphics 11(2):192-198, 1977. SIGGRAPH 77. [5] Coleman, G. B. and Andrews, H. C. Image Segmentation by Clustering. Proc. IEEE 67(5):773-785, May, 1979. [6] Cook, R. L. and Torrance, K. E. A Reflectance Model for Computer Graphics. Computer Graphics 15(3):307-316, August, 1981. SIGGRAPH '81. (7] Egan, W. G. and Hilgeman, T. W. Optical Properties of Inhomogeneous Material. Academic Press, New York, 1979. [8] Foley, J. D. and Van Dam, A. Fundamentals of Interactive Computer Graphics. Addison-Wesley, Reading, Mass., 1983. [91 Grum, F. and Becherer, R. J. Optical Radiation Measurements, Volume 1: Radiometry. Academic Press, New York, 1979. [10] Grum, F. and Bartleson, C. J., editors. Optical Radiation Measurements, Volume 2: Colorimetry. Academic Press, New York, 1980. Chapter 7, "Colorant Formulation and Shading", by E. Allen. [11] Horn, B. K. P. Understanding Image Intensities. Artificial Intelligence 8:201.231,1977. [12] Horn, B. K. P. and Sjoberg, R. W. Calculating the Reflectance Map. Applied Optics 18:1770-1779, 1979. 22 [13) Hunter, R. S. High Gloss Measurements. Official Digest of the Federation of Societies for Paint Technology 36:348-356, April, 1964. [14] Hunter, R. S. The Measurement of Appearance. J. Wiley and Sons, New York, 1975. (15] Jenkins, F. A. and White, H. E. Fundamentals of Optics. McGraw-Hill, New York, 1976. [16] Judd, D. B. and Wyszecki, G. Color in Business, Science and Industry. J. Wiley and Sons, New York, 1975. [171 Kender, J. R. Instabilities in Color Transformations. In PRIP.77, pages 266.274. IEEE Computer Society, Troy NY, June, 1977. [18] Levine, M. D. and Shaheen, S. I. A Modular Computer Vision System for Picture Segmentation and Interpretation, Part I. In PRIP.79, pages 523.533. IEEE Computer Society, Chicago, II., August, 1979. (19] Nagao, M., Matsuyama, T., and Ikeda, Y. Region Extraction and Shape Analysis in Aerial Photographs. Computer Graphics and Image Processing 10:195-223, 1979. (20] Nagin, P. A., Hanson, A. R., and Riseman, E. M. Region Extraction and Description Through Planning. COINS TR 77-8, U. Mass., May, 1977. [21) Nevatia, R. A Color Edge Detector and Its Use in Scene Segmentation. IEEE Trans. Systems, Man, and Cybernetics TSMC.7(11):820-826, November, 1977. [22] Nicodemus, F. E., Richmond, J. C., Hsia, J. J., Ginsberg, I. W., and Limperis, T. Geometrical Considerations and Nomenclature for Reflectance. NBS Monograph 160, National Bureau of Standards, October, 1977. [231 Ohlander, R. Analysis of Natural Scenes. PhD thesis, Carnegie-Mellon Univ. Computer Science Dbpt., 1975. [24] Ohta, Y., Kanade, T., and Sakai, T. Color Information for Region Segmentation. Computer Graphics and Image Processing 13:222-241, 1980. - (25] Phong, Bui Tuong. Illumination for Computer Generated Pictures. Communications of the ACM 18:311-317, 1975. [28] Rubin, S. The ARGOS Image Understanding System. PhD thesis, Carnegie.Mellon Univ. Computer Science Dept., 1978. I%~ ~ -. %..... _ 23 [271 Rubin, J. M. and Richards, W. A. Color Vision and Image Intensities: When are Changes Material?." Al Memo 631, MIT, May, 1981. [281 Shafer, S. A. Describing Light Mixtures Through Linear Algebra.. J. Optical Soc. Am. 72(2):299-300, February, 1982. [29] Shibata, T., Frei, W., and Sutton, M. Digital Correction of Solar Illumination and Viewing Angle Artifacts in Remotely Sensed Images. In Proc. 7th Symposium on Machine Processing of Remotely Sensed Data, pages 169-177. 1981. [30] Simpson, L. A. Measuring Gloss and Factors that Affect It. Technical Report, BTP Tioxide Limited, Stockton-on-Tees, England, 1981. [31] Sloan, K. World Model Driven Recognition of Natural Scenes. PhD thesis, U. Penna. Moore School of Electrical Engineering, June, 1977. [32] Tenenbaum, J. M. and Weyl, S. A Region.Analysis Subsystem for Interactive Scene Analysis. In Proc. 4th IJCAI, pages 682-687, September, 1975. [33] Torrance, K. E. and Sparrow, E. M. Theory for Off-Specular Reflection from Roughened Surfaces. J. Optical Soc. Am. 57:1105-1114, September, 1967. [34] Weymouth, T. E., Griffith, J. S., Hanson, A. R., and Rseman, E. M. Rule Based Strategies for Image Interpretation. In Proc. NCAI-83, pages 429-532. AAAI, August, 1983. [351 Williamson, S. J. and Cummins, H. Z.Light and Color in Nature and Art. -: J. Wiley and Sons, New York, 1963. [36] Young, 1. T. The Classification of White Blood Cells. IEEE Trans. Biomedical Engineering BME-19(4):291-298, July, 1972. e -- e 0.l