Detection of highway warning signs in natural video images using color image processing and neural network techniques on a PC

by Kellmeyer, David

Abstract (Summary)
This thesis deals with the possible use of color and neural networks in the further development of the image processing field. Its specific problem is concerned with the detection and location of highway warning signs in natural roadway video images. The goal is to show that through the use of color and neural networks, a robust target detection procedure capable of detecting highway warning signs under varying conditions can be developed. Previous studies have shown color to be a valuable asset in the development of a robust imaging system that must perform under variable lighting conditions. Also, since highway signs rely heavily on color to provide drivers with information, they present an application where color provides an extremely valuable attribute. The imaging system can operate on any IBM AT Style PC or 100% compatible equipped with a Targa+ 64 image acquisition board and some method of inputting images to the system. The basic approach is to digitize a roadway image and segment this image into basic color regions using a set of colors that are both important in highway sign recognition and that provide an even span of the color spectrum. The next step is to search this segmented image for color regions that could possibly represent a highway warning sign. These possible regions are then further analyzed to determine if their shape corresponds to highway signs of that region's color. The system is capable of digitizing either S-Video, Composite Video or RGB Video input into a 640 × 480 color image with 16 bit pixel depth. This image's resolution is then reduced through software to simulate an image digitized at a resolution of 160 × 120 still with 16 bit pixel depth. Since the image is to be segmented into regions of basic colors, the colorfulness of each pixel is now increased using a natural log function so that saturated pixels become more colorful and unsaturated pixels are basically left alone. This saturated image is now ready to be segmented into eight major colors using an artificial neural network. A supervised learning back-propagation network with two hidden layers is used to segment the image into eight basic colors. The neural network learned these colors from examples selected from standard highway images. The inputs to the network are provided by the color value of the pixel being segmented along with the color values of each pixel in a surrounding 3 × 3 neighborhood. These color values are represented using two inputs for each pixel that are derived from color differences that simulate the manner in which the human eye may use to encode color information. For each pixel, the color segmentation network must decide from the neighborhood of inputs which of eight colors (red, orange, yellow, green, blue, purple, brown or achromatic) most closely represents the pixel of interest's color. The result is a 158 × 118 (edge pixels are excluded) image represented using eight colors. A scanning routine then searches the segmented image for regions of interest based on the color of the sign to be detected. Since the goal is to detect highway warning signs, the routine searches for first yellow and then orange regions. Upon finding a yellow or orange region, the system outlines the region and submits this region to be checked for parametric shape features to determine if the region could possibly be a highway warning sign. If the system feels the region is a possible sign, it then bounds the region for further analysis by a sign recognition neural network. Before entering the sign recognition network the possible sign region is converted into a 10 × 10 boundary square of binary values. Binary values are used because the sign recognition network no longer uses color but only the shape of the region to determine if it is a warning sign. This sign recognition network is another supervised learning, back-propagation network but this time with 100 inputs. The network's 100 inputs are binary values provided by each of the 100 pixels in the 10 × 10 boundary square and are used to produce two outputs that judge the region as either a sign or a non-sign. A post-processing algorithm analyzes these two outputs to give the final determination of whether or not the region represents a highway warning sign. For each detected warning sign, a set of attributes is now stored that consists of the coordinates of the sign's bounding rectangle in the original 640 × 480 image, the intensity value of each pixel within this boundary rectangle and the sign's basic color. These attributes are stored for further evaluation of the region under higher resolution to confirm the presumption that the region contains a warning sign and if so to determine what type of a sign it is. This final recognition procedure is the topic of a another study which is presently in progress. The results of the this study prove that neural networks can be used as an alternative to complex image processing algorithms like color image segmentation. The ability to substitute a neural network for a complex image processing algorithm has two main advantages. First, image processing is no longer a highly intense mathematical process only for the Albert Einstein's of the world, but rather, understandable to anyone who has a practical application for machine vision. Second and most importantly, the complexity of traditional image processing algorithms usually stems from trying to develop a more robust algorithm and most still end up requiring some sort of threshold that fails under changing circumstances. This problem can be alleviated by substituting a neural network that is trained to operate under varying conditions for the image processing algorithm. This statement is supported by the results of the color segmentation procedure that is consistently able to depict a variety of natural images in only eight colors (ten, including gray levels) using a neural network. The main observation found in using neural networks is that the paradigm, the number of hidden layers, the learning rate nor any other obscure factor is of greatest importance when developing a neural network. Rather, the way the information is presented to the network and the complexity of the network's task are the major factors in determining if a network will provide desirable results. The use of color proved very effective in locating highway signs but its use is not recommend for all machine vision applications as the complexity and processing time for the algorithms are each increased. The developer should first look at his application to determine if color is a prominent attribute (as in detecting highway signs) and if color may be needed because of poor or uncontrollable lighting. If one of these conditions is present, then the use of color may be justified instead of using luminance values. The system seems to provide very positive results even when a sign is set against a similar background. Of 35 testing images the system located 86% of the warning signs within these images. The processing time requires about 4 minutes per image on 486, 25 MHz processor. About half of this processing time is spent reducing the resolution of the image, which would not be needed in a dedicated system. The saturation routine takes about nine seconds and the color segmentation using the neural network takes about 1.5 minutes. The time required to scan for signs varies for each image. For real-time applications the use of designated hardware or parallel processing would be essential, especially if each frame is to be analyzed using a live video at 30 frames per second.
Bibliographical Information:


School:Ohio University

School Location:USA - Ohio

Source Type:Master's Thesis

Keywords:highway warning signs natural video images color image processing neural network techniques


Date of Publication:01/01/1992

© 2009 All Rights Reserved.