GB2294171A - Signal processing arrangement - Google Patents

Abstract

A video signal is processed as a series of sequential line signals by a number of cascaded processing stages. A correlator 13 compares the video signal with reference data after it has passed through an initial processor stage 6 - 11. The arrangement is organised to process a large amount of data rapidly, with data from a particular stage being processed while data representing different line signals of the same video signals is being processed by a preceding or following stage. The video signal may represent a scene viewed by camera 6 and the reference data represents the expected scene (or a part) as the camera moves along an expected path. <IMAGE>

Description

Signal Processing Arrangement.

This invention relates to a signal processing arrangement of a kind which enables the position of a moving body to be precisely determined in relation to its surroundings. It has been proposed that the position of a moving body can be obtained by using an image sensor which is carried by the body to view its surroundings and to correlate the scene which it observes with stored data representing reference scene information. This reference scene consists of preassembled information which is indicative of the scene which the sensor on the moving body expects to observe if it moves along the path which it is intended to take.

By notinq the actual position of the body with regard to its surroundings its path can be corrected to compensate for errors in the motion of the moving body.

This enables it to be kept to its required path with a high degree of precision.

The amount of signal processing associated with such a system is very large indeed,particularly if the moving body moves at an appreciable speed with respect to surroundings having a complex appearance.

The present invention seeks to provide a signal processing arrangement which is suitable for this purpose and is able to operate at a sufficiently high speed without being excessively cumbersome.

According to this invention a data processor comprises a plurality of sequentially connected processing stages, including a video signal generator arranged to produce a video signal as a succession of lines of digital data, a digital filter; and a correlator; the correlator being arranged to correlate data derived from the video signal with reference data; and means for feeding data from one stage to the next such that each stage (apart from the first) receives lines (or parts thereof) of processed data from a preceding stage and processes them, whilst said preceding stage processes subsequent lines of data of the same video signal.

This organisation of processing stages permits a complex data processing to be achieved very rapidly, as it is not necessary for a particular processing stage to wait until all preceding stages have processed the entire video signal.

Preferably the reference data represents a smaller quantity of data than that of said video signal, so that the whole of the reference data can be correlated with a portion of the derived data before the video signal has been completely processed by preceding stages.

The invention is advantagously used to process video signals representative of the scene viewed by a television camera, or the like, and to correlate the viewed scene with stored data representing visual features which the television camera is expected to see.

In order to achieve correlation it usually necessary to orientate the two sets of data together, and to adjust their effective size, so that the two sets of data elements, are derived from the television camera, and the other derived from the stored data, which represents the same visual feature,have the same effective size and are aligned with each other. This function is performed by a geometric processor which very advantageously operates upon the stored data. As the quantity of reference data, for a particular correlation is less than the data derived from the television camera ("scene data") the time taken to process it by the geometric processor is correspondirqly less. Furthermore, the geometric processor can be operating whilst the previously mentioned processing stages are performing the necessary operations upon the scene data.

The invention is further described with reference to the accompanying drawing which illustrates the signal processing arrangement in a simplified diagrammatic fashion.

As a body moves over a particular surface so as to conform at least approximately with a predetermined path, an image sensor such as a television camera or infrared detector views the stationery surroundings of the moving body. The viewed scene is compared periodicaily with stored reference data which represents small localised portions of the surroudings which the moving body will expect to see if it adheres to its predetermined path. The stored data represents a simplified pattern of the dominant visual features. Insignificant and minor details are not stored, as this would make the comparison process unnecessarily difficult. The data is pre-processed so that is consists of binary digits, i.e.

l's and O's,in which te presence of a digital 1 indicates a visual transition from light to dark or vice versa.

Stored representations of a plurality of these small areas are held within a bulk store 1, and data relating to a particular area are extracted as and when needed under the control of a processor 2, which receives information over lead 3 concerning the nature of the movement of the body. From this information, which typically include an indication of the speed and direction of the moving body, the processor 1 is able to determine to a first approximation the current position of the moving body so that it is able to select from the bulk store 1 the data relating to the most appropriate reference area. The reference data is fed to a geometric processor 4 which acts to modify the reference data in dependence on the attitude and distance of the moving body from its viewed surroundings.This stage is necessary so that the reference data can be placed into a format which is compatible with the actual scene as viewed by the image sensor. In this example the image sensor is a television camera 6. To facilitate this geometric processing in a sufficiently rapid and economical manner, the data is received from the bulk store in the format of a two dimensional raster pattern, which is akin- to the conventional television picture raster, so that "lines" of binary data are fed sequentially to the processor 4. The geometric processor 4 cas conveniently take the form disclosed in our earlier British Patent application published under number G.B. 2100956 A, which performs the necessary operations sequentially on successive lines of data.The parameters necessary to perform the geometric processing are obtained from the processor 2 which derives this information via lead 3 from the indication of the distance of the moving body from the viewed scene and the attitude or orientation of the moving body with respect to that scene. Thus the processor 4 rotates the data pattern to correctly orientate it with respect to the viewed scene, and magnifies its size depending on the magnification and viewina distance of the camera 6.

The processed information which is output by the geometric processor 4 is assembled in a reference memory 5 so that it corresponds to the format in which data relating to the viewed scene is expected to be received.

The television camera 6 is mounted on the moving body so as to be able to view the surroundings which are assumed to be stationary. It generates a video signal in a conventional two-dimensional raster pattern.

The television camera may be of the conventional kind in which its video output signal represents the normal visible scene, or alternatively it may be a camera which is responsive to the infra-red content of a scene. The raw video information is passed to a sensor interface 7 which converts the data into a format which is acceptable to the subsequent stages of processing arrangement. The output from the sensor interface 7 consists of a stream of multi-level digital bits, each "pixel" or picture point being represented by a digital word having a sufficiently number of bits to give the amplitude of the video signal at that point with the required accurracy.

A two-dimensional array containing such a pattern of data is of a very complex nature and it is extremely difficult to process it in a realistic and economical fashion. Furthermore, because of the presence of multi-level data which represents all shades of grey of the viewed scene (assuming a monochrome camera), it is difficult to reliably distinguish important or dominant visual features from irrelevant background information or noise.

The data from the sensor interface 7 is passed to a digital filter 10 whose purpose is to detect the presence of dominant features, termed "edges", and to provide a digital binary output signal which represents only these edges. A suitable edge detector is described in our copending British patent application published under number G.B. 2100955 A. Thus, a gradual transition from light to dark regions of the viewed scene will be converted by the action of the digital filter 10 into an abrupt transition between the two regions, and be represented as a thin line (e.g. digital l's) on an otherwise uniform background (e.g. consisting of digital O's). In this way the presence of important visual features is enhanced and they are presented in data format in which they can be conveniently compared with the reference features from the bulk store 1.

The filter 10 is arranged to commence processing as soon as the first few lines of data from the interface 7 become available, and to output the results as soon as those few lines have been processed. Thus it processes particular lines of data, whilst following lines of data are being generated by the camera 6 and fed through the interface 7.

The output from the digital filter 10 is passed to a scene memory 11 where it is temporarily stored before being correlated with the reference data in the reference memory 5. Typically the area of the scene viewed by the television camera 6 is about sixteen times as large as the effective area of each region of reference data held in the reference memory 5. As a reference memory typically consists of a two dimensional data array having 64 x 64 pixels, the scene memory is equivalent to a two dimensional array comprising of 256 x 256 pixels.

It is extremely advantageous to locate the geometric processor 4 in the reference data arm of the processing arrangement, as in this position it is required to process a much smaller quantity of data, i.e. only 64 x 64 pixels as compared to the much larger quantity 256 x 256 pixels which it would be required to process if it were located at the input of the scene memory 11. Furthermore, the video signal from the television camera 6 is being assembled at an extremely rapid rate in real time, and it is desirable to minimise the amount of processing which it must undergo. Thus the reference data can be extracted from the bulk store 1 shortly before it is needed, and then processed by the geometric processor 4 so that it can be entered into the reference memory 5 at or shortly before the time that a corresponding scene is expected to be entered into the scene memory 11.

Both the scene memory 11 and the reference memory 5 contain two dimensional arrays of binary data and the purpose the sequencer 12 and correlator 13 is to compare every possible position of the reference array with that of the two-dimensional scene array. As the reference area is so much smaller than the scene area, the processes of sequencing and correlation commences before the entire scene memory 11 has been completely assembled. In practice, the sequencer 12 and 13 are arranged to operate on a line by line basis using a sequential array of correlation cells. This greatly reduces the time needed to complete the correlation process as it is not necessary to wait until the entire two-dimensional array of scene data has been assembled in the memory 11 before starting the correlation process.

A high speed correlator which is capable of correlating large quantities of binary data is described in our co-pending patent application (I/6786/ELL).

For each possible position of the small reference area relative to the scene, an output is provided by the correlator 13 to a correlation analyser 14. The output on the correlator represents the score, i.e. that is to say, a number representing the degree of similarity between the reference memory and that localised portion of the scene memory with which it is currently being compared. Each time a correlation score is generated a control signal is passed over line 16 to the sequencer, to initiate the next correlation process. In the present example there are 193 x 193 different possible positions, of the reference area relative to the scene area, and hence a corresponding number of different scores are generated by the correlator 13.The correlator analyser 14 inspects all of the scoresand determines which is the highest, i.e. which region of the scene memory is most like the reference memory. The information identifying position of the scene memory which gives rise to the highest score is provided by the analyser 14 and passed to output control 15, which provides a control signal to the control signal processor 2 to indicate that correlation is complete.

When the correlation process has been completed for all possible position of the reference area with respect to the score area the position of the moving body which carries the television camera 6 is now known very precisely. If the moving body has departed even slightly from its expected path, the result of the correlation can be used to provide a correlation factor so as to compensate for the errors, and to bring the moving body back onto its required path. By performing correlation processes at regular intervals along the expected path of the moving body correction can be applied before the deviation errors accumulate to any significant extent.Clearly the correlation processes must be repeated at sufficiently frequent intervals so that any accumulated errors or drift, are not so large that the scene viewed by the television camera does not include a reference area held within the bulk store 1.

It will be appreciated that the architecture of the described processing arrangement permits a particularly rapid correlation process to be performed. For example with reference to the drawing, each block in the signal path from the television camera 6 to the correlation analyser 14 can pass its partial results onto the next block in the sequence before the entire two-dimensional array of scene data has been completely processed. It is merely necessary for the first few raster lines of data to be processed by any one block, so that they can be made available to the next block. The results are then passed on to subsequent blocks which can commence processing whilst the previous blocks are still handling the remaining data lines of the two-dimensional array.

This feature greatly enhances the speed of the processor and can be likened to a pipeline" process. Using this architecture it is practicable to process the video signals generated by the camera 6 in real time, and to use the results to accurkely control the movement of the body which carries the camera. In the case of a scene organised as 256 raster lines, all lines apart from the first and last few lines can in theory be processed simultaneously by all of the processing blocks.

Claims (4)

Claims

1. A data processor comprising a plurality of sequentially connected processing stages, including a video signal generator arranged to produce a video signal as a succession of lines of digital data, a digital filter; and a correlator; the correlator being arranged to correlate data derived from the video signal with reference data; and means for feeding data from one stage to the next such that each stage (apart from the first) receives lines (or parts thereof) of processed data from a preceding stage and processes them, whilst said preceding stage processes subsequent lines of data of the same video signal.

2. A data processor as claimed in claim 1 and wherein the reference data represents a smaller quantity of data than that of said video signal.

3. A data processor as claimed in claim 2 and wherein a geometric processor is arranged to operate upon said reference data before it is presented to the correlator, the geometric processor controlling the size and orientation of the reference data with respect to that of the data derived from the video signal.

4. A data processor substantially as illustrated in and described with reference to the accompanying drawing.

4. A data processor in accordance with any of the preceding claims, in which respective memories are provided in the two input paths to the correlator, data being extracted from the memories and fed to the correlator as needed under the control of a sequencer.

5. A data processor substantially as illustrated in and described with reference to the accompanying drawing.

Amendments to the claims have been tiled as follows ~ II 1. A data processor comprising a plurality of sequentially connected processing stages, including a video signal generator arranged to produce a video signal as a succession of lines of digital data, a digital filter; and a correlator; the correlator being arranged to correlate data derived from the video signal with reference data; and means for feeding data from one stage to the next such that each stage (apart from the first) receives lines (or parts thereof) of processed data from a preceding stage and processes them, whilst said preceding stage processes subsequent lines of data of the same video signal and a geometric processor arranged to operate upon said reference data before it is presented to the correlator, the geometric processor controlling the size and orientation of the reference data with respect to that of the data derived from the video signal.

2. A data processor as claimed in claim 1 and wherein the reference data represents a smaller quantity of data than that of said video signal.

3. A data processor in accordance with claim 1 or 2 in which respective memories are provided in the two input paths to the correlator, data being extracted from the memories and fed to the correlator as needed under the control of a sequencer.