CN112837256A

CN112837256A - Circuit system for Harris angular point detection and detection method

Info

Publication number: CN112837256A
Application number: CN201911067401.8A
Authority: CN
Inventors: 方励; 李绍斌; 陈恒
Original assignee: Gree Electric Appliances Inc of Zhuhai; Zhuhai Zero Boundary Integrated Circuit Co Ltd
Current assignee: Gree Electric Appliances Inc of Zhuhai; Zhuhai Zero Boundary Integrated Circuit Co Ltd
Priority date: 2019-11-04
Filing date: 2019-11-04
Publication date: 2021-05-25
Anticipated expiration: 2039-11-04
Also published as: CN112837256B

Abstract

The application discloses a circuit system and a detection method for Harris corner detection, which are used for solving the problem of low management efficiency of a line cache module. In the circuit system: the line cache module is used for dividing the received input image into single-line image information and storing the single-line image information into a plurality of cache blocks in the line cache module; the window generation module is used for reading the cached image information in a preset mode and performing storage processing on the read or received information in a preset format to obtain rectangular window data; the partial derivative calculation module is used for receiving first rectangular window data corresponding to the cached image information and performing parallel partial derivative operation on the first rectangular window data by using a row unit to obtain horizontal partial derivative data and vertical partial derivative data; the corner response module is used for calculating the corner response values in parallel to obtain a plurality of corner response values, so that the extreme value suppression module performs maximum suppression processing on second rectangular window data corresponding to the corner response values to determine the corner values.

Description

Circuit system for Harris angular point detection and detection method

Technical Field

The application relates to the technical field of circuit design, in particular to a circuit system and a detection method for Harris corner detection.

Background

In the prior art, corner detection plays an important role in image processing related scenes such as target identification, target tracking and the like, and the Harris corner detection algorithm is a signal-based point feature extraction algorithm, so that the Harris corner detection algorithm is superior to other corner detection algorithms in the aspect of processing noise-containing images, and is generally concerned by people.

However, the existing Harris corner detection algorithm needs to set a plurality of line buffer units during circuit design due to a large calculation amount, and the area proportion consumed by the line buffer is increased along with the increase of the resolution of the processed image, so that the management efficiency of the line buffer unit in the prior art is low.

Disclosure of Invention

The embodiment of the application provides a circuit system for Harris corner detection and a method for detecting a corner, and aims to solve the technical problem that the line cache unit management efficiency is low in the prior art.

In a first aspect, there is provided circuitry for Harris corner detection, the circuitry comprising: the device comprises a line cache module, a window generation module, a partial derivative calculation module, a corner response module and an extreme value suppression module, wherein:

the line cache module is used for receiving an input image, dividing the input image into a plurality of cache blocks in the line cache module according to a single line of image information, and storing the divided image into the plurality of cache blocks, wherein each cache block in the plurality of cache blocks correspondingly stores a line of image information;

the window generation module is used for reading the image information cached in the line cache module in a preset mode and performing storage processing on the read or received information in a preset format to obtain rectangular window data;

the partial derivative calculation module is used for receiving first rectangular window data corresponding to the cached image information and performing parallel partial derivative operation on the first rectangular window data in a row unit to obtain horizontal partial derivative data and vertical partial derivative data which are output in a parallel mode;

the corner response module is configured to calculate corner response values in parallel to obtain a plurality of corner response values, so that the extremum suppression module performs maximum suppression processing on second rectangular window data corresponding to the plurality of corner response values to determine a corner value.

In a possible implementation manner, the preset manner is that the window generation module reads a single-line cache image cached in the plurality of cache blocks, and simultaneously reads information of the single-line image to be cached.

In a possible embodiment, the window generating module processes the read or received information through a cascaded pipeline structure with a number of rows set corresponding to the information, so as to output rectangular window data in a predetermined format.

In a possible implementation manner, the circuit system further includes a gaussian filtering module, where the gaussian filtering module is configured to perform gaussian filtering on third rectangular window data corresponding to the horizontal partial derivative data and the vertical partial derivative data output in a parallel manner after being processed by the partial derivative calculation module, and send the data after the gaussian filtering to the corner response module.

In a possible implementation, the gaussian filtering module includes a plurality of gaussian filtering units to process the third rectangular window data in parallel.

In a second aspect, a detection method is provided, which is applied to a circuit system for Harris corner detection, where the circuit system includes a line buffer module, a window generation module, a partial derivative calculation module, a corner response module, and an extremum suppression module, and the method includes:

receiving input image information, and dividing and storing the image information cache into a plurality of cache blocks in the line cache module by using single-line image information;

reading the image information cached in the line cache module in a preset mode, and performing storage processing on the cached image in a preset format to obtain first rectangular window data;

performing parallel partial derivative operation on the first rectangular window data by using a row unit to obtain horizontal partial derivative data and vertical partial derivative data which are output in a parallel mode;

performing storage processing on the horizontal partial derivative data and the vertical partial derivative data which are output in a parallel mode in a preset format to obtain second rectangular window data, and calculating corner response values in parallel according to the second rectangular window data to obtain a plurality of corner response values;

and performing storage processing on the plurality of corner response values in a preset format to obtain third rectangular window data, and performing maximum suppression processing on the third rectangular window data to determine the corner values.

In a possible implementation manner, reading the image information cached in the line cache module in a preset manner specifically includes:

reading the single-line cache image cached in the plurality of cache blocks, and simultaneously reading the information of the single-line image to be cached.

In one possible implementation, the predetermined format storage process is performed, and includes:

and processing the read or received information through a cascade pipeline structure set by the line number corresponding to the information so as to output rectangular window data in a preset format.

In a possible implementation, if the circuitry further includes a gaussian filtering module, after obtaining the second rectangular window data, the method further includes:

and performing parallel Gaussian filtering processing on the second rectangular window data to obtain the data after the Gaussian filtering processing.

In one possible embodiment, the method further comprises:

and carrying out parallel calculation on the data after the Gaussian filtering to obtain a plurality of corner response values corresponding to the data after the Gaussian filtering.

The technical scheme provided by the embodiment of the application at least has the following beneficial effects:

in the embodiment of the application, a setting mode of dividing a line cache module into cache blocks and storing input image information into a single cache block in a single line is adopted, centralized storage management can be performed on the image information cached in the single line, and further the management efficiency of a line cache unit can be improved.

Specifically, the specific processing procedure of the corner detection method is as follows: the method comprises the steps of dividing a cache block by a line cache module, and storing input image information into a single cache block in a single line. Further, the window generation module reads the image information cached in the line cache module in a preset mode, and performs storage processing on the cached image information in a preset format to obtain first rectangular window data; then, the offset-derivative calculation module performs offset-derivative operation on the first rectangular window data to obtain horizontal offset-derivative data and vertical offset-derivative data which are output in a parallel mode, so that the window generation module processes the horizontal offset-derivative data and the vertical offset-derivative data which are output in the parallel mode to obtain second rectangular window data, so that the corner response module calculates the second rectangular window data to obtain a plurality of corner response values, and the extreme value suppression module processes the plurality of processed corner response values to obtain corner values.

That is to say, in the embodiment of the present application, since the cache blocks in the line cache module are subjected to centralized storage management, and a plurality of parallel processing sub-units are provided for the partial derivative calculation module, the corner response module, and the extremum suppression module, the number of line cache units can be effectively reduced, and particularly when a high-resolution image is processed, a large area advantage is provided, that is, no line cache unit needs to be added, so that the detection of a corner can be rapidly realized.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application and are not to be construed as limiting the application.

FIG. 1 is a schematic diagram of a prior art circuit cache system;

FIG. 2 is a schematic diagram of circuitry for a Harris corner detection algorithm according to an embodiment of the present application;

FIG. 3 is a diagram illustrating a process of caching an image by a line cache module according to an embodiment of the present application;

FIG. 4 is a diagram illustrating a window generation module processing information according to an embodiment of the present application;

FIG. 5 is another schematic diagram of circuitry for a Harris corner detection algorithm according to an embodiment of the present application;

FIG. 6 is a diagram illustrating a calculation process of a partial derivative calculation module according to an embodiment of the present disclosure;

fig. 7 is a schematic flowchart of a detection method according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application. In the present application, the embodiments and features of the embodiments may be arbitrarily combined with each other without conflict. Also, while a logical order is shown in the flow diagrams, in some cases, the steps shown or described may be performed in an order different than here.

The term "comprising" and any variations thereof in the description and claims of this application and the above-described drawings are intended to cover non-exclusive protection. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

As mentioned above, in the prior art, due to its algorithmic characteristics, Harris corner detection requires a plurality of line buffer units when designing a circuit, and as the resolution of a processed image increases, the area consumed by line buffer increases.

Specifically, referring to fig. 1, the line cache units in the prior art are arranged in a distributed manner, and the existing window generator can only read the information of the 3-line cache, and after performing the partial derivative calculation on the information of the 3-line cache, the window generator needs to store the data after the partial derivative calculation into the line cache module in the distributed storage again, and then reads the information again, and then performs the subsequent processing, that is, the management efficiency of the line cache is low, and a large circuit area is required.

In view of this, the present application provides a circuit system for a Harris corner detection algorithm, and the system can provide a novel internal line cache module to perform centralized cache on an input image according to the characteristic of the Harris algorithm, and then, by combining a multi-line window generation module and a parallel circuit design, the number of line cache units can be effectively reduced, thereby improving the management efficiency of the line cache units.

The technical scheme provided by the embodiment of the application is described in the following with the accompanying drawings of the specification.

Referring to fig. 2, an embodiment of the present application provides a circuit system for a Harris corner detection algorithm, and a block diagram of the circuit system is shown in fig. 2.

Referring to fig. 2, the circuit system includes a line buffer module, a window generation module, a partial derivative calculation module, a corner response module, a gaussian filter module, and an extremum suppression module.

In this embodiment of the present application, the line cache module is configured to receive an input image, divide the input image into a plurality of cache blocks in the line cache module according to single-line image information, and store the divided image into the cache blocks, where each cache block in the cache blocks stores a line of image information; the window generation module is used for reading the image information cached in the line cache module in a preset mode and performing storage processing on the read or received information in a preset format to obtain rectangular window data; the partial derivative calculation module is used for receiving first rectangular window data corresponding to the cached image information and performing parallel partial derivative operation on the first rectangular window data in a row unit to obtain horizontal partial derivative data and vertical partial derivative data which are output in a parallel mode; and the corner response module is used for calculating the corner response values in parallel to obtain a plurality of corner response values, so that the extreme value suppression module performs maximum value suppression processing on the second rectangular window data corresponding to the plurality of corner response values to determine the corner values.

Furthermore, the circuit system in the application further includes a gaussian filtering module, configured to perform gaussian filtering on third rectangular window data corresponding to the horizontal partial derivative data and the vertical partial derivative data that are output in a parallel manner and processed by the partial derivative calculation module, and send the data after the gaussian filtering to the corner response module.

In this embodiment of the present application, the line cache module includes a read/write control unit and a cache unit including a plurality of cache blocks, and each cache block may cache a line of map information correspondingly. In a specific implementation process, the cache unit may use an SRAM (Static Random Access Memory), for example, the cache unit is composed of six SRAMs, and certainly, the cache unit may also be composed of seven SRAMs, which is not limited in this embodiment of the present application.

In this embodiment of the present application, when the read-write control unit in the line cache module performs read-write control on the cache units in the line cache module, and when an input image cache unit is not full of cache blocks, the input image cache unit may be sequentially written into the cache blocks of the cache unit according to a single-line division sequence of the input image until the cache blocks of the cache unit are full of cache blocks. When the cache block of the cache unit is full, if a new line of data is input, the read-write control module may read multiple lines of image data cached in the cache block, and simultaneously read the new line of data, and simultaneously output the read data to the window generation module, and at the same time, may write the new line of data into the first cache block.

Referring to fig. 3, a schematic diagram of information cached by the line cache module in the embodiment of the present application is provided by taking an example that the cache unit is composed of six cache blocks, in a specific implementation process, the read-write control unit in the line cache module performs read-write control on the cache units in the line cache module, and when the cache units are not full of six rows of data, the cache units are sequentially written into the cache units according to a single-line division order of the input image until the six rows of data are full of data. When the buffer unit is full of six rows of data, namely, the six rows of data can be understood as LineN-N +5 in FIG. 3, wherein N is a positive integer greater than or equal to 1, when the LineN +6 data is input, the read-write control module can read the LineN-N +5 data, output the seven rows of data of LineN-N +6 to the window generation module in columns, and at the same time, write the LineN +6 data into the buffer block originally written into the LineN image information, namely, when the LineN + 1-N +6 data is read, the complete LineN + 1-N +6 data can be obtained, and then the input images are sequentially buffered in such a way, so that the buffer processing of the input whole frame image can be completed.

In this embodiment of the application, after the cache block included in the cache unit in the line cache module is cached fully, the window generation module may read the single-line cache image cached in the plurality of cache blocks and simultaneously read the information of the single-line image to be cached, specifically, the predetermined line number corresponds to the number of cache blocks of the line cache module, for example, the cache image of six lines may be simultaneously read by columns, and the data of the cache image of the seventh line may be simultaneously read.

In this embodiment, when the window generation module reads the image information, a specific processing manner may be that the read image information is processed through a cascade pipeline structure in which the number of lines corresponding to the image information is set, so as to output rectangular window data in a predetermined format. In a specific implementation process, when the window generation module reads seven lines of image information, the register corresponding to each line adopts a cascaded pipeline structure, and data of 7 × 3 windows can be output in each clock cycle. Specifically, please refer to fig. 4, where fig. 4 is a schematic diagram of processing information by the window generation module in the embodiment of the present application.

In a specific implementation process, after the window generation module receives the horizontal partial derivative data and the vertical partial derivative data input by the partial derivative calculation module, if the partial derivative calculation module includes five parallel partial derivative calculation units, the window generation module may store five rows of data in the register array, a register corresponding to each row adopts a cascaded pipeline structure, and data of 5 × 3 windows is output every clock cycle.

In a specific implementation process, after the window generation module receives data sent by the corner response module, if the corner response module includes three corner response operation processing units, the window generation module may store three lines of received information in the register array, a register corresponding to each line adopts a cascaded pipeline structure, and data of 3 × 3 windows is output every clock cycle.

It should be noted that, in this embodiment of the application, the window generation module may be understood as a window generator, and the registers in each row of the window generation module that adopt a cascaded pipeline structure and a specific processing manner may be understood as the same or similar manner, which may refer to the schematic diagram of fig. 4 and will not be described herein again.

To facilitate understanding of the circuit system provided in the present application, a specific example is used for the following description, please refer to fig. 5, specifically, for example, Harris corner detection of a 3 × 3 input template may be taken as an example, when an input image L is written into a line cache module by lines, and the line cache module is provided with six SRAMs in total.

When six lines of images are buffered in the line buffer module, the window generator 1 in the window generation module may read the six lines of images by columns and simultaneously read data of a seventh line adjacent to the sixth line. Then, the window generator 1 stores the seven read lines of images by an internal 7 × 3 register array, and outputs the seven lines of images to the partial derivative calculation module according to a 7 × 3 rectangular window.

In a specific implementation process, the partial derivative calculation module adopts a pipeline design, and can calculate the partial derivative Ix in the X direction and the partial derivative Iy in the Y direction for the input 7 × 3 data. The specific processing procedure can be shown in fig. 6, wherein the partial derivative calculation module can be divided into 5 parallel partial derivative calculation units (i.e., the partial derivative calculation unit 1, the partial derivative calculation unit 2, the partial derivative calculation unit 3, the partial derivative calculation unit 4, and the partial derivative calculation unit 5 in fig. 6), calculate partial derivatives of 5 3 × 3 arrays, and output Ix and Iy of 5 rows of images in parallel, i.e., Ix _0/Iy _0 in fig. 6; ix _1/Iy _ 1; ix _2/Iy _ 2; ix _3/Iy _ 3; ix _4/Iy _ 4.

Further, the window generator 2 in the window generation module may receive 5 rows of data output by the partial derivative calculation module and convert the data into 5 × 3 rectangular windows of data, so as to output the data to the gaussian filter module, and the gaussian filter module may longitudinally divide the input 5 × 3 windows into 3 × 3 windows for gaussian filter processing, specifically, the gaussian filter module may be provided with three gaussian filter units, so as to process the data in parallel, and finally output three rows of gaussian filtered data (e.g. Ix ^2/Iy ^2/Ixy ^2 in fig. 5).

Then the corner response module can calculate the response R of the corner for three lines of data, and input the R values of the three lines into the window generator 3 in the window generation module after calculating the R values of the three lines, then can generate the R rectangular window of 3x3, then calculate the corner maximum value by the extreme value suppression module, and finally calculate the corner and output.

The window generation module, the partial derivative calculation module, the Gaussian filter module, the angular point response module and the extreme value suppression module in the circuit system are all designed in a production line mode, and the partial derivative calculation module, the Gaussian filter module and the angular point response module are all designed in a parallel mode, so that multiple lines of data can be calculated and output simultaneously.

The line cache module carries out centralized management on the cache units, combines the multi-line window generation module, the parallel partial derivative calculation module, the Gaussian filter module and the angular point response module, can realize the minimum number of the line cache units, and further can improve the management efficiency of the line cache units.

In addition, as can be seen from multiple tests by the applicant of the present application, taking a corner detection window of 3 × 3 as an example, only 6 lines of input image cache units are needed to implement Harris corner detection, and compared with a distributed line cache unit design, as shown in the structural design of fig. 1, that is, 9 line cache units are needed in the prior art, the line cache module provided in the embodiment of the present application can reduce 3 line cache units. And the area effect of the line cache unit is particularly obvious in images with higher resolution, for example, images with a high definition of 1080p, each cache unit needs to store 1920 image pixels, and the method provided by the application only uses 6 SRAMs with corresponding sizes as the line cache unit, so that compared with the prior art that 3 SRAMs with the same size can be saved, the cost can be saved, and the management of the line cache unit can be improved.

Based on the same inventive concept, the present application further provides a detection method, which is applied to the circuit system for the Harris corner detection algorithm, where the circuit system includes a line cache module, a window generation module, a partial derivative calculation module, a corner response module, and an extremum suppression module, and a flowchart of the method is shown in fig. 7 and described in detail below.

Step 701: receiving input image information, caching the image information, dividing the image information into single-line image information, and storing the single-line image information into a plurality of cache blocks in a line cache module;

step 702: reading image information cached in a line cache module in a preset mode, and performing storage processing on the cached image in a preset format to obtain first rectangular window data;

step 703: performing parallel partial derivative operation on the first rectangular window data by using a row unit to obtain horizontal partial derivative data and vertical partial derivative data which are output in a parallel mode;

step 704: performing predetermined format storage processing on the horizontal partial derivative data and the vertical partial derivative data which are output in a parallel mode to obtain second rectangular window data, and calculating corner response values in parallel according to the second rectangular window data to obtain a plurality of corner response values;

step 705: and performing preset format storage processing on the plurality of corner response values to obtain third rectangular window data, and performing maximum suppression processing on the third rectangular window data to determine the corner values.

In one possible embodiment, the method further comprises:

As will be appreciated by one skilled in the art, embodiments of the present disclosure may be provided as a method, system, or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.

The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims

1. Circuitry for a Harris corner detection algorithm, the circuitry comprising: the device comprises a line cache module, a window generation module, a partial derivative calculation module, a corner response module and an extreme value suppression module, wherein:

2. The circuit system of claim 1, wherein the predetermined manner is that the window generation module reads a single-line buffer image buffered in the plurality of buffer blocks and simultaneously reads information of the single-line image to be buffered.

3. The circuitry of claim 1, wherein the window generation module processes the read or received information through a cascaded pipeline structure having a set number of rows corresponding to the information to cause rectangular window data of a predetermined format to be output.

4. The circuitry of claim 1, further comprising a gaussian filter module configured to perform gaussian filtering on the third rectangular window data corresponding to the parallel output horizontal and vertical partial derivative data processed by the partial derivative calculation module, and send the gaussian filtered data to the corner response module.

5. The circuitry of claim 4, wherein the Gaussian filter module includes multiple Gaussian filter units to process the third rectangular window of data in parallel.

6. A detection method applied to a circuit system for a Harris corner detection algorithm, the circuit system comprising a line buffer module, a window generation module, a partial derivative calculation module, a corner response module, and an extremum suppression module, the method comprising:

7. The method of claim 6, wherein reading the image information cached in the line cache module in a preset manner specifically comprises:

8. The method of claim 6, wherein performing a predetermined format storage process comprises:

9. The method of claim 6, wherein after obtaining second rectangular window data if the circuitry further comprises a Gaussian filter module, the method further comprises:

10. The method of claim 9, wherein the method further comprises: