CN112488177A - Image matching method and related equipment - Google Patents
Image matching method and related equipment Download PDFInfo
- Publication number
- CN112488177A CN112488177A CN202011353325.XA CN202011353325A CN112488177A CN 112488177 A CN112488177 A CN 112488177A CN 202011353325 A CN202011353325 A CN 202011353325A CN 112488177 A CN112488177 A CN 112488177A
- Authority
- CN
- China
- Prior art keywords
- image
- matched
- resolution
- preprocessed
- matching
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 104
- 238000012545 processing Methods 0.000 claims abstract description 82
- 230000008569 process Effects 0.000 claims abstract description 57
- 238000012935 Averaging Methods 0.000 claims description 13
- 238000001914 filtration Methods 0.000 claims description 10
- 238000004801 process automation Methods 0.000 claims description 9
- 230000002085 persistent effect Effects 0.000 claims description 3
- 230000001960 triggered effect Effects 0.000 claims description 3
- 230000001052 transient effect Effects 0.000 claims description 2
- 238000012795 verification Methods 0.000 abstract description 2
- 238000001514 detection method Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 6
- 238000007781 pre-processing Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000002222 downregulating effect Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000007667 floating Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Databases & Information Systems (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application discloses an image matching method, which is applied to the automatic working process of a robot process and comprises the following steps: acquiring a template image; setting a preset resolution ratio based on the template image; acquiring an image to be matched; judging whether the resolution of the image to be matched meets a preset resolution or not; if the resolution of the image to be matched does not accord with the preset resolution, performing resolution processing on the image to be matched to obtain a preprocessed image; and matching the preprocessed image with the template image. According to the technical scheme, the resolution verification is carried out on the picture to be matched before the picture matching process is executed, and whether the picture to be matched meets the preset resolution is judged. And if the image to be matched does not meet the requirement, performing resolution processing on the image to be matched to enable the resolution of the image to be matched to meet the requirement, obtaining a preprocessed image, and performing matching processing by using the preprocessed image to finish the image matching process, so that the problem of identification accuracy caused by large resolution difference in the image matching process is solved.
Description
Technical Field
The embodiment of the application relates to the field of image processing, in particular to an image matching method and related equipment.
Background
The Robot Process Automation (RPA) system is an application program, and the RPA system provides another way to automate a manual operation Process of a user by simulating a manual operation mode of the user at a computer.
In conventional workflow automation technology tools, an action list of automation tasks is generated by a programmer and an internal application program interface or a dedicated scripting language is used as an interface with a background system. Robot process automation monitors the User's work done on a Graphical User Interface (GUI) in an application and automatically repeats the work directly on the GUI.
Template pictures are preset for a subsequent detection matching process during the design of the conventional RPA robot. When the RPA robot runs, the image displayed by the current equipment is obtained to be used as the image to be matched, the image to be matched and the template image are matched to find whether the template image contains the detection image or not, and the detection result is returned. However, there may be a difference between the resolution of the picture to be matched obtained in the detection process and the resolution of the template picture, and this difference may cause unnecessary errors in the matching detection process, which causes certain inconvenience in use.
Disclosure of Invention
The first aspect of the embodiments of the present application provides an image matching method, which is applied to robot process automation, and includes:
executing an RPA service flow to obtain an image to be matched;
acquiring a template image, and acquiring a preset resolution of the template image;
judging whether the resolution of the image to be matched meets the preset resolution or not;
if the resolution of the image to be matched does not accord with the preset resolution, performing resolution processing on the image to be matched to obtain a preprocessed image;
and matching the preprocessed image with the template image.
Based on the image matching method provided in the first aspect of the embodiment of the present application, optionally, if the resolution of the image to be matched does not meet the preset resolution, performing resolution processing on the image to be matched to obtain a preprocessed image, including:
and if the resolution of the image to be matched is smaller than the preset resolution, carrying out bicubic interpolation processing on the image to be matched to obtain a preprocessed image.
Based on the image matching method provided in the first aspect of the embodiment of the present application, optionally, if the resolution of the image to be matched does not meet the preset resolution, performing resolution processing on the image to be matched to obtain a preprocessed image, including:
and if the resolution of the image to be matched is greater than the preset resolution, carrying out bit operation averaging processing on the image to be matched to obtain a preprocessed image.
Based on the image matching method provided in the first aspect of the embodiment of the present application, optionally, before the averaging processing is performed on the image to be matched to obtain the preprocessed image, the method further includes:
and performing Gaussian filtering on the image to be matched.
Based on the image matching method provided in the first aspect of the embodiment of the present application, optionally, the method is applied to a working process of a robot process automation system.
Based on the image matching method provided in the first aspect of the embodiment of the present application, optionally, the matching processing of the preprocessed image and the template image includes:
and judging whether the preprocessed image comprises the template image or not, and returning a judgment result.
Based on the image matching method provided in the first aspect of the embodiment of the present application, optionally, the image to be matched is obtained from a screenshot of a current user interface.
A second aspect of the embodiments of the present application provides an image matching apparatus, which is applied to robot process automation, and includes:
the first acquisition unit is used for executing the RPA service process and acquiring an image to be matched;
the second acquisition unit is used for acquiring an image to be matched and acquiring the preset resolution of the template image;
the judging unit is used for judging whether the resolution of the image to be matched meets the preset resolution or not, and if the resolution of the image to be matched does not meet the preset resolution, the resolution processing unit is triggered;
the resolution processing unit is used for carrying out resolution processing on the image to be matched to obtain a preprocessed image;
and the matching unit is used for matching the preprocessed image with the template image.
A third aspect of embodiments of the present application provides a computer, including:
a central processing unit, a memory;
the memory is a transient memory or a persistent memory;
the central processing unit is configured to communicate with the memory, and the instructions in the memory are executed on the computer to perform the method according to any one of the first aspect of the embodiments of the present application.
A fourth aspect of embodiments of the present application is a computer-readable storage medium, including instructions, which, when executed on a computer, cause the computer to perform the method according to any one of the first aspect of embodiments of the present application.
According to the technical scheme, the embodiment of the application has the following advantages: in the RPA process, before the picture matching process is executed, resolution identification verification is carried out on the picture to be matched, and whether the picture to be matched meets the preset resolution is judged. The preset resolution is set by the template picture. And if the resolution ratio of the image to be matched is not met, performing resolution ratio processing on the image to be matched to enable the resolution ratio of the image to be matched to meet the requirement, obtaining a preprocessed image, and performing matching processing by using the preprocessed image to finish the image matching process, so that the problem of identification accuracy caused by large resolution ratio difference in the image matching process is solved, and the accuracy and the usability of RPA execution are improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a schematic flow chart diagram illustrating an embodiment of an image matching method according to the present application;
FIG. 2 is a schematic flow chart of an embodiment of an image matching method according to the present application;
FIG. 3 is a schematic structural diagram of an embodiment of an image matching apparatus according to the present application;
fig. 4 is another schematic structural diagram of an embodiment of the image matching apparatus of the present application.
Detailed Description
The embodiment of the application provides an image matching method, which is used for completing a matching process between images to be matched with different resolutions and template images. The preset resolution is set by the template picture. And if the resolution ratio of the image to be matched is not met, performing resolution ratio processing on the image to be matched to enable the resolution ratio of the image to be matched to meet the requirement, obtaining a preprocessed image, and performing an image matching process with the template image by using the preprocessed image.
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the drawings described above, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that the descriptions in this application referring to "first", "second", etc. are for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present application.
Referring to fig. 1, an embodiment of an image matching method of the present application includes: step 101-step 105. The image matching method is applied to a Robot Process Automation (RPA) working Process.
101. And executing the RPA service process to obtain the image to be matched.
The RPA robot executes the RPA service process to obtain an image to be matched, the image to be matched is an image to be matched with the template image, specifically, the image to be analyzed can be obtained by capturing a picture of a current user interface, and the RPA robot executes a corresponding service process to capture the current user interface to obtain a corresponding image to be matched. It is understood that the image to be matched may also be an image obtained by the RPA robot executing other business processes, which may be determined according to actual situations, and is not limited herein. Since the image to be analyzed is obtained based on the current use environment of the user, the obtained image to be matched may have a difference in resolution from the template image, and resolution processing of the image to be matched is required.
102. And acquiring a template image and acquiring the preset resolution of the template image.
And acquiring a template image and acquiring the preset resolution of the template image. The template image is used for performing a subsequent matching process, and the template image can be manually selected and preset by a user in the implementation process of the scheme, specifically, images such as buttons, input fields and the like are selected. The image may also be an image such as a screenshot of the current user node, which may be determined according to the actual situation, and is not limited herein. The server sets a preset resolution ratio based on the template image, and the server can set the preset resolution ratio according to the template image for ensuring the efficiency of the image matching process. Specifically, the server can identify the resolution of the template image, obtain the resolution data of the template image and set the value of the preset resolution to be the same as the resolution of the template image, and can adjust the resolution of the template image to a certain degree, such as floating up a certain value or down-regulating the value, so as to obtain the data range corresponding to the preset resolution.
103. And judging whether the resolution of the image to be matched meets the preset resolution or not.
And the server judges whether the resolution of the image to be matched meets the preset resolution or not. The server extracts and identifies the resolution of the image to be matched, obtains resolution data of the image to be matched, compares the resolution of the image to be matched with a preset resolution, judges whether the resolution is the same as the preset resolution or meets the requirement of a preset resolution range, if the resolution of the image to be matched does not meet the preset resolution, if the resolution of the image to be matched is larger than or smaller than the value or range of the preset resolution, the step 104 is executed, the resolution of the image to be matched is processed, and a preprocessed image is obtained. If the resolution of the image to be matched is the same as the preset resolution or meets the range requirement of the preset resolution, it is indicated that there is no difference or a small difference between the resolution of the image to be matched and the resolution of the template image, and the accuracy of matching the image to be matched by using the template image is high, and then the image matching process of the image to be matched and the template image or other steps can be executed, and the specific details are not limited herein.
104. And carrying out resolution processing on the image to be matched to obtain a preprocessed image.
And the server carries out resolution processing on the image to be matched to obtain a preprocessed image. The method comprises the following steps that a preprocessed image is obtained by processing a resolution of an image to be matched, a resolution processing mode of the image to be matched can be determined according to actual conditions, specifically, a comparison result between the resolution of the image to be matched and a preset resolution by a server can be a preprocessed vector used for representing the resolution condition of the image to be matched, corresponding processing is carried out based on the pointing direction of the vector, if the resolution of the image to be matched is smaller than the preset resolution, the preprocessed vector points to a processing mode with smaller resolution, and specifically, if the resolution of the image to be matched is smaller than the preset resolution, the processing mode can be: the image is interpolated, namely corresponding pixel points are inserted into the image by utilizing the known gray value or tristimulus value of the adjacent pixel points in the image to be matched, the gray value or tristimulus value of the inserted pixel points is obtained by the corresponding attribute of the known pixel points, and the resolution of the image to be matched is improved. If the resolution of the image to be matched is greater than the preset resolution, the preprocessing vector points to the processing mode when the resolution is greater, and specifically, for the case that the resolution of the image to be matched is greater than the preset resolution, the processing mode may be: and (3) carrying out averaging treatment, namely carrying out average operation on the gray values or the tristimulus values of a plurality of pixel points of adjacent areas and fusing the calculated plurality of pixel points into a similar point, wherein the gray values or the tristimulus values of the pixel points are average values obtained after carrying out average operation on the gray values or the tristimulus values of the plurality of pixel points. It can be understood that, in the actual implementation process of the present solution, the pictures to be matched belonging to different situations may be processed in different ways to obtain the preprocessed pictures meeting the requirements, which may be determined specifically according to the actual situations, and is not limited herein.
105. And matching the preprocessed image with the template image.
And the server performs matching processing on the preprocessed image and the template image. Specifically, the template image may be set as an image of some buttons, connections, or input boxes in a GUI (Graphical User Interface), so that the processing result of matching the preprocessed image and the template image is generally output as that the preprocessed image includes the template image or that the preprocessed image does not include the template image, and the server based on the processing result of matching may perform the next processing procedure, such as clicking a button or connecting to input a corresponding character, and the like, which is not limited herein.
According to the technical scheme, the embodiment of the application has the following advantages: according to the scheme, before the picture matching process is executed, the resolution of the picture to be matched is verified, and whether the picture to be matched meets the preset resolution or not is judged. The preset resolution is set by the template picture. And if the resolution ratio of the image to be matched is not met, performing resolution ratio processing on the image to be matched to enable the resolution ratio of the image to be matched to meet the requirement, obtaining a preprocessed image, and performing matching processing by using the preprocessed image to finish the image matching process, so that the problem of identification accuracy rate caused by large resolution ratio difference in the image matching process is solved, and the accuracy and the usability of the RPA robot in the service execution process are improved.
Based on the embodiment described in fig. 1, a detailed embodiment of the present invention that can be selectively executed in the implementation process is provided below, and referring to fig. 2, an embodiment of the present application includes: step 201-step 208.
201. And executing the RPA service process to obtain the image to be matched.
And the RPA robot executes the RPA service process to obtain the image to be matched. A Robotic Process Automation (RPA) system is an application that provides another way to automate an end user's manual process by mimicking the way an end user manually operates on a computer. The image to be matched is an image to be matched with the template image, specifically, the image to be analyzed can be obtained by capturing a current user interface, and the RPA robot can capture the current user interface by executing a corresponding service process to obtain a corresponding image to be matched. It is understood that the image to be matched may also be an image obtained by the RPA robot executing other business processes, which may be determined according to actual situations, and is not limited herein. Since the image to be analyzed is obtained based on the current use environment of the user, the obtained image to be matched may have a difference in resolution from the template image, and resolution processing of the image to be matched is required.
202. And acquiring a template image and acquiring the preset resolution of the template image.
And acquiring a template image and acquiring the preset resolution of the template image. The template image is used for performing a subsequent matching process, and the template image can be manually selected and preset by a user in the implementation process of the scheme, specifically, images such as buttons, input fields and the like are selected. The image may also be an image such as a screenshot of the current user node, which may be determined according to the actual situation, and is not limited herein. When the RPA robot is used, if a picture which is the same as or similar to the template picture is detected, clicking and other operations are triggered so as to complete the functions of the RPA robot, and the manual consumption of a user is reduced. The template picture can be stored in the server together with the designed RPA robot so as to directly work on other equipment without setting the template picture again. The server sets a preset resolution ratio based on the template image, and the server can set the preset resolution ratio according to the template image for ensuring the efficiency of the image matching process. Specifically, the server can identify the resolution of the template image, obtain the resolution data of the template image and set the value of the preset resolution to be the same as the resolution of the template image, and can adjust the resolution of the template image to a certain degree, such as floating up a certain value or down-regulating the value, so as to obtain the data range corresponding to the preset resolution.
203. And carrying out gray processing on the image to be matched.
And carrying out gray processing on the image to be matched. Generally, to ensure the accuracy of the identification process, gray processing needs to be performed on the image to be matched so as to avoid the identification failure caused by problems such as color difference, and the like. It can be understood that, for the case that the template image is a gray image, the matching image can be adjusted only so as to perform the subsequent matching process, and for the case that the template image is a color, the gray processing can be performed on the template image and the matching image at the same time so as to avoid the adverse effect of the RGB color factors on the matching process.
204. And judging whether the resolution of the image to be matched meets the preset resolution or not.
And the RPA robot judges whether the resolution of the image to be matched meets the preset resolution or not. The RPA robot extracts the resolution of an image to be matched, specifically, after a current interface screenshot image is obtained, the resolution is stored in a screenshot image file as attribute data of the image on one hand, a server can read the image file to obtain the resolution data of the image to be matched, the resolution data of the image to be matched is compared with a preset resolution to obtain a preprocessing vector, if the preprocessing vector is equal to 1, the resolution of the image to be matched is the same as that of a template image, and the matching process of the image to be matched and the template image can be directly carried out without carrying out resolution processing on the image to be matched. If the preprocessing vector is smaller than 1, it indicates that the resolution of the image to be matched is lower than the resolution of the template image, and step 205 is executed to perform interpolation processing on the image to be matched to obtain a preprocessed image. If the preprocessing vector is greater than 1, it indicates that the resolution of the image to be matched is higher than that of the template image, and step 206 is executed to perform gaussian filtering on the image to be matched.
It can be understood that, in the process of determining the resolution of the image to be matched, other processing manners not only including step 205 or step 206 may be set for the conditions of different resolutions, and if the resolution of the image to be matched is too small, that is, smaller than a certain predetermined value, the image to be matched is determined to be invalid, and the image to be matched meeting the requirement is obtained again, which may be determined according to the actual situation, and is not limited herein.
205. And carrying out bicubic interpolation processing on the image to be matched to obtain a preprocessed image.
And carrying out bicubic interpolation processing on the image to be matched to obtain a preprocessed image. If the resolution of the image to be matched is lower than that of the template image, performing bicubic interpolation processing on the image to be matched so as to improve the resolution of the image to be matched and further improve the matching accuracy between the subsequent preprocessed image and the template image, wherein the interpolation processing is to insert corresponding pixel points into the image by utilizing the known gray values or the known tristimulus values of adjacent pixel points in the image to be matched, and the gray values or the tristimulus values of the inserted pixel points are obtained by the corresponding attributes of the known pixel points so as to improve the resolution of the image to be matched, in the actual implementation process, the interpolation processing can be performed by adopting a GetBuicyVal (bicubic interpolation) function of OPENCV (a cross-platform computer vision and machine learning software library which is licensed based on Berkeley software suite), and it can be understood that in the interpolation processing process, the used software can be adjusted according to the actual situation, the interpolation times can also be adjusted according to the self requirements and the attributes of the images to be matched, if the resolution of the images to be matched is too low, the interpolation times are increased so as to obtain the preprocessed images meeting the requirements, and the interpolation times can be determined according to the actual situation, and are not limited here. And after the RPA robot carries out interpolation processing on the image to be matched, acquiring a required preprocessed image, executing step 208 based on the acquired preprocessed image, judging whether the preprocessed image comprises the template image, and returning a judgment result.
206. And performing Gaussian filtering on the image to be matched.
If the resolution of the image to be matched is higher than that of the template image, firstly, gaussian filtering is performed on the image to be matched. Specifically, the RPA robot may perform smoothing and denoising processing on the image to be matched by using a gaussian function (gaussian filter) of OPENCV. The Gaussian filtering is a filtering mode for noise reduction, the Gaussian noise in the image can be effectively reduced through the Gaussian filtering, the influence of the Gaussian noise in the high-resolution image to be matched on the resolution reduction processing process of the matched image is further avoided, and the accuracy of image matching between the preprocessed image and the template image is further improved.
It can be understood that, in the actual implementation process, the present solution may also directly execute step 207 to perform averaging processing on the image without performing gaussian filtering, so as to reduce the computing resources consumed by the operation of the RPA robot and improve the processing speed, which may be determined specifically according to the actual situation, and is not limited herein.
207. And carrying out bit operation averaging processing on the image to be matched to obtain a preprocessed image.
And carrying out bit operation averaging processing on the image to be matched to obtain a preprocessed image. The method comprises the steps of performing Gaussian filtering on an image to be matched, performing bit operation averaging processing on the image to be matched, processing bottom data of the image in a bit operation mode, obtaining an averaging processing result, and recombining the image to form an image with reduced resolution, specifically, performing average operation on gray values or tristimulus values of a plurality of pixel points in adjacent regions, and fusing the calculated plurality of pixel points into a similar point, wherein the gray values or the tristimulus values of the pixel points are average values obtained after the average operation is performed on the gray values or the tristimulus values of the plurality of pixel points. In this way, the number of pixel points included in the image to be matched is reduced, and the resolution of the image to be matched is further reduced, so that the preprocessed image meeting the preset resolution requirement is obtained. It can be understood that the resolution of the preprocessed image obtained after the single averaging processing may not meet the requirement, and the resolution of the processed image may be identified again and repeatedly processed to obtain the preprocessed image meeting the requirement. The RPA robot performs averaging processing on the image to be matched to obtain a required preprocessed image, performs step 208 based on the obtained preprocessed image, determines whether the preprocessed image includes the template image, and returns a determination result.
208. And judging whether the preprocessed image comprises the template image or not, and returning a judgment result.
And the RPA robot judges whether the preprocessed image comprises the template image or not and returns a judgment result. Specifically, the RPA robot can use an OPENCV image frame to match the template picture and the detection picture by adopting a standard correlation matching algorithm (parameter setting: CV _ TM _ CCOEFF _ NORMED), find whether the template picture contains the detection picture, and return a detection result. It is understood that the software and algorithm used in the image matching process may be adjusted according to the actual situation, and are not limited herein. After obtaining the determination result, the RPA robot may click or input other information according to the identification result to complete the use process of the RPA robot, which may be determined according to the actual situation, and is not limited herein.
According to the technical scheme, the embodiment of the application has the following advantages: according to the scheme, before the picture matching process is executed, the resolution of the picture to be matched is verified, and whether the picture to be matched meets the preset resolution or not is judged. The preset resolution is set by the template picture. And if the resolution of the picture to be matched is greater than the preset resolution, performing averaging processing on the picture to be matched to reduce the resolution of the picture to be matched. The method comprises the steps of respectively processing the pictures to be processed in different modes under different conditions to further obtain the preprocessed pictures which are consistent with or have less difference with the preset resolution, matching the preprocessed pictures with the template pictures, judging whether the template pictures exist in the preprocessed pictures or not, and returning a corresponding judgment result. The problem of errors caused by the difference of the resolution of the input pictures when the RPA robot runs and designs is solved, and the applicability of the RPA robot is improved
The image matching method in the embodiment of the present application is described above, and the image matching apparatus in the embodiment of the present invention is described below. Referring to fig. 3, an embodiment of an image matching apparatus of the present application includes:
a first obtaining unit 301, configured to execute an RPA service flow and obtain an image to be matched;
a second obtaining unit 302, configured to obtain an image to be matched, and obtain a preset resolution of the template image;
a determining unit 303, configured to determine whether the resolution of the image to be matched meets the preset resolution, and if the resolution of the image to be matched does not meet the preset resolution, trigger a resolution processing unit 304;
a resolution processing unit 304, configured to perform resolution processing on the image to be matched to obtain a preprocessed image;
a matching unit 305, configured to perform matching processing on the preprocessed image and the template image.
In this embodiment, the flow executed by each unit in the image matching apparatus is similar to the method flow described in the embodiment corresponding to fig. 1, and is not described herein again.
Fig. 4 is a schematic structural diagram of a computer according to an embodiment of the present disclosure, where the server 400 may include one or more Central Processing Units (CPUs) 401 and a memory 405, and the memory 405 stores one or more application programs or data.
In this embodiment, the specific functional module division in the central processing unit 401 may be similar to the functional module division manner of each unit described in the foregoing fig. 4, and is not described here again.
The server 400 may also include one or more power supplies 402, one or more wired or wireless network interfaces 403, one or more input-output interfaces 404, and/or one or more operating systems, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
The central processing unit 401 may perform the operations performed by the image matching method in the embodiment shown in fig. 1, which are not described herein again.
The present invention also provides a computer-readable storage medium for implementing the functions of a land area measuring system, on which a computer program is stored which, when being executed by a processor, the processor may be adapted to carry out the image matching method as described in fig. 1.
It will be appreciated that the integrated units, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a corresponding one of the computer readable storage media or integrated as a computer program product for performing the above-described method. Based on such understanding, all or part of the flow of the method according to the above embodiments may be implemented by a computer program, which may be stored in a computer-readable storage medium and used by a processor to implement the steps of the above embodiments of the method. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. An image matching method, applied to robot process automation, comprising:
executing an RPA service flow to obtain an image to be matched;
acquiring a template image, and acquiring a preset resolution of the template image;
judging whether the resolution of the image to be matched meets the preset resolution or not;
if the resolution of the image to be matched does not accord with the preset resolution, performing resolution processing on the image to be matched to obtain a preprocessed image;
and matching the preprocessed image with the template image.
2. The image matching method according to claim 1, wherein if the resolution of the image to be matched does not meet the preset resolution, performing resolution processing on the image to be matched to obtain a preprocessed image, including:
and if the resolution of the image to be matched is smaller than the preset resolution, carrying out bicubic interpolation processing on the image to be matched to obtain a preprocessed image.
3. The image matching method according to claim 1, wherein if the resolution of the image to be matched does not meet the preset resolution, performing resolution processing on the image to be matched to obtain a preprocessed image, including:
and if the resolution of the image to be matched is greater than the preset resolution, carrying out bit operation averaging processing on the image to be matched to obtain a preprocessed image.
4. The image matching method according to claim 3, wherein before the bit operation averaging processing is performed on the image to be matched to obtain a preprocessed image, the method further comprises:
and performing Gaussian filtering on the image to be matched.
5. The image matching method according to any one of claims 1 to 4, wherein after the image to be matched is acquired, the method further comprises:
and carrying out gray processing on the image to be matched.
6. The image matching method according to any one of claims 1 to 4, wherein the matching the preprocessed image with the template image comprises:
and judging whether the preprocessed image comprises the template image or not, and returning a judgment result.
7. The image matching method according to claims 1 to 4,
and the image to be matched is obtained by screenshot of the current user interface.
8. An image matching device applied to robot process automation, comprising:
the first acquisition unit is used for executing the RPA service process and acquiring an image to be matched;
the second acquisition unit is used for acquiring an image to be matched and acquiring the preset resolution of the template image;
the judging unit is used for judging whether the resolution of the image to be matched meets the preset resolution or not, and if the resolution of the image to be matched does not meet the preset resolution, the resolution processing unit is triggered;
the resolution processing unit is used for carrying out resolution processing on the image to be matched to obtain a preprocessed image;
and the matching unit is used for matching the preprocessed image with the template image.
9. A computer, comprising:
a central processing unit, a memory;
the memory is a transient memory or a persistent memory;
the central processor is configured to communicate with the memory, the instructions in the memory being executable on the computer to perform the method of any of claims 1-7.
10. A computer-readable storage medium comprising instructions that, when executed on a computer, cause the computer to perform the method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011353325.XA CN112488177A (en) | 2020-11-26 | 2020-11-26 | Image matching method and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011353325.XA CN112488177A (en) | 2020-11-26 | 2020-11-26 | Image matching method and related equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112488177A true CN112488177A (en) | 2021-03-12 |
Family
ID=74935540
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011353325.XA Pending CN112488177A (en) | 2020-11-26 | 2020-11-26 | Image matching method and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112488177A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114500702A (en) * | 2022-01-30 | 2022-05-13 | 上海傲显科技有限公司 | Sub-pixel arrangement based segmentation method, mobile terminal and terminal-readable storage medium |
| CN116109839A (en) * | 2023-02-15 | 2023-05-12 | 北京拙河科技有限公司 | Picture difference comparison method and device |
| CN116304148A (en) * | 2022-09-07 | 2023-06-23 | 厦门创联享信息科技有限公司 | Data matching method for vehicle accessory transaction |
| CN117854256A (en) * | 2024-03-05 | 2024-04-09 | 成都理工大学 | Geological disaster monitoring method based on unmanned aerial vehicle video stream analysis |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101010694A (en) * | 2005-06-30 | 2007-08-01 | 奥林巴斯株式会社 | Searching system and searching method |
| US20120082385A1 (en) * | 2010-09-30 | 2012-04-05 | Sharp Laboratories Of America, Inc. | Edge based template matching |
| CN102521795A (en) * | 2011-12-15 | 2012-06-27 | 中国科学院自动化研究所 | Cross matching fingerprint image scaling method based on global ridge distance |
| CN106096659A (en) * | 2016-06-16 | 2016-11-09 | 网易（杭州）网络有限公司 | Image matching method and device |
| CN107610077A (en) * | 2017-09-11 | 2018-01-19 | 广东欧珀移动通信有限公司 | Image processing method and device, electronic installation and computer-readable recording medium |
| WO2019196560A1 (en) * | 2018-04-12 | 2019-10-17 | Oppo广东移动通信有限公司 | Image processing device testing method, device, equipment and storage medium |
| CN110390666A (en) * | 2019-06-14 | 2019-10-29 | 平安科技（深圳）有限公司 | Road damage detecting method, device, computer equipment and storage medium |
| CN110780965A (en) * | 2019-10-24 | 2020-02-11 | 深圳前海微众银行股份有限公司 | Vision-based process automation method, device and readable storage medium |
-
2020
- 2020-11-26 CN CN202011353325.XA patent/CN112488177A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101010694A (en) * | 2005-06-30 | 2007-08-01 | 奥林巴斯株式会社 | Searching system and searching method |
| US20120082385A1 (en) * | 2010-09-30 | 2012-04-05 | Sharp Laboratories Of America, Inc. | Edge based template matching |
| CN102521795A (en) * | 2011-12-15 | 2012-06-27 | 中国科学院自动化研究所 | Cross matching fingerprint image scaling method based on global ridge distance |
| CN106096659A (en) * | 2016-06-16 | 2016-11-09 | 网易（杭州）网络有限公司 | Image matching method and device |
| CN107610077A (en) * | 2017-09-11 | 2018-01-19 | 广东欧珀移动通信有限公司 | Image processing method and device, electronic installation and computer-readable recording medium |
| WO2019196560A1 (en) * | 2018-04-12 | 2019-10-17 | Oppo广东移动通信有限公司 | Image processing device testing method, device, equipment and storage medium |
| CN110390666A (en) * | 2019-06-14 | 2019-10-29 | 平安科技（深圳）有限公司 | Road damage detecting method, device, computer equipment and storage medium |
| CN110780965A (en) * | 2019-10-24 | 2020-02-11 | 深圳前海微众银行股份有限公司 | Vision-based process automation method, device and readable storage medium |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114500702A (en) * | 2022-01-30 | 2022-05-13 | 上海傲显科技有限公司 | Sub-pixel arrangement based segmentation method, mobile terminal and terminal-readable storage medium |
| CN116304148A (en) * | 2022-09-07 | 2023-06-23 | 厦门创联享信息科技有限公司 | Data matching method for vehicle accessory transaction |
| CN116109839A (en) * | 2023-02-15 | 2023-05-12 | 北京拙河科技有限公司 | Picture difference comparison method and device |
| CN117854256A (en) * | 2024-03-05 | 2024-04-09 | 成都理工大学 | Geological disaster monitoring method based on unmanned aerial vehicle video stream analysis |
| CN117854256B (en) * | 2024-03-05 | 2024-06-11 | 成都理工大学 | Geological disaster monitoring method based on unmanned aerial vehicle video stream analysis |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112488177A (en) | Image matching method and related equipment | |
| US20140050387A1 (en) | System and Method for Machine Vision Inspection | |
| CN110826372B (en) | Face feature point detection method and device | |
| CN110807110B (en) | Image searching method and device combining local and global features and electronic equipment | |
| EP3961495A1 (en) | System and method for finding an area of an eye from a facial image | |
| CN106910207B (en) | Method and device for identifying local area of image and terminal equipment | |
| US11798227B2 (en) | Image processing apparatus and image processing method | |
| CN115330657B (en) | Ocean exploration image processing method and device and server | |
| CN109523564B (en) | Method and apparatus for processing image | |
| US11698849B2 (en) | Automated application testing of mutable interfaces | |
| CN107527011B (en) | Non-contact skin resistance change trend detection method, device and equipment | |
| CN112686851B (en) | Image detection method, device and storage medium | |
| CN114004809A (en) | Skin image processing method, device, electronic equipment and medium | |
| CN114998172A (en) | Image processing method and related system | |
| CN111223054B (en) | Ultrasonic image evaluation method and device | |
| CN110245668B (en) | Terminal information acquisition method, acquisition device and storage medium based on image recognition | |
| CN111507944A (en) | Skin smoothness determination method and device and electronic equipment | |
| EP4231253A1 (en) | A method and system for dynamic cropping of full body pose images | |
| JP7147828B2 (en) | Image processing system, image processing method and program | |
| Premaratne et al. | Design and implementation of edge detection algorithm in dsPIC embedded processor | |
| US11941871B2 (en) | Control method of image signal processor and control device for performing the same | |
| CN114677443B (en) | Optical positioning method, device, equipment and storage medium | |
| CN115906043A (en) | Sliding block verification method, device, equipment and storage medium | |
| CN115862053A (en) | Safety shoe wearing detection method, device, equipment and storage medium | |
| CN118096674A (en) | Detection method, detection device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |