CN110596121A - Keyboard appearance detection method and device and electronic system - Google Patents

Keyboard appearance detection method and device and electronic system Download PDF

Info

Publication number
CN110596121A
CN110596121A CN201910869503.5A CN201910869503A CN110596121A CN 110596121 A CN110596121 A CN 110596121A CN 201910869503 A CN201910869503 A CN 201910869503A CN 110596121 A CN110596121 A CN 110596121A
Authority
CN
China
Prior art keywords
keyboard
picture
determining
detected
characters
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910869503.5A
Other languages
Chinese (zh)
Inventor
张宏毅
宋仁杰
魏秀参
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xuzhou Kuang Shi Data Technology Co Ltd
Nanjing Kuanyun Technology Co Ltd
Beijing Megvii Technology Co Ltd
Original Assignee
Xuzhou Kuang Shi Data Technology Co Ltd
Nanjing Kuanyun Technology Co Ltd
Beijing Megvii Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xuzhou Kuang Shi Data Technology Co Ltd, Nanjing Kuanyun Technology Co Ltd, Beijing Megvii Technology Co Ltd filed Critical Xuzhou Kuang Shi Data Technology Co Ltd
Priority to CN201910869503.5A priority Critical patent/CN110596121A/en
Publication of CN110596121A publication Critical patent/CN110596121A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/02—Input arrangements using manually operated switches, e.g. using keyboards or dials
    • G06F3/0202—Constructional details or processes of manufacture of the input device
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Biochemistry (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Input From Keyboards Or The Like (AREA)

Abstract

The invention provides a keyboard appearance detection method, a device and an electronic system, wherein the method comprises the following steps: acquiring a keyboard picture to be tested and a keyboard drawing corresponding to the keyboard picture to be tested; determining a standard position from a keyboard drawing according to the position information of keyboard characters in the keyboard drawing, and determining a specified position from a picture of a keyboard to be tested; determining a transformation matrix corresponding to the keyboard picture to be tested according to the standard position and the specified position; positioning keyboard characters in a keyboard picture to be detected by converting the matrix and the position information of the keyboard characters; and comparing the keyboard characters positioned in the keyboard picture to be detected with the keyboard characters on the corresponding positions in the keyboard drawing so as to detect the keyboard corresponding to the keyboard picture to be detected. The mode compares the keyboard picture to be detected with the keyboard drawing, saves the process of manufacturing the template library, improves the operating efficiency of the system, reduces the time of manual operation, and reduces the waste of manpower and material resources.

Description

Keyboard appearance detection method and device and electronic system
Technical Field
The invention relates to the technical field of image processing, in particular to a keyboard appearance detection method, a device and an electronic system.
Background
In the production and assembly processes of the keyboard, problems such as misprinted characters, keycap assembly errors and the like easily occur, so that the appearance of the keyboard needs to be subjected to defect detection to ensure the production quality of the keyboard. In the method for detecting appearance defects of a keyboard in the related art, a template library is usually manufactured by using a keyboard picture, and the template picture is compared with the keyboard picture to detect whether the appearance of the keyboard is defective. However, the process of making the template library is very complex, and a worker needs to manually cut out the keyboard area in the keyboard picture to be used as the template picture; moreover, the variety of the keyboard is many, and the amount of the required template is large, so that a large amount of manpower and material resources are wasted by the mode of detecting the appearance of the keyboard through the template.
Disclosure of Invention
The invention aims to provide a keyboard appearance detection method, a device and an electronic system, which are used for shortening the time of manual operation so as to reduce the waste of manpower and material resources.
In a first aspect, an embodiment of the present invention provides a method for detecting an appearance of a keyboard, where the method includes: acquiring a keyboard picture to be tested and a keyboard drawing corresponding to the keyboard picture to be tested; the keyboard drawing carries position information of keyboard characters; according to the position information of the keyboard characters, determining a standard position from a keyboard drawing, and determining a specified position from a keyboard picture to be tested; determining a transformation matrix corresponding to the keyboard picture to be tested according to the standard position and the designated position; positioning keyboard characters in the keyboard picture to be tested according to the transformation matrix and the position information of the keyboard characters; and comparing the keyboard characters positioned in the keyboard picture to be detected with the keyboard characters on the corresponding positions in the keyboard drawing so as to detect the keyboard corresponding to the keyboard picture to be detected.
In a preferred embodiment of the present invention, before the step of obtaining the keyboard drawing corresponding to the keyboard picture to be tested, the method further includes: carrying out binarization processing on a keyboard drawing corresponding to a keyboard picture to be detected; in the processed keyboard drawing, pixel values of the edges of the keyboard characters and the keycap areas are lower than a preset threshold value; detecting the keycap area in the processed keyboard drawing according to the pixel value of the edge of the keycap area; and detecting the keyboard characters in the keycap areas to obtain the position information of the keyboard characters.
In a preferred embodiment of the present invention, the step of determining the standard position from the keyboard drawing and determining the designated position from the to-be-tested keyboard picture according to the position information of the keyboard character includes: determining the position of a preset designated character from a keyboard drawing, and taking the position of the preset designated character as a standard position; and detecting the position of the designated character from the keyboard picture to be detected by adopting a template matching mode, and taking the detected position as the designated position.
In a preferred embodiment of the present invention, before the step of determining the position of the preset designated character from the keyboard drawing, the method further includes: determining the scale and the inclination angle of the keyboard in the picture of the keyboard to be detected in a template matching mode; and carrying out scaling and/or rotation processing on the keyboard drawing based on the dimension and the inclination angle.
In a preferred embodiment of the present invention, the step of determining the transformation matrix corresponding to the keyboard picture to be tested according to the standard position and the designated position includes: determining an initial transformation matrix according to the specified position and the standard position; performing first transformation processing on the standard position through the initial transformation matrix to obtain a transformation position; judging whether the conversion position is matched with the designated position or not to obtain a judgment result; and adjusting the initial transformation matrix according to the judgment result to obtain a final transformation matrix, and determining the final transformation matrix as the transformation matrix corresponding to the keyboard picture to be detected.
In a preferred embodiment of the present invention, the plurality of conversion positions are provided; the step of determining whether the transformed position matches the designated position to obtain a determination result includes: for each conversion position, judging whether the distance between the current conversion position and the designated position corresponding to the current conversion position is within a preset distance range; if not, determining that the current transformation position is an error position; if so, the current transformed position is determined to be the correct position.
In a preferred embodiment of the present invention, the step of adjusting the initial transformation matrix according to the determination result includes: if the judgment result indicates that an error position exists in the conversion position, determining a first initial matrix according to the correct position in the conversion position and the standard position corresponding to the correct position; performing second transformation processing on the standard position corresponding to the error position through the first initial matrix to obtain an initial position corresponding to the error position; determining a second initial matrix according to the initial position corresponding to the error position, the correct position in the conversion position and the standard position; the second initial matrix is determined as the final transformation matrix.
In a preferred embodiment of the present invention, the step of determining the second initial matrix according to the initial position corresponding to the error position, the correct position in the conversion position, and the standard position includes: detecting a designated target of a designated position corresponding to the error position within a preset range by taking an initial position corresponding to the error position as a center, and taking the position of the detected designated target as a final position corresponding to the error position; and determining a second initial matrix according to the final position corresponding to the error position, the correct position in the conversion position and the standard position.
In a preferred embodiment of the present invention, the step of positioning the keyboard character in the to-be-detected keyboard picture by transforming the matrix and the position information of the keyboard character includes: transforming the keyboard picture to be tested through the transformation matrix; and positioning the keyboard characters in the converted keyboard picture to be detected according to the position information of the keyboard characters.
In a preferred embodiment of the present invention, the step of comparing the keyboard character located in the keyboard picture to be tested with the keyboard character at the corresponding position in the keyboard drawing to detect the keyboard corresponding to the keyboard picture to be tested includes: calculating the similarity between the current keyboard character in the keyboard picture to be tested and the keyboard character at the corresponding position in the keyboard drawing aiming at each keyboard character in the keyboard picture to be tested; and determining the detection result of the keyboard corresponding to the keyboard picture to be detected according to the similarity corresponding to each keyboard character.
In a preferred embodiment of the present invention, the step of determining the detection result of the keyboard corresponding to the keyboard picture to be detected according to the similarity corresponding to each keyboard character includes: judging whether keyboard characters with the similarity smaller than a preset similarity threshold exist in each keyboard character; if yes, determining that the detection result of the keyboard corresponding to the keyboard picture to be detected is defective; and if the detected keyboard picture does not exist, determining that the detection result of the keyboard corresponding to the keyboard picture to be detected does not exist defects.
In a preferred embodiment of the present invention, after the step of detecting the keyboard corresponding to the keyboard picture to be detected, the method further includes: and if the detection result of the keyboard corresponding to the keyboard picture to be detected indicates that the keyboard has defects, outputting an alarm signal.
In a second aspect, an embodiment of the present invention provides a keyboard appearance detecting apparatus, where the apparatus includes: the picture acquisition module is used for acquiring a picture of the keyboard to be detected and a keyboard drawing corresponding to the picture of the keyboard to be detected; the keyboard drawing carries position information of keyboard characters; the first character position determining module is used for determining a standard position from a keyboard drawing and determining a specified position from a keyboard picture to be detected according to the position information of the keyboard characters; the transformation matrix determining module is used for determining a transformation matrix corresponding to the keyboard picture to be tested according to the standard position and the specified position; the second character position determining module is used for positioning the keyboard characters in the keyboard picture to be detected through the transformation matrix and the position information of the keyboard characters; and the detection module is used for comparing the keyboard characters positioned in the keyboard picture to be detected with the keyboard characters on the corresponding positions in the keyboard drawing so as to detect the keyboard corresponding to the keyboard picture to be detected.
In a third aspect, an embodiment of the present invention provides an electronic system, including: the device comprises an image acquisition device, a processing device and a storage device; the image acquisition equipment is used for acquiring preview video frames or image data; the storage device has stored thereon a computer program which, when run by a processing apparatus, performs the steps of the keyboard appearance detection method described above.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processing device, the computer program performs the steps of the keyboard appearance detection method.
The embodiment of the invention has the following beneficial effects:
the invention provides a method, a device and an electronic system for detecting the appearance of a keyboard, which comprises the steps of firstly obtaining a picture of the keyboard to be detected and a keyboard drawing corresponding to the picture of the keyboard to be detected; then according to the position information of the keyboard characters in the keyboard drawing, determining a standard position from the keyboard drawing and determining a specified position from the picture of the keyboard to be tested; determining a transformation matrix corresponding to the keyboard picture to be tested according to the standard position and the specified position; further, keyboard characters are positioned in the keyboard picture to be detected through the transformation matrix and the position information of the keyboard characters; and finally, comparing the keyboard characters positioned in the keyboard picture to be detected with the keyboard characters on the corresponding positions in the keyboard drawing so as to detect the keyboard corresponding to the keyboard picture to be detected. The mode compares the keyboard picture to be detected with the keyboard drawing, saves the process of manufacturing the template library, improves the operating efficiency of the system, reduces the time of manual operation, and reduces the waste of manpower and material resources.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a schematic structural diagram of an electronic system according to an embodiment of the present invention;
fig. 2 is a flowchart of a keyboard appearance detection method according to an embodiment of the present invention;
FIG. 3 is a flowchart of another keyboard appearance detection method according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a keyboard drawing after binarization processing according to an embodiment of the present invention;
FIG. 5 is a flowchart of another keyboard appearance detection method according to an embodiment of the present invention;
FIG. 6 is a flowchart of another keyboard appearance detection method according to an embodiment of the present invention;
FIG. 7 is a flowchart of another keyboard appearance detection method according to an embodiment of the present invention;
FIG. 8 is a flowchart of another keyboard appearance detection method according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of a keyboard appearance detecting device according to an embodiment of the present invention.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In view of the problems in the related art that the accuracy of a transformation matrix is poor and the accuracy of a subsequent image obtained based on the transformation matrix is poor, embodiments of the present invention provide a method, an apparatus, and an electronic system for detecting a keyboard appearance, where the technology can be applied to image processing scenes such as image transformation, image matching, image target detection, and the like, and the technology can be implemented by using corresponding software and hardware, and the following describes embodiments of the present invention in detail.
The first embodiment is as follows:
first, an example electronic system 100 for implementing the keyboard appearance detection method, apparatus, and electronic system of embodiments of the present invention is described with reference to fig. 1.
As shown in FIG. 1, an electronic system 100 includes one or more processing devices 102, one or more memory devices 104, an input device 106, an output device 108, and one or more image capture devices 110, which are interconnected via a bus system 112 and/or other type of connection mechanism (not shown). It should be noted that the components and structure of the electronic system 100 shown in fig. 1 are exemplary only, and not limiting, and that the electronic system may have other components and structures as desired.
The processing device 102 may be a gateway or an intelligent terminal, or a device including a Central Processing Unit (CPU) or other form of processing unit having data processing capability and/or instruction execution capability, and may process data of other components in the electronic system 100 and may control other components in the electronic system 100 to perform desired functions.
The storage 104 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. On which one or more computer program instructions may be stored that may be executed by processing device 102 to implement client functionality (implemented by the processing device) and/or other desired functionality in embodiments of the present invention described below. Various applications and various data, such as various data used and/or generated by the applications, may also be stored in the computer-readable storage medium.
The input device 106 may be a device used by a user to input instructions and may include one or more of a keyboard, a mouse, a microphone, a touch screen, and the like.
The output device 108 may output various information (e.g., images or sounds) to the outside (e.g., a user), and may include one or more of a display, a speaker, and the like.
The image capture device 110 may capture preview video frames or image data and store the captured preview video frames or image data in the storage 104 for use by other components.
For example, the devices in the exemplary electronic system for implementing the keyboard appearance detection method, apparatus and electronic system according to the embodiments of the present invention may be integrally disposed, or may be disposed in a decentralized manner, such as integrally disposing the processing device 102, the storage device 104, the input device 106 and the output device 108, and disposing the image capturing device 110 at a designated position where a target image can be captured. When the above-described devices in the electronic system are integrally provided, the electronic system may be implemented as an intelligent terminal such as a camera, a smart phone, a tablet computer, a vehicle-mounted terminal, and the like.
Example two:
the embodiment provides a keyboard appearance detection method, which is executed by a processing device in the electronic system; the processing device may be any device or chip having data processing capabilities. The processing equipment can independently process the received information, can also be connected with the server, jointly analyzes and processes the information, and uploads a processing result to the cloud. As shown in fig. 2, the keyboard appearance detecting method includes the following steps:
step S202, acquiring a keyboard picture to be tested and a keyboard drawing corresponding to the keyboard picture to be tested; the keyboard drawing carries position information of keyboard characters.
The keyboard picture to be tested is usually a picture of a keyboard shot by a camera, the keyboard drawing corresponding to the keyboard picture to be tested is usually the drawing of the keyboard corresponding to the produced keyboard picture to be tested, and under the condition that the appearance of the produced keyboard is not defective, the positions of characters in the keyboard picture to be tested correspond to the positions of keyboard characters in the keyboard drawing one by one.
During specific implementation, the first character at the upper left corner in the keyboard drawing can be set as an origin, a rectangular coordinate system is established by taking the origin as the center, and the coordinate of each character in the keyboard drawing can be represented by the rectangular coordinate, namely the position information of the character.
And step S204, determining a standard position from a keyboard drawing according to the position information of the keyboard characters, and determining a specified position from a to-be-detected keyboard picture.
And step S206, determining a transformation matrix corresponding to the keyboard picture to be tested according to the standard position and the designated position.
The standard position determined in the keyboard drawing is usually a position of a plurality of characters randomly selected from the keyboard drawing, for example, 2 keyboard characters may be respectively selected from a first row of keyboard rows, a last row of keyboard rows, a first column of keyboard columns and a last column of keyboard columns of the keyboard in the keyboard drawing, and the specified position determined in the keyboard picture to be tested usually corresponds to the position of the character selected from the keyboard drawing.
The number of standard positions is usually consistent with the number of designated positions, and a plurality of designated positions can be mapped to corresponding standard positions to obtain a transformation matrix. In general, the transformation matrix is used to characterize a mapping relationship between the designated position and the standard position, which may be specifically one or more combinations of rotation, scaling, warping, and translation.
During specific implementation, the keyboard picture to be detected can be placed in the rectangular coordinate system to obtain the coordinate of the designated position, and the coordinate of the designated position is mapped to the coordinate of the standard position, so that the transformation matrix can be obtained through calculation. For example, the coordinates of the designated position are (a, B), the coordinates of the standard position are (a, B), and the transformation matrix W is obtained by calculating the formula (a, B) × W ═ a, B.
And S208, positioning the keyboard characters in the keyboard picture to be detected according to the transformation matrix and the position information of the keyboard characters.
In specific implementation, the position information in the keyboard drawing is transformed through the transformation matrix, so that the position of each keyboard character in the keyboard picture to be tested can be obtained, and the transformation can be understood to realize the alignment of the keyboard picture to be tested and the keyboard drawing.
Step S210, comparing the keyboard characters located in the keyboard picture to be tested with the keyboard characters on the corresponding position in the keyboard drawing, so as to detect the keyboard corresponding to the keyboard picture to be tested.
Determining a picture area where the keyboard is located according to the position of the keyboard character in the keyboard drawing, comparing the keyboard character in the keyboard picture to be detected with the keyboard character at the corresponding position in the keyboard drawing in a characteristic extraction (for example, gray level matching or neural network matching based) mode in the picture area, and judging whether the keyboard character in the picture area is consistent according to a comparison result, thereby realizing the detection of the keyboard corresponding to the keyboard picture to be detected. And judging whether the positions of the keyboard characters of the keyboard picture to be detected have errors according to the detection result so as to check whether the keyboard corresponding to the keyboard picture to be detected has character printing errors, namely whether the appearance of the keyboard has defects.
Firstly, acquiring a picture of a keyboard to be detected and a keyboard drawing corresponding to the picture of the keyboard to be detected; then according to the position information of the keyboard characters in the keyboard drawing, determining a standard position from the keyboard drawing and determining a specified position from the picture of the keyboard to be tested; determining a transformation matrix corresponding to the keyboard picture to be tested according to the standard position and the specified position; further, keyboard characters are positioned in the keyboard picture to be detected through the transformation matrix and the position information of the keyboard characters; and finally, comparing the keyboard characters positioned in the keyboard picture to be detected with the keyboard characters on the corresponding positions in the keyboard drawing so as to detect the keyboard corresponding to the keyboard picture to be detected. The mode compares the keyboard picture to be detected with the keyboard drawing, saves the process of manufacturing the template library, improves the operating efficiency of the system, reduces the time of manual operation, and reduces the waste of manpower and material resources.
Example three:
the present embodiment provides another keyboard appearance detection method, which is implemented on the basis of the embodiment shown in fig. 2; the embodiment mainly describes a specific process of determining the position information of keyboard characters carried in a keyboard drawing and a specific process of detecting a keyboard corresponding to a keyboard picture to be detected; as shown in fig. 3; the method specifically comprises the following steps:
step S302, performing binarization processing on the keyboard drawing corresponding to the keyboard picture to be tested; in the processed keyboard drawing, the pixel values of the edges of the keyboard characters and the keycap areas are lower than a preset threshold value.
The binarization processing may also be referred to as image binarization, and is generally a process of setting the gray value of a pixel point on an image to 0 or 255, that is, making the entire image exhibit an obvious black-and-white effect. The keyboard drawing is subjected to binarization processing, that is, in the keyboard drawing, the edges of the keyboard characters and the keycap regions are set to be black, the gray value (corresponding to the pixel value) is 0, and the rest regions are set to be white, and the gray value is 255, as shown in fig. 4. The preset threshold is typically set to 255.
And step S304, detecting the key cap area in the processed keyboard drawing according to the pixel value of the edge of the key cap area.
Through a contour detection algorithm, the peripheral contour of the keycap (equivalent to the edge of the keycap) can be detected, so that the position of the keycap area in the keyboard drawing is positioned. The contour detection algorithm may employ an opencvfindcount function that locates the keycap area by pixel values of the edge of the keycap area in the keyboard drawing.
And step S306, detecting keyboard characters in the keycap area to obtain the position information of the keyboard characters.
Because the keyboard characters are black after binarization processing, black pixel points are searched in the obtained keycap area, and the positions corresponding to the black pixel points are determined as the positions of the keyboard characters.
Step S308, according to the position information of the keyboard characters, a standard position is determined from a keyboard drawing, and a specified position is determined from a keyboard picture to be tested.
And step S310, determining a transformation matrix corresponding to the keyboard picture to be tested according to the standard position and the specified position.
Step S312, the keyboard characters are positioned in the keyboard picture to be detected through the transformation matrix and the position information of the keyboard characters.
Step S314, calculating, for each keyboard character in the keyboard picture to be tested, a similarity between a current keyboard character in the keyboard picture to be tested and a keyboard character at a corresponding position in the keyboard drawing.
In a specific implementation, the similarity between the current keyboard character in the keyboard picture to be tested and the keyboard character at the corresponding position in the keyboard drawing can be obtained according to a method for calculating the similarity between two images provided by OpenCV, for example, standardized correlation matching CV _ TM _ CCORR _ normal. The OpenCV is generally an open source function library used for image processing, analysis, and machine vision.
In specific implementation, the similarity can be calculated based on a neural network matching mode, firstly, the neural network is adopted to respectively extract the keyboard characters in the keyboard drawing and the characteristics of the keyboard characters in the keyboard picture to be detected, the characteristics of the keyboard characters in the keyboard drawing and the characteristics of the keyboard characters in the keyboard picture to be detected are spliced and input to the full connection layer, and the similarity is output. Usually, a large number of keyboard pictures without defects and keyboard pictures with defects can be used as samples to train the neural network, and when the neural network is tested, keyboard characters in a keyboard drawing and keyboard characters in a keyboard picture to be tested are input, namely, the similarity of the keyboard characters and the keyboard pictures can be output.
And step S316, determining a detection result of the keyboard corresponding to the keyboard picture to be detected according to the similarity corresponding to each keyboard character.
The similarity corresponding to the keyboard characters determines the matching degree of the picture of the keyboard to be tested and the keyboard drawing, and under the normal condition, if the similarity between the picture of the keyboard to be tested and the corresponding keyboard drawing is high or completely similar, the appearance of the keyboard corresponding to the picture of the keyboard to be tested has no defects. In the specific implementation, the detection result of the keyboard corresponding to the keyboard picture to be detected can be determined through the following steps 10-12:
step 10, judging whether keyboard characters with the similarity smaller than a preset similarity threshold exist in each keyboard character; if so, executing step 11; if not, step 12 is performed.
And 11, determining that the detection result of the keyboard corresponding to the to-be-detected keyboard picture is defective.
And step 12, determining that the detection result of the keyboard corresponding to the to-be-detected keyboard picture is defect-free.
Firstly, carrying out binarization processing on a keyboard drawing corresponding to a keyboard picture to be detected; detecting a keycap area in a keyboard drawing according to the pixel value of the edge of the keycap area after binarization processing, and detecting a keyboard character in the keycap area to obtain position information of the keyboard character; then determining a transformation matrix corresponding to the keyboard picture to be tested according to the position information of the keyboard characters so as to position the keyboard characters in the keyboard picture to be tested, and calculating the similarity between the current keyboard characters in the keyboard picture to be tested and the keyboard characters at the corresponding positions in the keyboard drawing aiming at each keyboard character in the keyboard picture to be tested; and finally, determining the detection result of the keyboard corresponding to the keyboard picture to be detected according to the similarity corresponding to each keyboard character. According to the mode, the time for manually manufacturing the template can be saved, and the accuracy of key cap area and keyboard character positioning is improved, so that the accuracy of keyboard appearance defect detection is improved.
Example four:
the present embodiment provides another keyboard appearance detection method, which is implemented on the basis of the embodiment shown in fig. 2; the embodiment mainly describes a specific process of positioning keyboard characters in a to-be-detected keyboard picture by converting the position information of the matrix and the keyboard characters; as shown in fig. 5; the method comprises the following steps:
step S502, acquiring a keyboard picture to be tested and a keyboard drawing corresponding to the keyboard picture to be tested; the keyboard drawing carries position information of keyboard characters.
Step S504, determining a position of a preset designated character from the keyboard drawing, and using the position of the preset designated character as a standard position.
The preset designated characters are usually several keyboard characters randomly selected from the keyboard drawing, for example, 2 keyboard characters can be taken from the middle and four edges of the keyboard drawing.
Because the size of the keyboard picture to be tested may not be consistent with the size of the keyboard drawing, and the keyboard picture to be tested obtained through shooting may have a certain angle of inclination, before the step of determining the position of the preset designated character from the keyboard drawing, the scale and the inclination angle of the keyboard in the keyboard picture to be tested are generally determined in a template matching manner; and carrying out scaling and/or rotation processing on the keyboard drawing based on the dimension and the inclination angle. The template matching mode generally refers to a mode of searching for a specific target in an image, and the mode generally needs to adjust the scale and the inclination of a keyboard drawing to be matched with the scale and the inclination of a keyboard in a keyboard picture to be detected, and the scale and the inclination of the keyboard drawing with the highest matching degree are used as the scale and the inclination of the keyboard in the keyboard picture to be detected. For example, the selectable range of the inclination angle can be preset to be-30 degrees to 30 degrees, and the selectable range of the scaling scale is 0.5 to 2 times; selecting an inclination angle from the selectable range of the inclination angles and selecting a zoom scale from the selectable range of the zoom scales when template matching is performed each time; then, carrying out inclination processing on the keyboard drawing according to the selected inclination angle, carrying out scaling processing according to the selected scaling scale, and carrying out template matching on the processed keyboard drawing and the picture of the keyboard to be detected to obtain a matching score; then, re-selecting the inclination angle and the scaling scale, and after repeating for multiple times, each group of inclination angle and scaling scale corresponds to a matching score; and then determining the scaling scale and the angle with the highest matching score as the scale and the inclination angle of the keyboard in the picture of the keyboard to be detected.
In specific implementation, the keyboard drawing needs to be amplified, reduced or rotated based on the scale and the inclination angle of the keyboard in the obtained keyboard picture to be detected, or the keyboard picture is subjected to both scaling and rotation so that the size of the keyboard picture to be detected is consistent with that of the keyboard drawing, and therefore the accuracy of the character position of the keyboard can be improved.
Step S506, detecting the position of the designated character from the keyboard picture to be detected by template matching, and using the detected position as the designated position.
The template matching method generally refers to a method of finding a specific target in an image, and the method generally needs to traverse each possible position in a keyboard picture to be tested, compare whether each position is similar to a designated character, and use the position with the highest similarity as the position of the designated character.
Step S508, determining a transformation matrix corresponding to the keyboard picture to be tested based on the standard position and the designated position.
And step S510, transforming the keyboard picture to be tested through the transformation matrix.
The keyboard characters in the keyboard drawing are converted into the matrix, so that the positions of the keyboard characters corresponding to the keyboard picture to be tested can be obtained, and the keyboard picture to be tested is also converted.
And S512, positioning keyboard characters in the converted keyboard picture to be tested according to the position information of the keyboard characters, so that the position of each keyboard character in the keyboard picture to be tested can be obtained.
Step S514, comparing the keyboard characters located in the keyboard picture to be tested with the keyboard characters on the corresponding position in the keyboard drawing, so as to detect the keyboard corresponding to the keyboard picture to be tested.
Step S516, if the detection result of the keyboard corresponding to the to-be-detected keyboard picture indicates that the keyboard has a defect, outputting an alarm signal.
The alarm information can be alarm display output in the detection equipment or buzzing sound sent by the detection equipment so as to remind workers to process the defective keyboard.
The keyboard appearance detection method comprises the steps of firstly determining the position of a preset designated character from an obtained keyboard drawing; taking the position of a preset designated character as a standard position, detecting the position of the designated character from a keyboard picture to be detected in a template matching mode, and taking the detected position as a designated position; then determining a transformation matrix corresponding to the keyboard picture to be tested based on the standard position and the specified position, and carrying out transformation processing on the keyboard picture to be tested through the transformation matrix so as to position keyboard characters; and comparing the keyboard characters positioned in the keyboard picture to be detected with the keyboard characters on the corresponding positions in the keyboard drawing so as to detect the keyboard corresponding to the keyboard picture to be detected, and if the detection result of the keyboard corresponding to the keyboard picture to be detected indicates that the keyboard has defects, outputting an alarm signal. The method accurately positions the position of the character by transforming the matrix, and can remind workers to process the keyboard with defects.
Example five:
the present embodiment provides another keyboard appearance detection method, which is implemented on the basis of the embodiment shown in fig. 1; the embodiment mainly describes a specific process of determining a transformation matrix based on a specified position and a standard position; as shown in fig. 6; the method comprises the following steps:
step S602, acquiring a keyboard picture to be tested and a keyboard drawing corresponding to the keyboard picture to be tested; the keyboard drawing carries position information of keyboard characters.
Step S604, determining a standard position from the keyboard drawing according to the position information of the keyboard character, and determining a designated position from the to-be-detected keyboard picture.
Step S606, determining an initial transformation matrix according to the designated position and the standard position.
In a specific implementation, the coordinates of the designated position may be mapped to the coordinates of the standard position, i.e., the initial transformation matrix may be obtained through calculation.
Step S608, a first transformation process is performed on the standard position through the initial transformation matrix to obtain a transformed position.
In specific implementation, the coordinates of the standard position are mapped through the initial transformation matrix, so that the transformation position can be obtained, that is, the coordinates of the standard position are multiplied by the initial transformation matrix, so that the transformation position can be obtained. Because the initial transformation matrix is obtained by calculating the specified position and the standard position, when the initial transformation matrix is accurate enough and the initial transformation matrix is multiplied by the standard position, the specified position should be obtained theoretically, namely, the transformation position is coincident with the specified position; however, considering that the accuracy of the initial transformation matrix may be low, the transformation position obtained by multiplying the initial transformation matrix by the standard position usually has a deviation from the designated position, or a deviation exists between a part of the transformation positions and the designated position, and based on this, it is necessary to continue to adjust the initial transformation matrix to improve the accuracy of the transformation matrix.
Step S610, determining whether the transformed position matches the designated position, and obtaining a determination result.
In practical implementation, if the transformation position completely coincides with the designated position, or the deviation is within a preset deviation threshold, it can be determined that the transformation position matches the designated position, which means that the initial transformation matrix can accurately transform the standard position to the designated position, and the initial transformation matrix is accurate enough and does not need to be adjusted again. However, if the deviation of the transformed position from the standard position is large and the deviation threshold is exceeded, it can be determined that the transformed position does not match the specified position, and the initial transformation matrix needs to be adjusted.
And step S612, adjusting the initial transformation matrix according to the judgment result to obtain a final transformation matrix.
For example, if the determination result indicates that the transformation position does not match the designated position, the initial transformation matrix may be adjusted for the position with the larger deviation, for example, the coefficients and structure of the initial transformation matrix may be adjusted, or the transformation matrix may be recalculated, until all the transformation positions have deviations from the designated position within the deviation threshold. At this time, the obtained final transformation matrix can accurately represent the mapping relation between the designated position and the standard position.
The keyboard appearance detection method determines an initial transformation matrix according to the designated position and the standard position; then, carrying out first transformation processing on the standard position through the initial transformation matrix to obtain a transformation position; and adjusting the initial transformation matrix according to the matching result of the transformation position and the standard position to obtain a final transformation matrix. In the method, after the initial transformation matrix is obtained, the standard position is transformed through the initial transformation matrix, and the transformation matrix is adjusted again according to the matching degree of the transformation position corresponding to the standard position and the specified position, so that the obtained transformation matrix can accurately represent the position relation between each specified position and the corresponding standard position, the accuracy of the transformation matrix is improved, and the accuracy of the image obtained based on the subsequent transformation matrix is also improved.
Example six:
the embodiment provides another keyboard appearance detection method, which is implemented on the basis of the embodiment; this embodiment focuses on describing a specific process of adjusting the transformation matrix according to the matching result between the transformation position and the specified standard position when the transformation position is multiple, as shown in fig. 7, the method specifically includes the following steps:
step S702, acquiring a picture of a keyboard to be tested and a keyboard drawing corresponding to the picture of the keyboard to be tested; the keyboard drawing carries position information of keyboard characters.
Step S704, according to the position information of the keyboard characters, a standard position is determined from a keyboard drawing, and a designated position is determined from a to-be-detected keyboard picture.
Step S706, determining an initial transformation matrix according to the specified position and the standard position.
In step S708, a first transformation process is performed on the standard positions by using the initial transformation matrix to obtain a plurality of transformation positions.
Usually, the standard position is also plural; the initial transformation matrix and each standard position are subjected to first transformation processing, so that a plurality of corresponding transformation positions can be obtained, namely one standard position corresponds to one transformation position, and the standard positions in the keyboard drawing and the designated positions in the keyboard picture to be tested are also in one-to-one correspondence relationship.
Step S710, aiming at each transformation position, judging whether the distance between the current transformation position and the specified position corresponding to the current transformation position is within a preset distance range; if so, go to step S712; if not, step S714 is performed.
Step S712, determining that the current transformed position is the correct position, and step S716 is performed.
In step S714, it is determined that the current transformation position is an error position.
And when the distance between the current transformation position and the corresponding specified position is not within the preset distance range, namely the difference value between the coordinate of the current transformation position and the coordinate of the corresponding standard position is greater than a preset threshold value, determining that the current transformation position is wrong. And when the difference value between the coordinates of the current transformation position and the coordinates of the corresponding designated position is not more than a preset threshold value, determining that the current transformation position is correct, if all the current transformation positions are correct, determining the corresponding transformation matrix as a final transformation matrix, and if at least one current transformation position is wrong, adjusting the transformation matrix.
Step S716, determining a first initial matrix according to the correct position in the transform position and the standard position corresponding to the correct position.
Considering that there is an error position in the transformed position, at this time, the error position can be eliminated, and a transformed matrix, i.e. the first initial matrix, is calculated again according to the correct position in the transformed position and the standard position corresponding to the correct position. In comparison, the first initial matrix is calculated by using a part of the designated positions and corresponding standard positions; in step S706, all the designated positions and their corresponding standard positions are used for calculation. In specific implementation, the coordinates of the correct position in the transformation positions may be mapped to the coordinates of the corresponding standard position, and a first initial matrix may be obtained through calculation.
Step S718, performing a second transformation process on the standard position corresponding to the error position through the first initial matrix to obtain an initial position corresponding to the error position.
In specific implementation, the coordinates of the standard position corresponding to the error position are mapped through the first initial matrix, so that the initial transformation position corresponding to the error position can be obtained. The initial position is typically located near the designated location corresponding to the standard location corresponding to the error location.
Step S720, determining a second initial matrix according to the initial position corresponding to the error position, the correct position in the conversion position, and the standard position.
It should be noted that the standard position is the same as the standard position in step S706, the correct position in the conversion position is the corresponding designated position, and the initial position corresponding to the error position may be the same as or different from the corresponding designated position. In a particular implementation, the second initial matrix may be determined by the following steps 30-31:
and step 30, detecting the designated target of the designated position corresponding to the error position within a preset range by taking the initial position corresponding to the error position as a center, and taking the position of the detected designated target as a final position corresponding to the error position.
The position with the highest similarity of the designated positions corresponding to the error positions can be detected within a preset range, and the position is taken as the final position corresponding to the error position. For example, each possible position within the preset range may be traversed, the similarity between the possible position and the designated position corresponding to the error position is calculated, and the position with the highest similarity is selected as the final position corresponding to the error position.
And step 31, determining a second initial matrix according to the final position corresponding to the error position, the correct position in the conversion position and the standard position.
The final position corresponding to the error position and the coordinate of the correct position in the conversion position can be reflected to the coordinate of the corresponding standard position, and the second initial matrix can be obtained through calculation.
In step S722, the second initial matrix is determined as a final transformation matrix.
For example, assuming that there are 26 letters in the picture to be processed, the position of each letter is fixed, when the positions of the 26 letters in the picture to be processed need to be located according to the preset standard position of the 26 characters, first, the designated positions of five letters a, b, c, d and e (the number of letters may be any) are detected from the picture to be processed, and according to the designated positions of the five letters and the corresponding standard positions, an initial transformation matrix can be obtained; then, carrying out first transformation processing on the standard position through a transformation matrix to obtain transformation positions of five letters; and calculating the distance between the conversion position of each letter and the corresponding designated position of each letter, if the distance is greater than a preset value, determining that the conversion position of the letter is an error position, and if the distance is not greater than the preset value, determining that the conversion position of the letter is a correct position.
Assuming that only the transformed positions of the letter e are error positions, the first initial matrix is obtained by the transformed positions of the letters a, b, c and d and their corresponding standard positions. And performing second conversion processing on the standard position corresponding to the letter e through the first initial matrix to obtain a conversion position of the letter e, namely the initial position corresponding to the letter e, and finding out the correct position corresponding to the letter e according to the initial position. And when the transformation positions of the five letters are all correct positions, the obtained transformation matrix is the final transformation matrix.
In a specific implementation, if there are multiple error positions in the transformation position, the transformation matrix needs to be adjusted once for each error position, i.e., the above steps S716-S720 are performed once for each error position. For example, among the transformation positions of the letters a, b, c, d and e, the transformation positions of two letters b and e are error positions, and a first initial matrix is obtained by the transformation positions of the letters a, c and d and corresponding standard positions; then, performing second conversion processing on the standard position corresponding to the letter b through the first initial matrix to obtain a conversion position of the letter b, namely the initial position corresponding to the letter b, and finding out the correct position corresponding to the letter b according to the initial position; then obtaining a new first initial matrix according to the correct positions of the letters a, b, c and d and the corresponding standard positions; secondly, performing second transformation processing on the standard position corresponding to the letter e through the new first initial matrix to obtain a transformation position of the letter e, and finding out a correct position corresponding to the letter e according to the transformation position; and when the transformation positions of the five letters are all correct positions, the obtained transformation matrix is the final transformation matrix.
Step S724, positioning the keyboard character in the keyboard picture to be tested according to the final transformation matrix and the position information of the keyboard character.
Step S726, comparing the keyboard characters located in the keyboard picture to be detected with the keyboard characters on the corresponding position in the keyboard drawing, so as to detect the keyboard corresponding to the keyboard picture to be detected.
According to the keyboard appearance detection method, an initial transformation matrix is determined according to the designated position and the standard position; then, performing first transformation processing on the standard position through the initial transformation matrix to obtain transformation positions, and judging whether the distance between the current transformation position and the corresponding specified position is within a preset distance range or not according to each transformation position; if so, determining the current conversion position as a correct position, otherwise, determining the current conversion position as an error position. If the judgment result indicates that an error position exists in the conversion position, determining a first initial matrix according to the correct position in the conversion position and the standard position corresponding to the correct position; performing second transformation processing on the standard position corresponding to the error position through the first initial matrix to obtain an initial position corresponding to the error position; determining a second initial matrix according to the initial position, the correct position and the standard position corresponding to the error position; the second initial matrix is determined as the final transformation matrix. The transformation matrix determined by the method has high accuracy, so that the accuracy of determining the designated target position in the image to be processed can be improved.
Example seven:
the present embodiment provides another keyboard appearance detection method, which is implemented on the basis of the embodiment shown in fig. 4; the embodiment mainly describes a specific process of detecting the appearance of the keyboard in a rectangular coordinate system; as shown in fig. 8; the method comprises the following steps:
step S802, obtaining a keyboard picture to be tested and a keyboard drawing corresponding to the keyboard picture to be tested; and the keyboard picture and the keyboard drawing to be tested comprise keyboard characters.
Step S804, the keyboard picture to be tested and the keyboard drawing corresponding to the keyboard picture to be tested are set in the rectangular coordinate system to determine the position coordinates corresponding to the specified number of keyboard characters.
In a specific implementation, a rectangular coordinate system is established by taking the keyboard character at the upper left corner or the upper right corner of the keyboard drawing as an origin, so as to determine the position coordinate of each keyboard character in the rectangular coordinate system.
Step S806, randomly selecting a designated number of keyboard characters from the keyboard drawing.
Step S808, determining the position coordinates corresponding to the specified number of keyboard characters in the to-be-tested keyboard picture in a template matching manner.
Step S810, mapping the position coordinates of the specified number of keyboard characters in the keyboard picture to be tested to corresponding position coordinates in the keyboard drawing to obtain a first transformation matrix.
Step S812, obtaining the position coordinates of the specified number of keyboard characters in the keyboard picture to be tested through the first transformation matrix and the position coordinates of the specified number of keyboard characters in the keyboard drawing.
Step S814, calculating a difference between the position coordinates of the keyboard character in the keyboard picture to be tested and the position coordinates of the keyboard character in the keyboard drawing, for each keyboard character in the designated number.
The difference value can be generally expressed by the distance between the position coordinates of the keyboard character in the keyboard picture to be tested and the position coordinates of the keyboard character in the keyboard drawing.
Step S816, judging whether the difference value is larger than a preset difference value threshold value; if yes, go to step S818; otherwise, step S820 is performed.
Step S818, determining the position coordinates of the keyboard characters in the keyboard picture to be tested as error position coordinates; step S822 is performed.
Step S820, determining the position coordinates of the keyboard characters in the keyboard picture to be tested as correct position coordinates; step S828 is performed.
Step S822, mapping the position coordinates of the correct position coordinates in the specified number in the to-be-tested keyboard picture to the corresponding position coordinates in the keyboard drawing to obtain a second transformation matrix.
Step S824, obtaining a position coordinate corresponding to the error position coordinate in the keyboard picture to be tested through the second transformation matrix and the error position coordinate in the keyboard drawing.
Step S826, obtaining a final position coordinate of the wrong position coordinate by a template matching mode near the position coordinate corresponding to the wrong position coordinate in the obtained keyboard picture to be tested; and the final position coordinate is used as a correct position coordinate corresponding to the wrong position coordinate.
Step S828, when the position coordinates of each keyboard character in the specified number are all correct position coordinates, a final transformation matrix is obtained.
And step S830, positioning the keyboard characters in the keyboard picture to be detected according to the final transformation matrix and the position information of the keyboard characters.
In step S832, the keyboard characters located in the keyboard picture to be tested are compared with the keyboard characters at the corresponding positions in the keyboard drawing, so as to detect the keyboard corresponding to the keyboard picture to be tested.
According to the keyboard appearance detection method, the appearance of the keyboard to be detected can be detected through the keyboard drawing, the process of manually manufacturing the template library is reduced, a large amount of manpower and material resources are saved, and the characters in the keyboard picture can be accurately positioned through the transformation matrix obtained in the mode.
Example eight:
in correspondence with the above method embodiment, referring to fig. 9, a keyboard appearance detecting apparatus is shown, which includes:
the picture acquiring module 90 is used for acquiring a picture of the keyboard to be detected and a keyboard drawing corresponding to the picture of the keyboard to be detected; the keyboard drawing carries position information of keyboard characters;
a first character position determining module 91, configured to determine a standard position from a keyboard drawing according to the position information of the keyboard character, and determine a specified position from a to-be-detected keyboard picture;
a transformation matrix determining module 92, configured to determine a transformation matrix corresponding to the keyboard picture to be tested according to the standard position and the specified position;
a second character position determining module 93, configured to locate a keyboard character in the to-be-detected keyboard picture according to the transformation matrix and the position information of the keyboard character;
the detection module 94 is configured to compare the keyboard characters located in the keyboard picture to be detected with the keyboard characters on the corresponding position in the keyboard drawing, so as to detect the keyboard corresponding to the keyboard picture to be detected.
Further, before the image obtaining module 90, the keyboard appearance detecting device further includes: the picture processing module is used for carrying out binarization processing on the keyboard drawing corresponding to the keyboard picture to be detected; in the processed keyboard drawing, pixel values of edges of keyboard characters and key cap areas are lower than a preset threshold value; the key cap area determining module is used for detecting the processed key cap area in the keyboard drawing according to the pixel value of the edge of the key cap area; and the position determining module is used for detecting keyboard characters in the keycap area to obtain the position information of the keyboard characters.
Further, the first character position determining module 91 is configured to: determining the position of a preset designated character from the keyboard drawing, and taking the position of the preset designated character as a standard position; and detecting the position of the designated character from the keyboard picture to be detected by adopting a template matching mode, and taking the detected position as the designated position.
Further, the keyboard appearance detecting device further includes: the image position adjusting module is used for determining the scale and the inclination angle of the keyboard in the keyboard image to be detected in a template matching mode; and carrying out scaling and/or rotation processing on the keyboard drawing based on the dimension and the inclination angle.
Further, the transformation matrix determining module 92 includes:
the specified position determining module is used for determining a specified position from a preset image to be processed;
a first matrix determining module, configured to determine an initial transformation matrix according to the specified position and the standard position;
the transformation position determining module is used for carrying out first transformation processing on the standard position through the initial transformation matrix to obtain a transformation position;
the judging module is used for judging whether the transformation position is matched with the specified position or not to obtain a judgment result;
and the transformation matrix determining module is used for adjusting the initial transformation matrix according to the judgment result to obtain a final transformation matrix, and determining the final transformation matrix as the transformation matrix corresponding to the keyboard picture to be detected.
Furthermore, the conversion positions are multiple; the judging module is configured to: for each conversion position, judging whether the distance between the current conversion position and the designated position corresponding to the current conversion position is within a preset distance range; if not, determining that the current transformation position is an error position; if so, the current transformed position is determined to be the correct position.
Further, the transformation matrix determining module is configured to: if the judgment result indicates that an error position exists in the conversion position, determining a first initial matrix according to the correct position in the conversion position and the standard position corresponding to the correct position; performing second transformation processing on the standard position corresponding to the error position through the first initial matrix to obtain an initial position corresponding to the error position; determining a second initial matrix according to the initial position corresponding to the error position, the correct position in the conversion position and a preset standard position; the second initial matrix is determined as the final transformation matrix.
Further, the transformation matrix determining module is further configured to: detecting a designated target of a designated position corresponding to the error position within a preset range by taking an initial position corresponding to the error position as a center, and taking the position of the detected designated target as a final position corresponding to the error position; and determining a second initial matrix according to the final position corresponding to the error position, the correct position in the conversion position and a preset standard position.
Further, the second character position determining module 93 is configured to: transforming the keyboard picture to be tested through the transformation matrix; and positioning the keyboard characters in the converted keyboard picture to be detected according to the position information of the keyboard characters.
Further, the detecting module 94 is configured to: calculating the similarity between the current keyboard character in the keyboard picture to be tested and the keyboard character at the corresponding position in the keyboard drawing aiming at each keyboard character in the keyboard picture to be tested; and determining the detection result of the keyboard corresponding to the keyboard picture to be detected according to the similarity corresponding to each keyboard character.
Further, the detecting module 94 is further configured to: judging whether keyboard characters with the similarity smaller than a preset similarity threshold exist in each keyboard character; if yes, determining that the detection result of the keyboard corresponding to the keyboard picture to be detected is defective; and if the detected keyboard picture does not exist, determining that the detection result of the keyboard corresponding to the keyboard picture to be detected does not exist defects.
Further, after the detecting module 94, the keyboard appearance detecting apparatus further includes: and the alarm module is used for outputting an alarm signal if the detection result of the keyboard corresponding to the to-be-detected keyboard picture indicates that the keyboard has defects.
The keyboard appearance detection device firstly acquires a picture of a keyboard to be detected and a keyboard drawing corresponding to the picture of the keyboard to be detected; then according to the position information of the keyboard characters in the keyboard drawing, determining a standard position from the keyboard drawing and determining a specified position from the picture of the keyboard to be tested; determining a transformation matrix corresponding to the keyboard picture to be tested according to the standard position and the specified position; further, keyboard characters are positioned in the keyboard picture to be detected through the transformation matrix and the position information of the keyboard characters; and finally, comparing the keyboard characters positioned in the keyboard picture to be detected with the keyboard characters on the corresponding positions in the keyboard drawing so as to detect the keyboard corresponding to the keyboard picture to be detected. The mode compares the keyboard picture to be detected with the keyboard drawing, saves the process of manufacturing the template library, improves the operating efficiency of the system, reduces the time of manual operation, and reduces the waste of manpower and material resources.
Example nine:
an embodiment of the present invention provides an electronic system, including: the device comprises an image acquisition device, a processing device and a storage device; the image acquisition equipment is used for acquiring preview video frames or image data; the storage device stores a computer program which, when executed by the processing device, performs the steps of the keyboard appearance detection method.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working process of the electronic system described above may refer to the corresponding process in the foregoing method embodiments, and is not described herein again.
Further, the present embodiment also provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processing device, the steps of the keyboard appearance detection method are executed.
The method, the device and the computer program product of the electronic system for detecting the appearance of the keyboard provided by the embodiment of the invention comprise a computer readable storage medium storing program codes, wherein instructions included in the program codes can be used for executing the method described in the previous method embodiment, and specific implementation can refer to the method embodiment, which is not described herein again.
In addition, in the description of the embodiments of the present invention, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (15)

1. A method for detecting the appearance of a keyboard, the method comprising:
acquiring a keyboard picture to be tested and a keyboard drawing corresponding to the keyboard picture to be tested; the keyboard drawing carries position information of keyboard characters;
according to the position information of the keyboard characters, determining a standard position from the keyboard drawing, and determining a specified position from the keyboard picture to be tested;
determining a transformation matrix corresponding to the keyboard picture to be tested according to the standard position and the designated position;
positioning keyboard characters in the keyboard picture to be detected through the transformation matrix and the position information of the keyboard characters;
and comparing the keyboard characters positioned in the keyboard picture to be detected with the keyboard characters on the corresponding positions in the keyboard drawing so as to detect the keyboard corresponding to the keyboard picture to be detected.
2. The method according to claim 1, wherein before the step of obtaining the keyboard drawing corresponding to the keyboard picture to be tested, the method further comprises:
performing binarization processing on the keyboard drawing corresponding to the keyboard picture to be detected; in the processed keyboard drawing, pixel values of edges of keyboard characters and key cap areas are lower than a preset threshold value;
detecting the key cap area in the processed keyboard drawing according to the pixel value of the edge of the key cap area;
and detecting keyboard characters in the keycap area to obtain the position information of the keyboard characters.
3. The method of claim 1, wherein the step of determining a standard position from the keyboard drawing and a designated position from the picture of the keyboard to be tested according to the position information of the keyboard characters comprises:
determining the position of a preset designated character from the keyboard drawing, and taking the position of the preset designated character as a standard position;
and detecting the position of the designated character from the keyboard picture to be detected in a template matching mode, and taking the detected position as a designated position.
4. The method of claim 3, wherein prior to the step of determining the location of the predetermined designated character from the keyboard drawing, the method further comprises:
determining the scale and the inclination angle of the keyboard in the picture of the keyboard to be detected in a template matching mode;
and carrying out scaling and/or rotation processing on the keyboard drawing based on the scale and the inclination angle.
5. The method according to claim 1, wherein the step of determining the transformation matrix corresponding to the keyboard picture to be tested according to the standard position and the designated position comprises:
determining an initial transformation matrix according to the designated position and the standard position;
performing first transformation processing on the standard position through the initial transformation matrix to obtain a transformation position;
judging whether the transformation position is matched with the specified position or not to obtain a judgment result;
and adjusting the initial transformation matrix according to the judgment result to obtain a final transformation matrix, and determining the final transformation matrix as the transformation matrix corresponding to the keyboard picture to be tested.
6. The method according to claim 5, wherein the conversion position is plural;
the step of judging whether the transformation position is matched with the specified position to obtain a judgment result comprises the following steps:
for each transformation position, judging whether the distance between the current transformation position and the designated position corresponding to the current transformation position is within a preset distance range;
if not, determining that the current conversion position is an error position;
if so, determining that the current conversion position is the correct position.
7. The method of claim 5, wherein the step of adjusting the initial transformation matrix according to the determination result comprises:
if the judgment result indicates that an error position exists in the conversion position, determining a first initial matrix according to a correct position in the conversion position and a standard position corresponding to the correct position;
performing second transformation processing on the standard position corresponding to the error position through the first initial matrix to obtain an initial position corresponding to the error position;
determining a second initial matrix according to the initial position corresponding to the error position, the correct position in the conversion position and the standard position;
determining the second initial matrix as a final transformation matrix.
8. The method according to claim 7, wherein the step of determining a second initial matrix according to the initial position corresponding to the error position, the correct position in the transformed position, and the standard position comprises:
detecting a designated target of a designated position corresponding to the error position within a preset range by taking an initial position corresponding to the error position as a center, and taking the position of the detected designated target as a final position corresponding to the error position;
and determining a second initial matrix according to the final position corresponding to the error position, the correct position in the conversion position and the standard position.
9. The method according to claim 1, wherein the step of locating the keyboard character in the keyboard picture to be tested by the transformation matrix and the position information of the keyboard character comprises:
transforming the keyboard picture to be tested through the transformation matrix;
and positioning keyboard characters in the converted keyboard picture to be detected according to the position information of the keyboard characters.
10. The method according to claim 1, wherein the step of comparing the keyboard characters located in the keyboard picture to be tested with the keyboard characters at the corresponding positions in the keyboard drawing to detect the keyboard corresponding to the keyboard picture to be tested comprises:
calculating the similarity between the current keyboard character in the keyboard picture to be tested and the keyboard character at the corresponding position in the keyboard drawing aiming at each keyboard character in the keyboard picture to be tested;
and determining the detection result of the keyboard corresponding to the keyboard picture to be detected according to the similarity corresponding to each keyboard character.
11. The method according to claim 10, wherein the step of determining the detection result of the keyboard corresponding to the keyboard picture to be detected according to the similarity corresponding to each keyboard character comprises:
judging whether keyboard characters with the similarity smaller than a preset similarity threshold exist in each keyboard character;
if so, determining that the detection result of the keyboard corresponding to the keyboard picture to be detected is defective;
and if the detected keyboard picture does not exist, determining that the detection result of the keyboard corresponding to the to-be-detected keyboard picture does not exist defects.
12. The method according to any one of claims 1 to 11, wherein after the step of detecting the keyboard corresponding to the keyboard picture to be detected, the method further comprises:
and if the detection result of the keyboard corresponding to the to-be-detected keyboard picture indicates that the keyboard has defects, outputting an alarm signal.
13. A keyboard appearance detecting apparatus, comprising:
the image acquisition module is used for acquiring a keyboard image to be detected and a keyboard drawing corresponding to the keyboard image to be detected; the keyboard drawing carries position information of keyboard characters;
the first character position determining module is used for determining a standard position from the keyboard drawing and determining a specified position from the keyboard picture to be detected according to the position information of the keyboard character;
the transformation matrix determining module is used for determining a transformation matrix corresponding to the keyboard picture to be tested according to the standard position and the specified position;
the second character position determining module is used for positioning keyboard characters in the keyboard picture to be detected through the transformation matrix and the position information of the keyboard characters;
and the detection module is used for comparing the keyboard characters positioned in the keyboard picture to be detected with the keyboard characters on the corresponding positions in the keyboard drawing so as to detect the keyboard corresponding to the keyboard picture to be detected.
14. An electronic system, characterized in that the electronic system comprises: the device comprises an image acquisition device, a processing device and a storage device;
the image acquisition equipment is used for acquiring preview video frames or image data;
the storage device has stored thereon a computer program that, when executed by the processing apparatus, performs the steps of the keyboard appearance detection method of any one of claims 1 to 12.
15. A computer-readable storage medium, on which a computer program is stored, wherein the computer program, when executed by a processing device, performs the steps of the keyboard appearance detection method of any one of claims 1 to 12.
CN201910869503.5A 2019-09-12 2019-09-12 Keyboard appearance detection method and device and electronic system Pending CN110596121A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910869503.5A CN110596121A (en) 2019-09-12 2019-09-12 Keyboard appearance detection method and device and electronic system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910869503.5A CN110596121A (en) 2019-09-12 2019-09-12 Keyboard appearance detection method and device and electronic system

Publications (1)

Publication Number Publication Date
CN110596121A true CN110596121A (en) 2019-12-20

Family

ID=68859731

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910869503.5A Pending CN110596121A (en) 2019-09-12 2019-09-12 Keyboard appearance detection method and device and electronic system

Country Status (1)

Country Link
CN (1) CN110596121A (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111046878A (en) * 2019-12-30 2020-04-21 合肥联宝信息技术有限公司 Data processing method and device, computer storage medium and computer
CN111060527A (en) * 2019-12-30 2020-04-24 歌尔股份有限公司 Character defect detection method and device
CN111815621A (en) * 2020-07-27 2020-10-23 北京联想软件有限公司 Data processing method and device and electronic equipment
CN111932491A (en) * 2020-06-23 2020-11-13 联宝(合肥)电子科技有限公司 Component detection method, device and storage medium
CN112525925A (en) * 2020-11-24 2021-03-19 英业达(重庆)有限公司 Keyboard detection method, system, electronic equipment and medium
CN112634259A (en) * 2020-12-30 2021-04-09 凌云光技术股份有限公司 Automatic modeling and positioning method for keyboard keycaps
CN113295617A (en) * 2021-05-18 2021-08-24 广州慧炬智能科技有限公司 Multi-target offset detection method without reference point
CN113379673A (en) * 2021-04-28 2021-09-10 合肥联宝信息技术有限公司 Method, device and equipment for detecting small red cap of keyboard

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080124004A1 (en) * 2006-11-29 2008-05-29 Koichi Suse Method of selecting image, program and storage medium for the method, and image processing apparatus using the method
CN101762231A (en) * 2008-12-26 2010-06-30 比亚迪股份有限公司 Device and method for detecting appearance of mobile phone keys
CN106570511A (en) * 2016-11-02 2017-04-19 上海大学 Keyboard character defect online detection system based on computer vision and keyboard character defect online detection method thereof
CN108416765A (en) * 2018-01-30 2018-08-17 华南理工大学 A kind of character defect automatic testing method and system
CN109919211A (en) * 2019-02-26 2019-06-21 南京旷云科技有限公司 Commodity recognition method, device, system and computer-readable medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080124004A1 (en) * 2006-11-29 2008-05-29 Koichi Suse Method of selecting image, program and storage medium for the method, and image processing apparatus using the method
CN101762231A (en) * 2008-12-26 2010-06-30 比亚迪股份有限公司 Device and method for detecting appearance of mobile phone keys
CN106570511A (en) * 2016-11-02 2017-04-19 上海大学 Keyboard character defect online detection system based on computer vision and keyboard character defect online detection method thereof
CN108416765A (en) * 2018-01-30 2018-08-17 华南理工大学 A kind of character defect automatic testing method and system
CN109919211A (en) * 2019-02-26 2019-06-21 南京旷云科技有限公司 Commodity recognition method, device, system and computer-readable medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
严圣华 等: "《计算机组装与维护 第2版》", 31 July 2016, 北京理工大学出版社, pages: 98 *

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111046878A (en) * 2019-12-30 2020-04-21 合肥联宝信息技术有限公司 Data processing method and device, computer storage medium and computer
CN111060527A (en) * 2019-12-30 2020-04-24 歌尔股份有限公司 Character defect detection method and device
CN111046878B (en) * 2019-12-30 2022-02-08 合肥联宝信息技术有限公司 Data processing method and device, computer storage medium and computer
US12002198B2 (en) 2019-12-30 2024-06-04 Goertek Inc. Character defect detection method and device
CN111932491A (en) * 2020-06-23 2020-11-13 联宝(合肥)电子科技有限公司 Component detection method, device and storage medium
CN111932491B (en) * 2020-06-23 2022-02-08 联宝(合肥)电子科技有限公司 Component detection method, device and storage medium
CN111815621A (en) * 2020-07-27 2020-10-23 北京联想软件有限公司 Data processing method and device and electronic equipment
CN112525925A (en) * 2020-11-24 2021-03-19 英业达(重庆)有限公司 Keyboard detection method, system, electronic equipment and medium
CN112634259A (en) * 2020-12-30 2021-04-09 凌云光技术股份有限公司 Automatic modeling and positioning method for keyboard keycaps
CN113379673A (en) * 2021-04-28 2021-09-10 合肥联宝信息技术有限公司 Method, device and equipment for detecting small red cap of keyboard
CN113379673B (en) * 2021-04-28 2022-02-01 合肥联宝信息技术有限公司 Method, device and equipment for detecting small red cap of keyboard
CN113295617A (en) * 2021-05-18 2021-08-24 广州慧炬智能科技有限公司 Multi-target offset detection method without reference point

Similar Documents

Publication Publication Date Title
CN110596121A (en) Keyboard appearance detection method and device and electronic system
CN110443110B (en) Face recognition method, device, terminal and storage medium based on multipath camera shooting
CN111179358A (en) Calibration method, device, equipment and storage medium
CN110189322B (en) Flatness detection method, device, equipment, storage medium and system
US20130202161A1 (en) Enhanced face detection using depth information
US20140270362A1 (en) Fast edge-based object relocalization and detection using contextual filtering
JP2010262601A (en) Pattern recognition system and pattern recognition method
US10395090B2 (en) Symbol detection for desired image reconstruction
CN114913121A (en) Screen defect detection system and method, electronic device and readable storage medium
CN111222452A (en) Face matching method and device, electronic equipment and readable storage medium
WO2018121414A1 (en) Electronic device, and target image recognition method and apparatus
CN111354038B (en) Anchor detection method and device, electronic equipment and storage medium
JP5264457B2 (en) Object detection device
CN114235815A (en) Method for detecting surface defects of outdoor electrical equipment of converter station based on scene filtering
CN111080683B (en) Image processing method, device, storage medium and electronic equipment
CN112036232A (en) Image table structure identification method, system, terminal and storage medium
CN115661131B (en) Image identification method and device, electronic equipment and storage medium
CN112084365A (en) Real-time image retrieval method of network camera based on OpenCV and CUDA acceleration
WO2020135097A1 (en) Method and apparatus for channel switch detection of display terminal
CN111885371A (en) Image occlusion detection method and device, electronic equipment and computer readable medium
JP2016206909A (en) Information processor, and information processing method
JP5754931B2 (en) Image analysis apparatus, image analysis method, and program
CN115439875A (en) Posture evaluation device, method and system
CN112418244A (en) Target detection method, device and electronic system
CN113870210A (en) Image quality evaluation method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20191220