CN106355592B - Educational toy set, circuit element thereof and wire identification method - Google Patents
Educational toy set, circuit element thereof and wire identification method Download PDFInfo
- Publication number
- CN106355592B CN106355592B CN201610698013.XA CN201610698013A CN106355592B CN 106355592 B CN106355592 B CN 106355592B CN 201610698013 A CN201610698013 A CN 201610698013A CN 106355592 B CN106355592 B CN 106355592B
- Authority
- CN
- China
- Prior art keywords
- image
- color
- circuit components
- circuit
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of computer vision detection processing, and provides an educational toy set and a circuit element and wire identification method thereof, wherein the method comprises the following steps: the bottom plate is placed on the plane, and the circuit components and the electric wires are placed on the bottom plate. The invention places circuit components and wires on a game bottom plate, installs game programs in a tablet personal computer, acquires images of the circuit components and the wires placed on the bottom plate through a camera of the tablet personal computer, identifies the circuit components and the wires based on predefined color, contour information and color coding information, and enables children to connect the circuit components and the wires to judge whether the connected circuits are correct or not, thereby enhancing the imagination of the children, increasing the game interest, enabling the children to learn basic circuit knowledge and cultivating the interests of the children.
Description
Technical Field
The invention relates to the technical field of computer vision detection processing, in particular to an educational toy set and a circuit element and wire identification method thereof.
Background
At present, a plurality of interesting preschool education game application programs or children games exist on a tablet personal computer, but the interaction is lacked, so that children can only point to draw pictures on a screen, the eyes are easily injured when the children look at the screen for a long time, physical knowledge education is lacked, and logical thinking culture is lacked; moreover, some traditional game toys with strong interactivity break away from the development of the times, so that the requirements of children on learning and playing cannot be met formally, and the interactive communication between the children and parents is not convenient.
In order to solve the above problems, the computer vision and image processing technology field has successfully developed an educational toy set comprising: the system comprises a support, a helmet detector and a bottom plate, wherein a game program is installed in a tablet personal computer, and images of the bottom plate placed on a plane are collected through a camera of the tablet personal computer. Although the education toy set solves the problem of lacking interactivity of games in the tablet personal computer, the education toy set is still single in form, only interacts simply, is not embedded with physical knowledge, cannot cultivate electrical knowledge for children from childhood, and cannot cultivate the knowledge and interest of children on electricity from childhood.
Therefore, an educational toy set, a circuit element and an electric wire identification method thereof are urgently needed in the technical field of computer vision detection processing, the circuit element and the electric wire are placed on a game bottom plate, a game program is installed in a tablet personal computer, the circuit element and the electric wire image placed on the bottom plate are collected through a camera of the tablet personal computer, the circuit element and the electric wire are identified based on predefined color, outline information and color coding information, and a child connects the circuit element and the electric wire to judge whether the connected circuit is correct or not, so that the imagination of the child is enhanced, the game interest is increased, the child can learn basic circuit knowledge and the interest of the child is developed.
Disclosure of Invention
In order to solve the above problems, the present invention provides an educational toy set, a circuit element thereof, and a method for identifying an electric wire, the technical scheme is as follows:
an educational toy set comprising: the tablet personal computer, the bottom plate, the circuit components and the wire bottom plate are placed on the plane, and the circuit components and the wire are placed on the bottom plate; and the corners of the bottom plate are provided with calibration angles.
Preferably, in the above-mentioned educational toy set, the base plate has a rectangular shape with rounded corners; the calibration angle is a red circular arc.
A method of identifying circuit elements and wires in an educational toy set, comprising the steps of:
step one, installing a game program in a tablet personal computer, and then placing a bottom plate on a plane to ensure that one surface of a calibration angle faces upwards
Step two, completing the connection of circuit components and wires on a bottom plate, collecting a color image in real time through a rear camera of a tablet computer, and moving the tablet computer to ensure that the color image collected by the rear camera at least contains 3 calibration angles;
step three, extracting an effective identification area from the color image in the step two;
detecting circuit components in the effective identification area of the color image;
step five, detecting the electric wire positioned in the effective identification area of the color image;
and step six, judging whether the connection between the circuit component and the electric wire is accurate.
Preferably, in the above method for identifying circuit components and wires in an educational toy set, the color image collected by the rear camera in the second step is Ixy，Ixy＝f(x,y)＝(Rxy,Gxy,Bxy) Wherein (x, y) represents the position coordinates of the pixel points of the color image, f (x, y) represents the pixel values of the image at the coordinate positions of the pixel points, RxyRepresenting the color value, G, of a pixel of an image in the red channelxyRepresenting the color value of a pixel of an image in the green channel, BxyAnd the color value of the pixel point of the image in the blue channel is represented.
Preferably, in the above method for identifying circuit elements and wires in an educational toy set, the specific steps of extracting the effective identification area from the color image in the third step are:
A) according to the priori knowledge, 4 calibration angle regions are segmented from the color image in the second step, and according to the priori threshold value in the HSV space, binarization processing is carried out on the 4 calibration angle region images to obtain 4 calibration angle binary images;
B) scanning the 4 calibration angle binary images obtained in the step A) to obtain corresponding edge contour images, and filtering unreasonable contours according to the prior knowledge of the eccentricity and the size of the edge contours;
C) calculating circumscribed rectangles of 4 calibration angles according to the residual edge profile obtained in the step B), wherein in the identification process, when at least three corner marks have qualified calibration angles, the circumscribed rectangles are the calculated effective identification areas.
Preferably, in the above method for identifying circuit components and wires in an educational toy set, the step of detecting circuit components located in the color image identification area in the fourth step comprises the steps of:
1, dividing each circuit component according to the color difference because the color of the shell of each circuit component is different, and extracting the inner contour of the shell of each circuit component;
2, calculating the position and deflection angle of each circuit component according to the inner contour of the shell of the circuit component extracted in the step 1;
and 3, rotating the circuit component according to the deflection angle calculated in the step 2, then dividing the circuit component, and identifying the type of the circuit component through color coding.
Preferably, in the above method for identifying circuit components and wires in an educational toy set, the specific steps of extracting the inner contour of each circuit component housing in step 1 are as follows:
a) because the colors of the circuit components are not beneficial to being divided in the RGB color space and are sensitive to illumination change, the extracted effective identification area image is converted into HSV color space which is emphasized on color representation from the RGB color space, and the specific conversion formula is as follows:
V＝max{C(R′)、C(G′)、C(B′)}；
h represents a hue value, S represents a saturation value, V represents a brightness value, max { C (R '), C (G'), C (B ') } represents the maximum value of a pixel point in three channels of red, green and blue in an original image, min { C (R'), C (G ') and C (B') } represent the minimum value of a pixel point in three channels of red, green and blue in the original image, and the value range of H is between 0 and 360;
b) in an HSV color space, carrying out binarization processing on the color image according to a prior threshold value of the color related to the circuit component in the HSV color space, wherein a specific formula is as follows:
when B (x, y) ═ B _ H (x, y) & B _ S (x, y) & B _ V (x, y) in the binary image, the binary image is generated;
b (x, y) represents a binary pixel value of an image pixel (x, y), and H (x, y), S (x, y) and V (x, y) respectively represent a hue value, a saturation value and a brightness value of the image pixel (x, y) in an HSV color space; b _ H (x, y), B _ S (x, y) and B _ V (x, y) respectively represent whether the image pixel points (x, y) are respectively in the designated H, S, V areas, if yes, the value is 1, otherwise, the value is 0; hmin、HmaxRespectively representing prior minimum and maximum values of hues of the color of a certain component shell in an HSV color space; smin、SmaxRespectively representing prior minimum and maximum values of the saturation of the color of a certain component shell in an HSV color space; vmin、VmaxRespectively representing the prior minimum and maximum values of the brightness of the color of a certain component shell in the HSV color space.
c) Scanning the binary image to find out all edge contours;
the binary image can be regarded as a gray image with only two values, the edge of the image refers to a part with severe gray change in the gray image, the change degree of the gray value is quantitatively expressed by the gradient change between adjacent pixels, the gradient is a two-dimensional equivalent formula of a first-order two-dimensional derivative, and the specific calculation process is as follows:
firstly, calculating the difference of adjacent pixels, wherein the specific formula is as follows:
Gx＝f[i，j+1]-f[i，j]
Gy＝f[i，j]-f[i+1，j]
wherein G isxRepresenting the difference of adjacent pixels in the x-direction, GyDenotes the difference in y direction between adjacent pixels, f [ i, j +1]Representing the pixel value of the image in the ith row, column j +1, f i, j]The pixel value of the image in the ith row and the jth column is represented; f [ i +1, j ]]The pixel value of the j th column in the i +1 th row of the image is represented;
further, the gradient between adjacent pixels is calculated by the following specific formula:
wherein G (x, y) represents a gradient value at an (x, y) point representing an image,
indicating that the pixel values are differentiated in the x-direction,
indicating that the pixel values are derived in the y-direction;
further, calculating the gradient amplitudes of the edge points, wherein the gradient amplitude set of all the edge points is the extracted edge profile;
furthermore, the method for calculating the ears of the non-stationary electric fan and other circuit components filters unreasonable outlines according to the priori knowledge of the shapes, sizes and eccentricities of the ear outlines of the circuit components, so that the positions of 2 ears of the circuit components are obtained;
because the ears of the stationary fan are possibly covered by the fan blades, the color image in the step b) needs to be segmented into a blue binary image and a green binary image;
further, scanning the blue binary image, judging whether the fan blades exist or not through the prior knowledge of the contour shape, eccentricity and size of the fan blades, and if so, judging that the circuit component is a fan; otherwise, directly jumping to the step 2;
further, after the circuit component is judged to be a fan, the calculation method for the ear of the stationary fan is as follows: clustering the green binary image in the step 1, clustering the green points with close distance into one class, filtering the point set clustered into one class through the number of the point sets of the same class, the size of the minimum outsourcing rectangle of the point sets, the eccentricity and the priori knowledge of the positions of the point sets, if only two classes of point sets are left after filtering, considering that the current fan blade position covers the ears of the electric component with high probability, and calculating the position of the center point of each ear of the electric component through the position of the center point between the two classes of point sets, namely the position of the center point of the electric component and the contour information of the component.
Preferably, in the above method for identifying circuit components and wires in an educational toy set, the step 2 of calculating the position and deflection angle of each circuit component comprises the following specific steps:
and calculating the rotation angle of the circuit component according to the calculated position of the central point of the circuit component and the position of the central point of each ear by taking the ear of the circuit component in the horizontal direction as a standard.
Preferably, in the above method for identifying circuit components and wires in an educational toy set, the specific step of identifying the type of the circuit component in step 3 is:
firstly, a coding rule needs to be set for all circuit components in advance, so that each circuit component has a unique code; because the number of the circuit components to be identified is limited, four easily-distinguished colors of red, yellow, blue and green are selected as the coding characteristic colors; the private printing of the circuit components is mainly concentrated in three areas, namely, an upper area, a middle area and a lower area, and when a certain color pixel of the three areas exceeds a color pixel threshold value, the color is considered as the color of the area;
according to the shell color of the circuit components, the private printing colors of the upper middle part, the middle part and the lower middle part, red is marked as 1, yellow is marked as 2, blue is marked as 3, green is marked as 4, no marking is marked as 0, the current color is also marked as 0 after neglecting, 11 circuit components can be coded, and the type of the current detection circuit component can be uniquely determined according to the coding of the circuit components;
and filtering circuit components which do not accord with the codes, and transmitting the types of the remaining circuit components, the central points of the circuit components and the rotation angles to upper-layer software together.
Preferably, in the above-mentioned method of recognizing circuit elements and electric wires in an educational toy set, the color pixel threshold is 30% of the area of each of the nine cells.
Preferably, in the above method for identifying circuit elements and wires in an educational toy set, the step five of detecting the wires located in the effective identification area of the color image comprises the specific steps of:
step 1), segmenting wires and extracting bones through color difference;
and 2) calculating the end point and the bifurcation point of the electric wire through the extracted bones.
Preferably, in the above method for identifying circuit elements and wires in an educational toy set, the specific steps of segmenting the wires by color difference and extracting bones in step 1) are:
firstly, in an effective identification area HSV space in the third step, carrying out binarization processing on an image according to a prior threshold value of the color of the wire in the HSV space to obtain a wire binarization image;
further, scanning the wire binary image, filtering out contours which do not accord with the characteristics of the wire through the priori knowledge of the shape and the size of the contour of the wire, and filling the remaining contours to obtain a new wire-only binary image;
further, the skeleton of the wire is obtained by thinning the binary image of the wire only.
Preferably, in the above method for identifying circuit elements and wires in an educational toy set, the specific steps of calculating the end points and branch points of the wires through the extracted skeleton in step 2) are: finding the end points and the branch points in the wire skeleton through the wire skeleton obtained in the step 1) and the prior knowledge of the end points and the branch point characteristics of the wire, if the branch points exist, separating the wire skeleton from the branch points to be divided into a plurality of line segments, if the branch points do not exist, the original wire skeleton is a line segment, and providing all the line segments and the types of the end points of the line segments in the skeleton to upper-layer software.
Preferably, in the above method for identifying a circuit component and an electric wire in an educational toy set, the step six of determining whether the connection between the circuit component and the electric wire is accurate includes the specific steps of: and the upper software connects the circuit component identified in the step four with the electric wire identified in the step five, and then compares the electric wire with a circuit diagram stored by the upper software, and if the electric wire is consistent with the circuit diagram, the circuit connection is considered to be accurate.
The invention has the beneficial effects that:
1. the invention skillfully combines the applied computer vision figure recognition technology with HSV color space, binarization processing and image cutting technology for use, can judge the types and electric wires of circuit components and can judge whether the circuit connection is accurate, has the advantages of high operation speed and accurate positioning, well unifies hardware and software technologies, and has ingenious game interaction design; beautiful and simple, the judgment is quicker, the imagination of the children is enhanced, the interesting of the game is increased, the children can learn the basic circuit knowledge, and the interest of the children is cultivated.
2. The detection algorithm is more scientific and mature, and the algorithms such as color conversion of images, image convolution, image cutting, skeleton thinning and the like are combined for use, so that the types of circuit components and wires can be quickly judged.
3. The invention has high calculation speed; each positioning detection takes about 200ms, and a smooth use experience is provided for the player.
4. The invention has stable performance, and the error recognition rate and the omission factor are below 0.2 percent when 3 thousand pictures are acquired and tested under the conditions of different illumination and installation of different tablet computers in the educational toy suite.
Drawings
The invention is described in detail below with reference to the following figures and detailed description:
fig. 1 is a schematic view showing the construction of an educational toy set of the present invention.
Fig. 2 is a flowchart of a method of identifying circuit elements and wires in an educational toy set according to the present invention.
Fig. 3 is a table diagram of specific circuit component codes of step 3 in step four of the educational toy set of the present invention.
Wherein, the corresponding relationship between the reference numbers and the part names in fig. 1-3 is:
Detailed Description
In order to make the measures, features, objectives and functions of the present invention easy to be understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, a toy set and a method for identifying circuit components and electric wires thereof, a base plate 1, a circuit component 2 and an electric wire 3, wherein the base plate 1 is placed on a plane, and the circuit component 2 and the electric wire 3 are placed on the base plate 3.
In this embodiment, the bottom plate is a rectangle with rounded corners, and 4 corners of the rectangle are provided with calibration corners; preferably, the alignment angle is a red circular arc.
The circuit components comprise a static electric fan, a non-static electric fan and other circuit components, and each circuit component is provided with 2 symmetrical ears.
As shown in fig. 2, a method for identifying circuit elements and wires in an educational toy set, comprising the steps of:
step one, installing a game program in a tablet personal computer, and then placing a bottom plate on a plane to ensure that one surface of a calibration angle faces upwards;
step two, accomplish the connection of circuit components and parts and electric wire on the bottom plate, gather the color image through the rear camera of panel computer in real time, remove the panel computer, guarantee that the color image that the rear camera was gathered contains 3 calibration angles at least, concrete step is:
the color image collected by the rear camera is Ixy，Ixy＝f(x,y)＝(Rxy,Gxy,Bxy) Wherein (x, y) represents the position coordinates of the pixel points of the color image, f (x, y) represents the pixel values of the image at the coordinate positions of the pixel points, RxyRepresenting the color value, G, of a pixel of an image in the red channelxyRepresenting the color value of a pixel of an image in the green channel, BxyRepresenting the color value of the image pixel point in a blue channel;
step three, extracting an effective identification area from the color image in the step two, and specifically comprising the following steps:
A) according to the priori knowledge, 4 calibration angle regions are segmented from the color image in the second step, and according to the priori threshold value in the HSV space, binarization processing is carried out on the 4 calibration angle region images to obtain 4 calibration angle binary images;
B) scanning the 4 calibration angle binary images obtained in the step A) to obtain corresponding edge contour images, and filtering unreasonable contours according to the prior knowledge of the eccentricity and the size of the edge contours;
C) calculating circumscribed rectangles of 4 calibration angles according to the residual edge profile obtained in the step B), wherein in the identification process, when at least three corner marks have qualified calibration angles, the circumscribed rectangles are calculated effective identification areas;
detecting circuit components in the effective identification area of the color image, and specifically comprising the following steps:
1, because the colour of every circuit components and parts shell is different, consequently through the colour difference, divide out each circuit components and parts to extract the interior profile of every circuit components and parts shell, concrete step is:
a) because the colors of the circuit components are not beneficial to being divided in the RGB color space and are sensitive to illumination change, the extracted effective identification area image is converted into HSV color space which is emphasized on color representation from the RGB color space, and the specific conversion formula is as follows:
V＝max{C(R′)、C(G′)、C(B′)}；
h represents a hue value, S represents a saturation value, V represents a brightness value, max { C (R '), C (G'), C (B ') } represents the maximum value of a pixel point in three channels of red, green and blue in an original image, min { C (R'), C (G ') and C (B') } represent the minimum value of a pixel point in three channels of red, green and blue in the original image, and the value range of H is between 0 and 360;
b) in an HSV color space, carrying out binarization processing on the color image according to a prior threshold value of a color related to a shell of a circuit component in the HSV color space, wherein a specific formula is as follows:
when B (x, y) ═ B _ H (x, y) & B _ S (x, y) & B-V (x, y) in the binary image, the binary image is generated;
b (x, y) represents a binary pixel value of an image pixel (x, y), and H (x, y), S (x, y) and V (x, y) respectively represent a hue value, a saturation value and a brightness value of the image pixel (x, y) in an HSV color space; b _ H (x, y), B _ S (x, y) and B _ V (x, y) respectively represent whether the image pixel points (x, y) are respectively in the designated H, S, V areas, if yes, the value is 1, otherwise, the value is 0; hmin、HmaxRespectively representing prior minimum and maximum values of hues of the color of a certain component shell in an HSV color space; smin、SmaxRespectively representing prior minimum and maximum values of the saturation of the color of a certain component shell in an HSV color space; vmin、VmaxRespectively representing prior minimum and maximum values of brightness of the color of a certain component shell in an HSV color space;
c) scanning the binary image to find out all edge contours;
the binary image can be regarded as a gray image with only two values, the edge of the image refers to a part with severe gray change in the gray image, the change degree of the gray value is quantitatively expressed by the gradient change between adjacent pixels, the gradient is a two-dimensional equivalent formula of a first-order two-dimensional derivative, and the specific calculation process is as follows:
firstly, calculating the difference of adjacent pixels, wherein the specific formula is as follows:
Gx＝f[i，j+1]-f[i，j]
Gy＝f[i，j]-f[i+1，j]
wherein G isxRepresenting the difference of adjacent pixels in the x-direction, GyDenotes the difference in y direction between adjacent pixels, f [ i, j +1]Representing the pixel value of the image in the ith row, column j +1, f i, j]The pixel value of the image in the ith row and the jth column is represented; f [ i +1, j ]]Representing the pixel value of the image at row i +1 and column j,
further, the gradient between adjacent pixels is calculated by the following specific formula:
wherein G (x, y) represents a gradient value at an (x, y) point representing an image,
indicating that the pixel values are differentiated in the x-direction,
indicating that the pixel value is at yDerivation in direction;
further, calculating the gradient amplitudes of the edge points, wherein the gradient amplitude set of all the edge points is the extracted edge profile;
furthermore, the method for calculating the ears of the non-stationary electric fan and other circuit components filters unreasonable outlines according to the priori knowledge of the shapes, sizes and eccentricities of the ear outlines of the circuit components, so that the positions of 2 ears of the circuit components are obtained;
because the ears of the stationary fan are possibly covered by the fan blades partially or completely, the color segmentation is carried out on the step b) to segment a blue binary image and a green binary image;
further, scanning the blue binary image, judging whether the fan blades exist or not through the prior knowledge of the contour shape, eccentricity and size of the fan blades, and if so, judging that the circuit component is a fan; otherwise, directly jumping to the step 2;
further, after the circuit component is judged to be a fan, the calculation method for the ear of the stationary fan is as follows: clustering the green binary image in the step 1, clustering the green points with close distance into one class, filtering the clustered point sets through the number of the point sets of the same class, the size of the minimum outsourcing rectangle of the point sets, the eccentricity and the priori knowledge of the positions of the point sets, if only two classes of point sets are left after filtering, considering that the current fan blade position covers the ears of the electric component with high probability, and calculating the position of the center point of each ear of the electric component through the position of the center point between the two classes of point sets, namely the position of the center point of the electric component and the contour information of the electric component;
2, calculating the position and deflection angle of each circuit component according to the contour of the circuit component extracted in the step 1, and the specific steps are as follows:
and calculating the rotation angle of the circuit component according to the calculated position of the central point of the circuit component and the position of the central point of each ear by taking the ear of the circuit component in the horizontal direction as a standard.
Rotating the circuit component according to the deflection angle calculated in the step 2, then dividing the circuit component, and identifying the category of the circuit component through color coding, wherein the method specifically comprises the following steps:
firstly, a coding rule needs to be set for all circuit components in advance, so that each circuit component has a unique code; because the number of the circuit components to be identified is limited, four easily-distinguished colors of red, yellow, blue and green are selected as the coding characteristic colors; the private printing of the circuit components is mainly concentrated in three areas, namely, an upper area, a middle area, a lower area and a middle area, and when the pixel value of a certain color of the three areas exceeds a prior threshold value, the color is considered as the color of the area;
according to the shell color of the circuit components, the private printing colors of the upper middle part, the middle part and the lower middle part, red is marked as 1, yellow is marked as 2, blue is marked as 3, green is marked as 4, no marking is marked as 0, the current color is also marked as 0 after neglecting, 11 circuit components can be coded, and the type of the current detection circuit component can be uniquely determined according to the coding of the circuit components;
filtering out circuit components which do not accord with codes, and transmitting the types of the remaining circuit components, the central points of the circuit components and the rotation angles to upper-layer software together, wherein the specific codes of the circuit components are shown in figure 3;
step five, detecting the electric wire positioned in the effective identification area of the color image, and the specific steps are as follows:
step 1), segmenting wires and extracting bones through color difference, and the method comprises the following specific steps:
firstly, in an effective identification area HSV space in the third step, carrying out binarization processing on an image according to a prior threshold value of the color of the wire in the HSV space to obtain a wire binarization image;
further, scanning the wire binary image, filtering out contours which do not accord with the characteristics of the wire through the priori knowledge of the shape and the size of the contour of the wire, and filling the remaining contours to obtain a new wire-only binary image;
further, thinning the bones of the binary image only with the electric wires to obtain bones of the electric wires;
step 2), calculating the end point and the bifurcation point of the electric wire through the extracted bones, and the specific steps are as follows: finding out the end points and branch points in the wire skeleton through the wire skeleton obtained in the step 1) and the prior knowledge of the end points and branch point characteristics of the wire, if the branch points exist, separating the wire skeleton from the branch points to form a plurality of line segments, if the branch points do not exist, the original wire skeleton is a line segment, and providing all the line segments and the types of the end points of the line segments in the skeleton to upper-layer software;
and step six, judging whether the connection between the circuit component and the electric wire is accurate or not, and specifically comprising the following steps:
and the upper software connects the circuit component identified in the step four with the electric wire identified in the step five, and then compares the electric wire with a circuit diagram stored by the upper software, and if the electric wire is consistent with the circuit diagram, the circuit connection is considered to be accurate.
In this embodiment, the color pixel threshold is 30% of the area of each of the nine-square squares.
The invention skillfully combines the applied computer vision figure recognition technology with HSV color space, binarization processing and image cutting technology for use, can judge the types and electric wires of circuit components and can judge whether the circuit connection is accurate, has the advantages of high operation speed and accurate positioning, well unifies hardware and software technologies, and has ingenious game interaction design; beautiful and simple, the judgment is quicker, the imagination of the children is enhanced, the interesting of the game is increased, the children can learn the basic circuit knowledge, and the interest of the children is cultivated.
The detection algorithm is more scientific and mature, and the algorithms such as color conversion of images, image convolution, image cutting, skeleton thinning and the like are combined for use, so that the types of circuit components and wires can be quickly judged.
The invention has high calculation speed; each positioning detection takes about 200ms, and a smooth use experience is provided for the player.
The invention has stable performance, and the error recognition rate and the omission factor are below 0.2 percent when 3 thousand pictures are acquired and tested under the conditions of different illumination and installation of different tablet computers in the educational toy suite.
The foregoing shows and describes the general principles, essential features, and advantages of the invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are given by way of illustration of the principles of the present invention, and that various changes and modifications may be made without departing from the spirit and scope of the invention as defined by the appended claims. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (6)
1. A method of identifying circuit elements and wires in an educational toy set, comprising the steps of:
step one, installing a game program in a tablet personal computer, and then placing a bottom plate on a plane to ensure that one surface of a calibration angle faces upwards;
step two, accomplish the connection of circuit components and parts and electric wire on the bottom plate, gather the color image through the rear camera of panel computer in real time, remove the panel computer, guarantee that the color image that the rear camera was gathered contains 3 calibration angles at least, concrete step is:
the color image collected by the rear camera is Ixy，Ixy＝f(x，y)＝(Rxy，Gxy，Bxy) Wherein (x, y) represents the position coordinates of the pixel points of the color image, f (x, y) represents the pixel values of the image at the coordinate positions of the pixel points, RxyRepresenting the color value, G, of a pixel of an image in the red channelxyRepresenting the color value of a pixel of an image in the green channel, BxyRepresenting the color value of the image pixel point in a blue channel;
step three, extracting an effective identification area from the color image in the step two;
detecting circuit components in the effective identification area of the color image;
step five, detecting the electric wire positioned in the effective identification area of the color image;
judging whether the connection between the circuit component and the electric wire is accurate or not;
the fourth step of detecting the circuit components in the effective identification area of the color image comprises the following specific steps:
1, segmenting each circuit component through color difference due to different colors of shells of each component, and extracting the inner contour of the shell of each circuit component;
2, calculating the position and deflection angle of each circuit component according to the inner contour of the shell of the circuit component extracted in the step 1;
rotating the circuit components according to the deflection angle calculated in the step 2, then dividing the circuit components, and identifying the types of the circuit components through color coding;
the specific steps of extracting the outline of each circuit component in the step 1 are as follows:
a) because the colors of the circuit components are not beneficial to being divided in the RGB color space and are sensitive to illumination change, the extracted effective identification area image is converted into HSV color space which is emphasized on color representation from the RGB color space, and the specific conversion formula is as follows:
V＝max{C(R′)、C(G′)、C(B′)}；
h represents a hue value, S represents a saturation value, V represents a brightness value, max { C (R '), C (G'), C (B ') } represents the maximum value of a pixel point in three channels of red, green and blue in an original image, min { C (R'), C (G ') and C (B') } represent the minimum value of a pixel point in three channels of red, green and blue in the original image, and the value range of H is between 0 and 360;
b) in an HSV color space, carrying out binarization processing on the color image according to a prior threshold value of a color related to a shell of a circuit component in the HSV color space, wherein a specific formula is as follows:
when B (x, y) ═ B _ H (x, y) & B-S (x, y) & B-V (x, y) in the binary image, the binary image is generated;
b (x, y) represents a binary pixel value of an image pixel (x, y), and H (x, y), S (x, y) and V (x, y) respectively represent a hue value, a saturation value and a brightness value of the image pixel (x, y) in an HSV color space; b _ H (x, y), B-S (x, y) and B-V (x, y) respectively represent whether the image pixel points (x, y) are respectively in the designated H, S, V areas, if yes, the value is 1, otherwise, the value is 0; hmin、HmaxRespectively representing prior minimum and maximum values of hues of the color of a certain component shell in an HSV color space; smin、SmaxRespectively representing prior minimum and maximum values of the saturation of the color of a certain component shell in an HSV color space; vmin、VmaxRespectively representing prior minimum and maximum values of brightness of the color of a certain component shell in an HSV color space;
c) scanning the binary image to find out all edge contours;
the binary image can be regarded as a gray image with only two values, the edge of the image refers to a part with severe gray change in the gray image, the change degree of the gray value is quantitatively expressed by the gradient change between adjacent pixels, the gradient is a two-dimensional equivalent formula of a first-order two-dimensional derivative, and the specific calculation process is as follows:
firstly, calculating the difference of adjacent pixels, wherein the specific formula is as follows:
Gx＝f[i，j+1]-f[i，j]
Gy＝f[i，j]-f[i+1，j]
wherein G isxRepresenting the difference of adjacent pixels in the x-direction, GyDenotes the difference in y direction between adjacent pixels, f [ i, j +1]Representing the pixel value of the image in the ith row, column j +1, f i, j]The pixel value of the image in the ith row and the jth column is represented; f [ i +1, j ]]Representing the pixel value of the image at row i +1 and column j,
further, the gradient between adjacent pixels is calculated by the following specific formula:
wherein G (x, y) represents a gradient value at an (x, y) point representing an image,
indicating that the pixel values are differentiated in the x-direction,
indicating that the pixel values are derived in the y-direction;
further, calculating the gradient amplitudes of the edge points, wherein the gradient amplitude set of all the edge points is the extracted edge profile;
furthermore, the method for calculating the ears of the non-stationary electric fan and other circuit components filters unreasonable outlines according to the priori knowledge of the shapes, sizes and eccentricities of the ear outlines of the circuit components, so that the positions of 2 ears of the circuit components are obtained;
the ears of the stationary fan are possibly covered by the fan blades, so that the color image in the step b) needs to be segmented into a blue binary image and a green binary image;
further, scanning the blue binary image, judging whether the fan blades exist or not through the prior knowledge of the contour shape, eccentricity and size of the fan blades, and if so, judging that the circuit component is a fan; otherwise, directly jumping to the step 2;
further, after the circuit component is judged to be a fan, the calculation method for the ear of the stationary fan is as follows: clustering the green binary image in the step 1, clustering the green points with close distance into one class, filtering the point set clustered into one class through the number of the point sets of the same class, the size of the minimum outsourcing rectangle of the point sets, the eccentricity and the priori knowledge of the positions of the point sets, if only two classes of point sets are left after filtering, considering that the current fan blade position covers the ears of the electric component with high probability, and calculating the position of the center point of each ear of the electric component through the position of the center point between the two classes of point sets, namely the position of the center point of the electric component and the contour information of the component.
2. The method as claimed in claim 1, wherein the step three of extracting the effective identification area from the color image comprises the following steps:
A) according to the priori knowledge, 4 calibration angle regions are segmented from the color image in the second step, and according to the priori threshold value in the HSV space, binarization processing is carried out on the 4 calibration angle region images to obtain 4 calibration angle binary images;
B) scanning the 4 calibration angle binary images obtained in the step A) to obtain corresponding edge contour images, and filtering unreasonable contours according to the prior knowledge of the eccentricity and the size of the edge contours;
C) calculating circumscribed rectangles of 4 calibration angles according to the residual edge profile obtained in the step B), wherein in the identification process, when at least three corner marks have qualified calibration angles, the circumscribed rectangles are the calculated effective identification areas.
3. The method as claimed in claim 1, wherein the step 2 of calculating the position and deflection angle of each circuit component comprises the following steps:
and calculating the rotation angle of the circuit component according to the calculated position of the central point of the circuit component and the position of the central point of each ear by taking the ear of the circuit component in the horizontal direction as a standard.
4. The method as claimed in claim 1, wherein the step of identifying the circuit components and the wires in step 3 comprises the following steps:
firstly, a coding rule needs to be set for all circuit components in advance, so that each circuit component has a unique code; because the number of the circuit components to be identified is limited, four easily-distinguished colors of red, yellow, blue and green are selected as the coding characteristic colors; the private printing of the circuit components is mainly concentrated in three areas, namely, an upper area, a middle area, a lower area and a middle area, and when the pixel value of a certain color of the three areas exceeds a prior threshold value, the color is considered to be the color of the area;
according to the shell color of the circuit components, the private printing colors of the upper middle part, the middle part and the lower middle part, red is marked as 1, yellow is marked as 2, blue is marked as 3, green is marked as 4, no marking is marked as 0, the current color is also marked as 0 after neglecting, 11 circuit components can be coded, and the type of the current detection circuit component can be uniquely determined according to the coding of the circuit components;
and filtering circuit components which do not accord with the codes, and transmitting the types of the remaining circuit components, the central points of the circuit components and the rotation angles to upper-layer software together.
5. The method as claimed in claim 2, wherein the step five of detecting the electric wires located in the effective identification area of the color image comprises the steps of:
step 1), segmenting wires and extracting bones through color difference, and the method comprises the following specific steps:
firstly, in the effective identification area HSV space in the third step, carrying out binarization processing on an image according to a prior threshold value of the color of the wire in the HSV space to obtain a wire binarization image;
further, scanning the wire binary image, filtering out contours which do not accord with the characteristics of the wire through the priori knowledge of the shape and the size of the contour of the wire, and filling the remaining contours to obtain a new wire-only binary image;
further, thinning the bones of the binary image only with the electric wires to obtain bones of the electric wires;
step 2), calculating the end point and the bifurcation point of the electric wire through the extracted bones, and the specific steps are as follows:
finding the end points and the branch points in the wire skeleton through the wire skeleton obtained in the step 1) and the prior knowledge of the end points and the branch point characteristics of the wire, if the branch points exist, separating the wire skeleton from the branch points to be divided into a plurality of line segments, if the branch points do not exist, the original wire skeleton is a line segment, and providing all the line segments and the types of the end points of the line segments in the skeleton to upper-layer software.
6. The method for identifying circuit components and wires in an educational toy set according to claim 4 or 5, wherein the step six of determining whether the connection between the circuit components and the wires is accurate comprises the specific steps of: and the upper software connects the circuit component identified in the step four with the electric wire identified in the step five, and then compares the electric wire with a circuit diagram stored by the upper software, and if the electric wire is consistent with the circuit diagram, the circuit connection is considered to be accurate.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610698013.XA CN106355592B (en) | 2016-08-19 | 2016-08-19 | Educational toy set, circuit element thereof and wire identification method |
| PCT/CN2016/105741 WO2018032631A1 (en) | 2016-08-19 | 2016-11-14 | Teaching toy kit and circuit element and electric wire identification method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610698013.XA CN106355592B (en) | 2016-08-19 | 2016-08-19 | Educational toy set, circuit element thereof and wire identification method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106355592A CN106355592A (en) | 2017-01-25 |
| CN106355592B true CN106355592B (en) | 2020-06-16 |
Family
ID=57845107
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610698013.XA Active CN106355592B (en) | 2016-08-19 | 2016-08-19 | Educational toy set, circuit element thereof and wire identification method |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN106355592B (en) |
| WO (1) | WO2018032631A1 (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106951836B (en) * | 2017-03-05 | 2019-12-13 | 北京工业大学 | crop coverage extraction method based on prior threshold optimization convolutional neural network |
| CN107590838B (en) * | 2017-08-18 | 2021-08-17 | 陕西维视智造科技股份有限公司 | Metal surface color visual detection system |
| CN107526888B (en) * | 2017-08-22 | 2024-02-20 | 珠海泓芯科技有限公司 | Method and device for generating circuit topology structure |
| CN108271765B (en) * | 2018-01-05 | 2021-01-12 | 湘潭大学 | Multifunctional claw head environment monitoring robot and plant identification method thereof |
| CN110263608B (en) * | 2019-01-25 | 2023-07-07 | 天津职业技术师范大学(中国职业培训指导教师进修中心) | Automatic electronic component identification method based on image feature space variable threshold measurement |
| CN114677586B (en) * | 2022-03-15 | 2024-04-05 | 南京邮电大学 | Automatic identification method for physical circuit experiment |
| CN114882520B (en) * | 2022-07-08 | 2022-11-11 | 成都西交智汇大数据科技有限公司 | Method, system and equipment for detecting circuit diagram and readable storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN2428833Y (en) * | 2000-06-12 | 2001-05-02 | 南京玻璃纤维研究设计院 | Display-screen picture calibration clamp |
| CN204655822U (en) * | 2015-02-27 | 2015-09-23 | 苏州雷泰医疗科技有限公司 | A kind of compound quality assurance die body |
| CN105852968A (en) * | 2016-04-06 | 2016-08-17 | 黄斌 | Trajectory tracking device for liver and gallbladder surgery |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7878891B2 (en) * | 2007-01-29 | 2011-02-01 | Fuji Xerox Co., Ltd. | Generating polyomino video game pieces and puzzle pieces from digital photos to create photominoes |
| US8340420B2 (en) * | 2009-10-05 | 2012-12-25 | National Taiwan University | Method for recognizing objects in images |
| CN102049137A (en) * | 2009-11-09 | 2011-05-11 | 季春香 | Combined-block type electronic element |
| CN104680519B (en) * | 2015-02-06 | 2017-06-23 | 四川长虹电器股份有限公司 | Seven-piece puzzle recognition methods based on profile and color |
| CN105194884A (en) * | 2015-10-27 | 2015-12-30 | 上海葡萄纬度科技有限公司 | Educational toy suite |
| CN205164140U (en) * | 2015-10-27 | 2016-04-20 | 上海葡萄纬度科技有限公司 | Education toy external member |
| CN105513086B (en) * | 2016-01-26 | 2019-02-19 | 上海葡萄纬度科技有限公司 | A kind of Educational toy external member and its localization method is detected based on the matched magic square of shape |
| CN105719307A (en) * | 2016-01-26 | 2016-06-29 | 上海葡萄纬度科技有限公司 | Educational toy set and method for detecting placing shape and position of tangram |
| CN105498200A (en) * | 2016-01-26 | 2016-04-20 | 上海葡萄纬度科技有限公司 | Educational toy set and tangram color recognition method thereof |
| CN105719318B (en) * | 2016-01-26 | 2018-07-10 | 上海葡萄纬度科技有限公司 | Magic square color identification method based on HSV in a kind of Educational toy external member |
-
2016
- 2016-08-19 CN CN201610698013.XA patent/CN106355592B/en active Active
- 2016-11-14 WO PCT/CN2016/105741 patent/WO2018032631A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN2428833Y (en) * | 2000-06-12 | 2001-05-02 | 南京玻璃纤维研究设计院 | Display-screen picture calibration clamp |
| CN204655822U (en) * | 2015-02-27 | 2015-09-23 | 苏州雷泰医疗科技有限公司 | A kind of compound quality assurance die body |
| CN105852968A (en) * | 2016-04-06 | 2016-08-17 | 黄斌 | Trajectory tracking device for liver and gallbladder surgery |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106355592A (en) | 2017-01-25 |
| WO2018032631A1 (en) | 2018-02-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106355592B (en) | Educational toy set, circuit element thereof and wire identification method | |
| CN105719318B (en) | Magic square color identification method based on HSV in a kind of Educational toy external member | |
| CN107766855B (en) | Chessman positioning method and system based on machine vision, storage medium and robot | |
| CN110232311B (en) | Method and device for segmenting hand image and computer equipment | |
| CN106097833B (en) | A kind of Educational toy external member and its digit recognition method | |
| US8126264B2 (en) | Device and method for identification of objects using color coding | |
| CN106297492B (en) | A kind of Educational toy external member and the method using color and outline identification programming module | |
| US20080101693A1 (en) | Video image based tracking system for identifying and tracking encoded color surface | |
| CN107154058B (en) | Method for guiding user to restore magic cube | |
| CN102521859B (en) | Reality augmenting method and device on basis of artificial targets | |
| WO2017128606A1 (en) | Educational toy kit and tangram color recognition method therefor | |
| CN103218605B (en) | A kind of fast human-eye positioning method based on integral projection and rim detection | |
| JP2017532076A (en) | Inspection system for appropriate oral hygiene procedures | |
| CN105498253B (en) | A kind of Educational toy external member and its positioning hole detection localization method | |
| CN105513086B (en) | A kind of Educational toy external member and its localization method is detected based on the matched magic square of shape | |
| CN107347152B (en) | The determination method, apparatus and electronic equipment at the grating inclination angle of naked eye 3D display equipment | |
| CN112883804B (en) | Error correction method and device for musical instrument playing hand actions and electronic equipment | |
| CN110032932B (en) | Human body posture identification method based on video processing and decision tree set threshold | |
| JP6779688B2 (en) | Image processing equipment, image processing method, computer program | |
| CN110569907B (en) | Method and device for identifying splicing pattern, computer storage medium and electronic equipment | |
| JP2007272435A (en) | Face feature extraction device and face feature extraction method | |
| CN106650628B (en) | Fingertip detection method based on three-dimensional K curvature | |
| WO2018032627A1 (en) | Teaching toy kit and marine mold identification method thereof | |
| CN108345867A (en) | Gesture identification method towards Intelligent household scene | |
| JP2017045331A (en) | Image processing method, image processor, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: Room 3001, building 22, No. 500 Jiajian Road, Jiading District, Shanghai Patentee after: Shanghai brocade Technology Group Co.,Ltd. Address before: 201822 room 3001, building 22, No. 500 Jiajian highway, Jiading District, Shanghai Patentee before: SHANGHAI PUTAO TECHNOLOGY Co.,Ltd. |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: An educational toy kit and its identification method for circuit components and wires Effective date of registration: 20230725 Granted publication date: 20200616 Pledgee: China Minsheng Banking Corp Shanghai branch Pledgor: Shanghai brocade Technology Group Co.,Ltd. Registration number: Y2023310000394 |