Disclosure of Invention
Based on the technical problem, the invention provides a system for determining the position of an entity pointer on the surface of an intelligent watch, for a mobile phone with low configuration, the method comprises the steps of identifying and positioning the current position of the pointer from an image by means of mobile phone video shooting, sending the current position and the angle to be adjusted to the watch through Bluetooth, and adjusting the watch to the correct position according to an instruction. The accuracy rate is extremely high according to the test at present. The user experience is improved due to the great convenience of the user.
The specific technical scheme of the invention is as follows:
a system for determining a smart watch surface entity pointer position, the system comprising a smart phone having an android system and a smart watch;
the intelligent watch comprises an MCU and a surface, wherein the surface is a circular surface, and red dots are arranged at the 12-point position of the circular surface;
the smart phone comprises a processor and a camera;
the camera can shoot the surface of the watch in real time to obtain a video; the camera may be focused at least once per time threshold;
the processor comprises an image processing unit, wherein the image processing unit can identify the video, and the identification is to extract the image of the video for image processing; the image processing comprises positioning preprocessing, wherein the positioning preprocessing comprises graying an image to enable the image to generate a 1-channel grayscale image from a 4-channel image, and performing Gaussian filtering processing on the grayscale image to obtain a smooth grayscale image; the processor can perform Hough transform image processing on the smooth gray level image to find out a circle in the smooth gray level image;
the processor presets a first diameter threshold value and a second diameter threshold value, and when the diameter of a circle in the smooth grayscale image is between the first diameter threshold value and the second diameter threshold value, the image processing unit takes the circle with the minimum diameter between the first diameter threshold value and the second diameter threshold value as an outline circle of the circular dial;
the processor can generate a positioning matrix of the outline circle according to the outline circle, cut the extracted image of the video by using the positioning matrix, reserve an image of the video in the positioning matrix of the outline circle, the reserved image is called a positioning image, the center of the positioning image is the center of the outline circle, and the radius of the positioning image is r;
the processor presets a red dot threshold value X, the image processing unit can scan the positioning image to obtain RGB values of each dot of the positioning image, if (R-B) > X and (R-G) > X, the processor judges that the dot is a preset red dot, and a connecting line of the red dot and the circle center is marked as L;
the image processing unit can perform marginalization processing on the positioning image, so that only lines are left in the positioning image; the processor can perform interference elimination processing on the remaining lines, specifically:
calculating the distance d1 from the start point of each line to the center of the positioning image and the distance d2 from the end point to the center of the positioning image, wherein d1 is less than d2, if d1 of the line is less than r/5 and d2 is less than 0.9r, the processor judges that the line is an effective line, otherwise, the line is an interference line; generating a set of effective lines by all the effective lines;
the processor may calculate an included angle between each line in the set and the L, and then classify the included angle, the classification being:
AN included angle between the first line and the L is A1, AN included angle between the second line and the L is A2, AN included angle between the third line and the L is A3, and AN included angle between the N-th line of … … and the L is AN;
if a line segments fall within the angle range of (Ap, Ap + 3), b line segments fall within the angle range of (Aq, Aq + 3), c line segments fall within the angle range of (As, As + 3), and … … h line segments fall within the angle range of (At, At + 3), wherein a + b + c + … … + h = N, Ap, Aq, As, … …, At are all values in a1, a2, A3, … …, AN;
when three hands including an hour hand, a minute hand and a second hand are available, taking three numerical values of a, b, c, … … and h with the largest numerical values, and positioning all lines in corresponding angle ranges of the three numerical values as hand lines;
the processor may perform the following pointer judgment calculation on the pointer line:
if the distance between one point and the circle center in one pointer line is larger than 0.7r and smaller than 0.75r, the line is judged to be a minute pointer line; if the distance between one point and the circle center in one pointer line is larger than 0.3r and smaller than 0.35r, the line is judged to be the pointer line; if the distance between one point and the circle center in one pointer line is larger than 0.8r and smaller than 0.9r, the line is judged to be a second pointer line;
the processor can perform the hand judgment calculation on the extracted video images of the consecutive times of threshold times to form a second hand line set, a minute hand line set and an hour hand line set;
the processor can filter out angles with larger errors in any one of the line sets of the second hand, the minute hand and the hour hand, and the filtering method comprises the following steps:
an included angle between the 1 st line and the L is B1, an included angle between the 2 nd line and the L is B2, an included angle between the 3 rd line and the L is B3, and an included angle between the … … th line and the L is Bn;
if A line segments fall within the angle range of (Bp, Bp + 3), B line segments fall within the angle range of (Bq, Bq + 3), C line segments fall within the angle range of (Bs, Bs + 3), and … … H line segments fall within the angle range of (Bt, Bt + 3), wherein A + B + C + … … + H = n, and Bp, Bq, Bs, … … and Bt are all values in B1, B2, B3, … … and Bn; and taking a value with the largest value in A, B, C, … … and H, positioning all lines in the corresponding angle range of the value as effective lines of the pointer, and calculating the angle average value of all effective lines in the line set as the angle of the pointer to the L.
Further, the time threshold is 1 second.
Further, the processor extracts the image of the video according to a rule, specifically: and numbering the image currently being processed as 0, and extracting each frame of image of the video, or extracting the image with odd number or even number, or extracting the image with the number being multiple of 3, or extracting the image with the number being multiple of 4.
Further, the extraction is performed according to the processing speed of the processor, if the current image is not processed, the sent image is not extracted for identification, and the first frame image is sent to the processor after the current image is processed and then extracted.
Further, the value of X is 50 or 80 or 90. X represents a red point threshold value, the range is 50-90, the red range included by 50 is wide, 90 is strict red, and due to the reason of some mobile phone cameras, the red point may become dark red or other colors close to red, so that X can take 50 or other strict values.
Further, the number threshold is equal to or greater than 7.
Further, the processor calculates the surface time of the intelligent watch according to the angles from the three pointers to the L, and compares the surface time with the actual time to obtain a time difference.
Further, the processor sends the time difference to the smart watch, and the MCU of the smart watch controls the three pointers to adjust the time according to the time difference.
Through the technical scheme, firstly, the position of the pointer of the watch on the surface can be accurately calculated in real time, secondly, the requirement on the android mobile phone processor of the mobile phone is not high, the APP of the mobile phone cannot be changed, and the low-end mobile phone can be very fluent in representation even if the image cannot be rapidly processed by adopting a selective image frame neglecting mode.
Detailed Description
The technical solution of the present invention is explained in detail below.
The invention aims to solve the algorithm problems of image recognition, noise coming and time calibration and the optimization problem of different requirements of the mobile phone processor.
The extraction of the surface pointer has the image extraction, however, due to the problems of light and color in the image extraction, a single picture is difficult to accurately determine the position of the pointer, and the pointer has a very high possibility to move due to the problem of time intervals in the shooting of multiple pictures.
The invention adopts a mode of extracting pictures from the video, the video has continuity, and a plurality of frames of pictures can be extracted from the video within 1 second, and the position of the pointer is basically unchanged when a plurality of frames are processed.
The specific implementation mode is as follows:
a system for determining a smart watch surface entity pointer position, the system comprising a smart phone having an android system and a smart watch;
the intelligent watch comprises an MCU and a surface, wherein the surface is a circular surface, and red dots are arranged at the 12-point position of the circular surface;
the smart phone comprises a processor and a camera;
the camera can shoot the surface of the watch in real time to obtain a video; the camera may be focused at least once per time threshold;
the processor comprises an image processing unit, wherein the image processing unit can identify the video, and the identification is to extract the image of the video for image processing; the image processing comprises positioning preprocessing, wherein the positioning preprocessing comprises graying an image to enable the image to generate a 1-channel grayscale image from a 4-channel image, and performing Gaussian filtering processing on the grayscale image to obtain a smooth grayscale image; the processor can perform Hough transform image processing on the smooth gray level image to find out a circle in the smooth gray level image;
the processor presets a first diameter threshold value and a second diameter threshold value, and when the diameter of a circle in the smooth grayscale image is between the first diameter threshold value and the second diameter threshold value, the image processing unit takes the circle with the minimum diameter between the first diameter threshold value and the second diameter threshold value as an outline circle of the circular dial;
the processor can generate a positioning matrix of the outline circle according to the outline circle, cut the extracted image of the video by using the positioning matrix, reserve an image of the video in the positioning matrix of the outline circle, the reserved image is called a positioning image, the center of the positioning image is the center of the outline circle, and the radius of the positioning image is r;
the processor presets a red dot threshold value X, the image processing unit can scan the positioning image to obtain RGB values of each dot of the positioning image, if (R-B) > X and (R-G) > X, the processor judges that the dot is a preset red dot, and a connecting line of the red dot and the circle center is marked as L;
the image processing unit can perform marginalization processing on the positioning image, so that only lines are left in the positioning image; the processor can perform interference elimination processing on the remaining lines, specifically:
calculating the distance d1 from the start point of each line to the center of the positioning image and the distance d2 from the end point to the center of the positioning image, wherein d1 is less than d2, if d1 of the line is less than r/5 and d2 is less than 0.9r, the processor judges that the line is an effective line, otherwise, the line is an interference line; generating a set of effective lines by all the effective lines;
the processor may calculate an included angle between each line in the set and the L, and then classify the included angle, the classification being:
AN included angle between the first line and the L is A1, AN included angle between the second line and the L is A2, AN included angle between the third line and the L is A3, and AN included angle between the N-th line of … … and the L is AN;
if a line segments fall within the angle range of (Ap, Ap + 3), b line segments fall within the angle range of (Aq, Aq + 3), c line segments fall within the angle range of (As, As + 3), and … … h line segments fall within the angle range of (At, At + 3), wherein a + b + c + … … + h = N, Ap, Aq, As, … …, At are all values in a1, a2, A3, … …, AN;
when three hands including an hour hand, a minute hand and a second hand are available, taking three numerical values of a, b, c, … … and h with the largest numerical values, and positioning all lines in corresponding angle ranges of the three numerical values as hand lines;
the processor may perform the following pointer judgment calculation on the pointer line:
if the distance between one point and the circle center in one pointer line is larger than 0.7r and smaller than 0.75r, the line is judged to be a minute pointer line; if the distance between one point and the circle center in one pointer line is larger than 0.3r and smaller than 0.35r, the line is judged to be the pointer line; if the distance between one point and the circle center in one pointer line is larger than 0.8r and smaller than 0.9r, the line is judged to be a second pointer line;
the processor can perform the hand judgment calculation on the extracted video images of the consecutive times of threshold times to form a second hand line set, a minute hand line set and an hour hand line set;
the processor can filter out angles with larger errors in any one of the line sets of the second hand, the minute hand and the hour hand, and the filtering method comprises the following steps:
an included angle between the 1 st line and the L is B1, an included angle between the 2 nd line and the L is B2, an included angle between the 3 rd line and the L is B3, and an included angle between the … … th line and the L is Bn;
if A line segments fall within the angle range of (Bp, Bp + 3), B line segments fall within the angle range of (Bq, Bq + 3), C line segments fall within the angle range of (Bs, Bs + 3), and … … H line segments fall within the angle range of (Bt, Bt + 3), wherein A + B + C + … … + H = n, and Bp, Bq, Bs, … … and Bt are all values in B1, B2, B3, … … and Bn; and taking a value with the largest value in A, B, C, … … and H, positioning all lines in the corresponding angle range of the value as effective lines of the pointer, and calculating the angle average value of all effective lines in the line set as the angle of the pointer to the L.
Further, the time threshold is 1 second.
Further, the processor extracts the image of the video according to a rule, specifically: and numbering the image currently being processed as 0, and extracting each frame of image of the video, or extracting the image with odd number or even number, or extracting the image with the number being multiple of 3, or extracting the image with the number being multiple of 4.
Further, the extraction is performed according to the processing speed of the processor, if the current image is not processed, the sent image is not extracted for identification, and the first frame image is sent to the processor after the current image is processed and then extracted.
Further, the value of X is 50 or 80 or 90. X represents a red point threshold value, the range is 50-90, the red range included by 50 is wide, 90 is strict red, and due to the reason of some mobile phone cameras, the red point may become dark red or other colors close to red, so that X can take 50 or other strict values.
Further, the number threshold is equal to or greater than 7.
Further, the processor calculates the surface time of the intelligent watch according to the angles from the three pointers to the L, and compares the surface time with the actual time to obtain a time difference.
Further, the processor sends the time difference to the smart watch, and the MCU of the smart watch controls the three pointers to adjust the time according to the time difference.
The method for determining the pointer by the system is as follows:
step one, setting a camera of a mobile phone with an installation system and a processor, so that the camera focuses at least once in a time threshold. Therefore, even if the camera is poor and the configuration is low, at least one frame in every several frames of images of the android phone is a clear image.
Further, in the step one, the time threshold is 1 second.
And step two, the camera shoots the surface of the watch in real time to obtain a video, and the mobile phone identifies the video shot by the camera, wherein the identification is to extract an image of the video for identification.
Further, in the second step, the processor extracts the image of the video according to a rule, specifically: and numbering the image currently being processed as 0, and extracting each frame of image of the video, or extracting the image with odd number or even number, or extracting the image with the number being multiple of 3, or extracting the image with the number being multiple of 4.
Further, in the second step, the extraction is performed according to the processing speed of the processor, and if the current image is not processed, the sent image is skipped over and is not extracted for recognition until the current image is processed and then the first frame image is sent to the processor after the extraction processing is finished.
And thirdly, performing positioning pretreatment on the extracted video image, wherein the positioning pretreatment comprises graying the image to enable the image to generate a 1-channel grayscale image from a 4-channel image, and performing Gaussian filtering treatment on the grayscale image to obtain a smooth grayscale image. When the calculation speed required by the processor is evaluated for multiple times, factors such as light, color and the like are found to have great obstacles to the subsequent extraction of the pointer, so that the conventional 4-channel image is creatively and directly reduced to the 1-channel gray image for processing, the processing speed is high, and the requirement on the performance of the mobile phone is reduced.
And fourthly, the dial plate of the intelligent watch is a round dial plate, and the processor performs Hough transform image processing on the smooth gray level image to find out the round shape in the smooth gray level image.
And fifthly, presetting a first diameter threshold value and a second diameter threshold value by the processor, and taking the circle with the minimum diameter between the first diameter threshold value and the second diameter threshold value as the outline circle of the circular dial plate when the diameter of the circle in the fourth step is between the first diameter threshold value and the second diameter threshold value.
And step six, the processor generates a positioning matrix of the outline circle according to the outline circle, cuts the image of the video extracted in the step two by using the positioning matrix, reserves the image of the video in the positioning matrix of the outline circle, the reserved image is called a positioning image, the circle center of the positioning image is the circle center of the outline circle, and the radius of the positioning image is r.
Step seven, presetting a red point threshold value X by the processor; presetting a red dot at the position of a 12 point of a dial plate of the watch, scanning the positioning image by the processor to obtain the RGB value of each point of the positioning image, if (R-B) > X and (R-G) > X, judging that the point is the preset red dot by the processor, and marking the connecting line of the red dot and the circle center as L.
Further, in the seventh step, the value of X is 50 or 80 or 90. X represents a red point threshold value, the range is 50-90, the red range included by 50 is wide, 90 is strict red, and due to the reason of some mobile phone cameras, the red point may become dark red or other colors close to red, so that X can take 50 or other strict values.
And step eight, performing image marginalization processing on the positioning image to enable the positioning image to only have lines. Since all the hands are straight, only the line needs to be reserved for pertinence.
And step nine, performing interference removal processing on the lines in the step seven, specifically:
calculating the distance d1 from the start point of each line to the center of the positioning image and the distance d2 from the end point to the center of the positioning image, wherein d1 is less than d2, if d1 of the line is less than r/5 and d2 is less than 0.9r, the processor judges that the line is an effective line, otherwise, the line is an interference line; a set of valid lines is generated.
Step ten, calculating an included angle between each line in the set and the L, and then classifying the included angles, wherein the classification is as follows:
AN included angle between the first line and the L is A1, AN included angle between the second line and the L is A2, AN included angle between the third line and the L is A3, and AN included angle between the N-th line of … … and the L is AN;
if a line segments fall within the angle range of (Ap, Ap + 3), b line segments fall within the angle range of (Aq, Aq + 3), c line segments fall within the angle range of (As, As + 3), and … … h line segments fall within the angle range of (At, At + 3), wherein a + b + c + … … + h = N, Ap, Aq, As, … …, At are all values in a1, a2, A3, … …, AN;
when there are three hands of the hour hand, minute hand and second hand, the three numerical values of a, b, c, … …, h with the largest numerical value are taken, and all lines positioned to the corresponding angular ranges of the three numerical values are taken as the hand lines.
Step eleven, calculating the pointer line: if the distance between one point and the circle center in one pointer line is larger than 0.7r and smaller than 0.75r, the line is judged to be a minute pointer line; if the distance between one point and the circle center in one pointer line is larger than 0.3r and smaller than 0.35r, the line is judged to be the pointer line; and if the distance between one point and the circle center in one pointer line is more than 0.8r and less than 0.9r, judging the line as a second hand line.
Step twelve, in the step two, extracting the next frame image, and repeating the step three to the step eleven of the number threshold. The number threshold is 7 or more.
And thirteen, forming a line set of a second hand, a line set of a minute hand and a line set of an hour hand, and filtering out angles with larger errors, wherein in the line set of any hand, the filtering method comprises the following steps:
an included angle between the 1 st line and the L is B1, an included angle between the 2 nd line and the L is B2, an included angle between the 3 rd line and the L is B3, and an included angle between the … … th line and the L is Bn;
if A line segments fall within the angle range of (Bp, Bp + 3), B line segments fall within the angle range of (Bq, Bq + 3), C line segments fall within the angle range of (Bs, Bs + 3), and … … H line segments fall within the angle range of (Bt, Bt + 3), wherein A + B + C + … … + H = n, and Bp, Bq, Bs, … … and Bt are all values in B1, B2, B3, … … and Bn; and taking a value with the largest value in A, B, C, … … and H, positioning all lines in the corresponding angle range of the value as effective lines of the pointer, and calculating the angle average value of all effective lines in the line set as the angle of the pointer to the L.
And the processor calculates the surface time of the intelligent watch according to the angles from the three pointers to the L obtained in the step thirteen, and compares the surface time with the actual time to obtain a time difference. And the processor sends the time difference to the intelligent watch, and the intelligent watch controls the three pointers to adjust the time according to the time difference.
Through the technical scheme, firstly, the position of the pointer of the watch on the surface can be accurately calculated in real time, secondly, the requirement on the android mobile phone processor of the mobile phone is not high, the APP of the mobile phone cannot be changed, and the low-end mobile phone can be very fluent in representation even if the image cannot be rapidly processed by adopting a selective image frame neglecting mode.