CN110633027A - Point reading implementation method, system, computer equipment and storage medium - Google Patents
Point reading implementation method, system, computer equipment and storage medium Download PDFInfo
- Publication number
- CN110633027A CN110633027A CN201910870427.XA CN201910870427A CN110633027A CN 110633027 A CN110633027 A CN 110633027A CN 201910870427 A CN201910870427 A CN 201910870427A CN 110633027 A CN110633027 A CN 110633027A
- Authority
- CN
- China
- Prior art keywords
- target shape
- image data
- finger
- characteristic region
- shape characteristic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000003860 storage Methods 0.000 title claims abstract description 11
- 238000009826 distribution Methods 0.000 claims abstract description 53
- 238000001514 detection method Methods 0.000 claims description 32
- 238000012549 training Methods 0.000 claims description 32
- 238000004422 calculation algorithm Methods 0.000 claims description 28
- 230000001537 neural effect Effects 0.000 claims description 22
- 238000012545 processing Methods 0.000 claims description 20
- 238000010586 diagram Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 5
- 238000004891 communication Methods 0.000 description 16
- 230000009466 transformation Effects 0.000 description 12
- 238000005516 engineering process Methods 0.000 description 10
- 238000012216 screening Methods 0.000 description 10
- 238000006243 chemical reaction Methods 0.000 description 8
- 238000013502 data validation Methods 0.000 description 8
- 241001465382 Physalis alkekengi Species 0.000 description 6
- 238000003062 neural network model Methods 0.000 description 6
- 238000012795 verification Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 230000003321 amplification Effects 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 4
- 238000013524 data verification Methods 0.000 description 4
- 238000001914 filtration Methods 0.000 description 4
- 238000003709 image segmentation Methods 0.000 description 4
- 238000003199 nucleic acid amplification method Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 235000002566 Capsicum Nutrition 0.000 description 2
- 239000006002 Pepper Substances 0.000 description 2
- 241000722363 Piper Species 0.000 description 2
- 235000016761 Piper aduncum Nutrition 0.000 description 2
- 235000017804 Piper guineense Nutrition 0.000 description 2
- 235000008184 Piper nigrum Nutrition 0.000 description 2
- 238000005520 cutting process Methods 0.000 description 2
- 238000005034 decoration Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003825 pressing Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 150000003839 salts Chemical class 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000010267 cellular communication Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/042—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
- G06F3/0425—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/107—Static hand or arm
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention provides a point reading implementation method, a point reading implementation system, computer equipment and a storage medium, wherein the method comprises the following steps: acquiring infrared image data; the infrared image data comprises a book area pointed by a user; carrying out image recognition on the infrared image data to obtain a target shape characteristic region; and acquiring a temperature distribution map of the target shape characteristic region, and responding to the content of the target shape characteristic region if the temperature of the target shape characteristic region is within a preset temperature range. The invention reduces the error recognition and improves the accuracy of finger point reading, thereby improving the use experience of users.
Description
Technical Field
The present invention relates to the field of point reading machine technology, and in particular, to a point reading implementation method, system, computer device, and storage medium.
Background
The current point reading function in the market is to place fingers on books, and recognize the positions pointed by the fingers of a user through an image recognition technology to realize coordinate positioning, so that the function of point reading of the fingers is realized.
However, during the process of reading with fingers, a user may have many illustrations in the book, and the color and shape of some of the illustrations are often very close to the fingers of the human body, for example, the illustrations are long sticks in brown. Due to the color and shape of many articles in life, the articles tend to be very close to the fingers of human bodies, such as long-bar articles like pencils and pens held in the hands of users. Therefore, the finger identification accuracy in the finger touch reading process can be greatly influenced, the touch reading intention of the user is further identified by mistake, and the finger touch reading experience of the user is influenced.
Therefore, how to reduce the misrecognition and improve the accuracy of finger point reading so as to improve the use experience of the user is an urgent problem to be solved by the invention.
Disclosure of Invention
The invention aims to provide a point reading implementation method, a point reading implementation system, computer equipment and a storage medium, so that misrecognition is reduced, the accuracy of finger point reading is improved, and the use experience of a user is improved.
The technical scheme provided by the invention is as follows:
the invention provides a point reading implementation method, which comprises the following steps:
acquiring infrared image data; the infrared image data comprises a book area pointed by a user;
carrying out image recognition on the infrared image data to obtain a target shape characteristic region;
and acquiring a temperature distribution map of the target shape characteristic region, and responding to the content pointed by the target shape characteristic region if the temperature of the target shape characteristic region is within a preset temperature range.
Further, the step of acquiring infrared image data comprises the steps of:
obtaining a sample image dataset;
training according to the sample image data set to obtain a finger recognition neural model;
the step of carrying out image recognition on the infrared image data to obtain a target shape characteristic region comprises the following steps:
and inputting the infrared image data into the finger recognition neural model, and recognizing all target shape characteristic areas which accord with the finger shape in the infrared image data through a preset target detection algorithm if the infrared image data accord with the finger shape.
Further, the obtaining of the temperature distribution map of the target shape feature region, and if the temperature of the target shape feature region falls within a preset temperature range, responding to the content of the target shape feature region includes:
segmenting the infrared image data to obtain a target shape characteristic region image corresponding to each target shape characteristic region;
carrying out gray level processing on the images of the characteristic regions of each target shape, converting the gray level values into temperature values, and obtaining a temperature distribution map corresponding to each characteristic region of the target shape;
judging whether the temperature of the current target shape characteristic region is within a preset temperature range or not according to the temperature distribution map corresponding to the current target shape characteristic region;
if the temperature of the current target shape characteristic region is within the preset temperature range, responding to the content of the target shape characteristic region;
and if the temperature of the current target shape characteristic region is out of the preset temperature range, not responding to the content of the target shape characteristic region.
Further, the acquiring infrared image data includes the steps of:
acquiring input information of a user, and triggering an infrared camera to shoot towards a book area pointed by the user to acquire infrared image data if the input information meets a preset triggering condition;
and if the operation instruction accords with a preset trigger instruction, triggering an infrared camera to shoot towards the book area pointed by the user to acquire the infrared image data.
The invention also provides a point reading implementation system, which comprises:
the acquisition module is used for acquiring infrared image data; the infrared image data comprises a book area pointed by a user;
the image recognition module is used for carrying out image recognition on the infrared image data to obtain a target shape characteristic region;
and the processing module is used for acquiring the temperature distribution map of the target shape characteristic region, and responding to the content pointed by the target shape characteristic region if the temperature of the target shape characteristic region is within a preset temperature range.
Further, the method also comprises the following steps:
a collection module for obtaining a sample image dataset;
the training module is used for training according to the sample image data set to obtain a finger recognition neural model;
the image recognition module is further configured to input the infrared image data to the finger recognition neural model, and if the infrared image data conforms to the shape of the finger, recognize all target shape feature areas conforming to the shape of the finger in the infrared image data through a preset target detection algorithm.
Further, the processing module comprises:
the extraction unit is used for segmenting the infrared image data to obtain a target shape characteristic region image corresponding to each target shape characteristic region;
the processing unit is used for carrying out gray processing on the images of the characteristic regions of the target shapes and converting the gray values into temperature values to obtain temperature distribution maps corresponding to the characteristic regions of the target shapes;
the judging unit is used for judging whether the temperature of the current target shape characteristic region is within a preset temperature range according to the temperature distribution map corresponding to the current target shape characteristic region;
the control unit is used for responding to the content of the target shape characteristic region if the temperature of the current target shape characteristic region is within the preset temperature range; if the temperature of the current target shape characteristic region is out of the preset temperature range, not responding to the content of the target shape characteristic region;
and the control unit is also used for switching to the temperature distribution diagram corresponding to the next target shape characteristic region to continue judging until the temperature distribution diagrams corresponding to all the target shape characteristic regions are judged.
Further, the method also comprises the following steps:
the input module is used for acquiring input information of a user;
and the control module is used for triggering the infrared camera to shoot towards the book area pointed by the user to acquire the infrared image data if the input information meets the preset triggering condition.
The invention also provides computer equipment, which comprises a processor and a memory, wherein the memory is used for storing the computer program; the processor is configured to execute the computer program stored in the memory, and implement the operations performed by the click-to-read implementation method.
The invention also provides a storage medium, which stores at least one instruction, and the instruction is loaded and executed by a processor to implement the operation performed by the point-reading implementation method.
Through the click-to-read implementation method, the click-to-read implementation system, the computer equipment and the storage medium, misrecognition can be reduced, the accuracy of finger click-to-read is improved, and therefore the use experience of a user is improved.
Drawings
The above features, technical features, advantages and implementations of a method, system, computer device, and storage medium for read-on-demand are further described in the following detailed description of preferred embodiments in conjunction with the accompanying drawings.
FIG. 1 is a flow chart of one embodiment of a method for implementing a point-and-read implementation of the present invention;
FIG. 2 is a flow chart of another embodiment of a method for implementing point-and-read of the present invention;
FIG. 3 is a flow chart of another embodiment of a method for implementing a point read of the present invention;
FIG. 4 is a flow chart of another embodiment of a method for implementing a point read of the present invention;
FIG. 5 is a schematic structural diagram of an embodiment of a read-on-demand implementation system of the present invention;
FIG. 6 is a schematic structural diagram of another embodiment of a read-on-demand system of the present invention;
FIG. 7 is a schematic structural diagram of another embodiment of a read-on-demand system of the present invention;
FIG. 8 is a schematic diagram of a computer device according to an embodiment of the present invention.
Detailed Description
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following description will be made with reference to the accompanying drawings. It is obvious that the drawings in the following description are only some examples of the invention, and that for a person skilled in the art, other drawings and embodiments can be derived from them without inventive effort.
For the sake of simplicity, the drawings only schematically show the parts relevant to the present invention, and they do not represent the actual structure as a product. In addition, in order to make the drawings concise and understandable, components having the same structure or function in some of the drawings are only schematically illustrated or only labeled. In this document, "one" means not only "only one" but also a case of "more than one".
An embodiment of the present invention, as shown in fig. 1, is a method for implementing point reading, including:
s100, acquiring infrared image data; the infrared image data comprises a book area pointed by a user;
specifically, a user can place a book in a reading area of the reading machine, the user points to a target to be read by using a finger, and then the infrared camera arranged on the reading machine shoots the book area pointed by the user to obtain infrared image data. Certainly, the user can place books in intelligent lamps and lanterns (including intelligent desk lamp, intelligent wall lamp etc.) below, is provided with infrared camera on the intelligent lamps and lanterns, and the user uses the finger to point to the self required object of reading, then shoots the books region that the user points through the infrared camera that sets up on the intelligent lamps and lanterns and obtain infrared image data. Certainly, the user can place books in infrared camera below, and the user uses the finger to point to the object that needs to click to read by oneself, then shoots the books region that the user points through infrared camera and obtain infrared image data.
The object to be read can be a word or a sentence, and the language of the word or the sentence can be any language form such as Chinese or English.
Of course, this is merely an example, and the user may adjust the orientation of the infrared camera toward the book area pointed by the user through any intelligent hardware (including but not limited to a point-reading machine, a computer, a tablet, and an intelligent lamp) provided with the infrared camera, so as to capture infrared image data. Certainly, the user can also directly shoot through infrared camera and obtain infrared image data, then, upload infrared image data to the server or be provided with the intelligent hardware of infrared camera not.
S200, carrying out image recognition on the infrared image data to obtain a target shape characteristic region;
specifically, the acquired infrared image data may include a finger pointed by the user to point the reading object, or a long strip-shaped article (including but not limited to a pencil or a pen) similar to the shape of the finger, or a long strip-shaped pattern or a picture inserted in a book pointed by the user and similar to the shape of the finger.
Since the acquired infrared image data may have many interference contents, such as a strip pattern or illustration similar to the shape of a finger, a strip article similar to the shape of a finger, and the finger of the user is also strip-shaped, the interference contents and the shape of the finger of the user are extremely similar, and the target shape is strip-shaped. Therefore, no matter what the infrared image data includes, image recognition can be performed on the infrared image data by using any image recognition technology in the prior art, so as to recognize the target shape feature region in the infrared image data, that is, recognize the strip-shaped feature region in the infrared image data.
S300, acquiring a temperature distribution map of the target shape characteristic region, and responding to the content of the target shape characteristic region if the temperature of the target shape characteristic region is within a preset temperature range.
Specifically, a temperature distribution map of a target shape feature region, that is, a long-strip-shaped feature region, is obtained through an existing temperature distribution map extraction technology, because any natural object radiates infrared rays to the surroundings, and the heat values of the infrared rays radiated by different objects are different, the temperature range of a user's finger is set to be a preset temperature range, and then after the temperature distribution map of the target shape feature region is obtained, it is determined that the temperature of the target shape feature region belongs to the preset temperature range, where the temperature may be the temperature of all pixels or the average value of the temperatures of all pixels. Once the temperature of the target shape characteristic region is within the preset temperature range, it is indicated that the target shape characteristic region corresponds to a human finger in the real world, and therefore, the content pointed by the target shape characteristic region, that is, the content pointed by the user finger, is responded, and accurate finger touch reading is achieved.
In this embodiment, infrared image data is acquired, a target shape feature region in the infrared image data is identified, a temperature distribution map is extracted for the target shape feature region, and only when the temperature of the target shape feature region falls within a preset temperature range, the content pointed by the target shape feature region is responded, so that, in combination with image identification and temperature detection, the remaining target shape feature regions except for the fingers of the user can be filtered, i.e., patterns similar to the shapes of the fingers and non-finger objects similar to the shapes of the fingers, such as pens, pencils and the like, can be filtered, therefore, misrecognition can be reduced, the accuracy of finger point reading is improved, and the use experience of the user is improved.
An embodiment of the present invention, as shown in fig. 2, is a method for implementing point reading, including:
s010 obtains a sample image dataset;
s020 training according to the sample image data set to obtain a finger recognition neural model;
in particular, the sample image dataset includes a finger sample image dataset and a finger-like sample image dataset. The finger sample image includes human finger portion content and does not include finger-like shaped interference content.
The finger sample image dataset is acquired in a manner comprising: the method comprises the steps of downloading a large amount of image data from a network, screening finger sample images which comprise human finger part contents and do not comprise interference contents similar to finger shapes from the image data, and screening a large amount of finger sample images to obtain a finger sample image data set. And secondly, acquiring a small amount of finger sample images which comprise the content of the finger part of the human body and do not comprise interference content similar to the shape of the finger, and then performing data amplification by taking the small amount of finger sample images as template images, namely performing pixel transformation and/or geometric transformation on the small amount of finger sample images respectively to obtain a large amount of finger sample images so as to obtain a finger sample image data set.
The acquisition mode of the finger-like sample image dataset comprises the following steps: firstly, a large amount of image data are downloaded from a network, finger-like sample images which do not include human finger part contents and include interference contents similar to finger shapes are screened out from the image data, and a large amount of finger-like sample images are screened out, so that a finger-like sample image data set is obtained. And secondly, acquiring a small amount of finger-like sample images which do not include the content of the human finger part and include interference content similar to the shape of the finger, and then performing data amplification by taking the small amount of finger-like sample images as template images, namely performing pixel transformation and/or geometric transformation on the small amount of finger-like sample images respectively to obtain a large amount of finger-like sample images so as to obtain a finger-like sample image dataset.
Wherein the pixel transformation comprises: 1. adding noise and filtering, wherein the noise is obtained by a mode including but not limited to salt and pepper noise, Gaussian noise and median filtering; 2. changing channels and adjusting the sequence of the three RBG channels; 3. adjusting contrast, brightness and saturation, and color dithering.
The geometric transformation includes: 1. turning over, for example: the face turning device is turned horizontally and vertically according to actual conditions, for example, regarding the face, the face turned up and down is changed into a face turned over, and the turning has no practical significance; 2. translating to simulate the situation that the picture in real life is not centered, and changing the position; 3. rotating; 4. black setting is carried out, and data samples which are partially shielded are simulated; 5. cutting; 6. and (4) zooming.
After the sample image data set is obtained in the above manner, if the sample image data set only includes the finger sample image data set, the finger sample image data set can be divided into a finger sample image data training set and a finger sample image data verification set, training is performed according to the finger sample images in the finger sample image data training set, verification is performed according to the finger sample images in the finger sample image data verification set, model parameters are adjusted according to verification results, verification is performed through multiple times of training, and the model parameters are adjusted, so that the finger recognition neural model is obtained.
Of course, after the sample image dataset is obtained in the above manner, if the sample image dataset includes a finger sample image dataset and a finger-like sample image dataset, the finger sample image dataset may be divided into a finger sample image data training set and a finger sample image data validation set, the finger-like sample image dataset may be divided into a finger-like sample image data training set and a finger-like sample image data validation set, a finger sample image in the finger sample image data training set and a finger-like sample image in the finger-like sample image data training set are trained according to the finger sample image in the finger sample image data training set, a finger sample image in the finger-like sample image data validation set and a finger-like sample image in the finger-like sample image data validation set are validated according to the finger sample image in the finger sample image data training set, a model parameter is adjusted according to the validation result, a model parameter is trained and, thereby obtaining the finger recognition neural model. Compared with the method for training the finger recognition neural network model only according to the finger sample image data set, the method for training the finger recognition neural network model through the positive and negative samples (namely the finger sample image and the finger-like sample image) can improve the accuracy and reliability of finger recognition of the trained finger recognition neural model.
S100, acquiring infrared image data; the infrared image data comprises a book area pointed by a user;
s210, inputting infrared image data into a finger recognition neural model, and recognizing all target shape characteristic areas which accord with the finger shape in the infrared image data through a preset target detection algorithm if the infrared image data accord with the finger shape;
specifically, the acquired infrared image data is input into a trained finger recognition neural model, and through the trained finger recognition neural model, whether the acquired infrared image data includes a target shape characteristic region conforming to the shape of the finger is judged first, and if not, the subsequent steps are not performed. And if the target shape characteristic area is included, performing target detection on the infrared image data through a preset target detection algorithm, so as to identify all target shape characteristic areas which accord with the finger shape in the infrared image data. The preset target detection algorithm includes a YOLO target detection algorithm, a YOLO2 target detection algorithm, an R-CNN target detection algorithm, a Fast R-CNN target detection algorithm, an FPN target detection algorithm, an SSD target detection algorithm, a RetinaNet target detection algorithm, and the like.
S300, acquiring a temperature distribution map of the target shape characteristic region, and responding to the content of the target shape characteristic region if the temperature of the target shape characteristic region is within a preset temperature range.
The same parts in this embodiment as those in the above embodiments are not described in detail herein. Through this embodiment, use finger identification neural network model to carry out the infrared image data that preliminary screening accords with the finger shape, to the infrared image data that accords with the finger shape after passing preliminary screening again, use and predetermine the target detection algorithm and discern wherein all target shape characteristic regions that accord with the finger shape, can promote all the identification accuracy who accords with the target shape characteristic region of finger shape, greatly reduced the error probability of discernment. And the calculation amount and the resource occupation can be reduced through preliminary screening, and the recognition efficiency of the target shape characteristic region conforming to the finger shape can be improved.
An embodiment of the present invention, as shown in fig. 3, is a method for implementing point reading, including:
s100, acquiring infrared image data; the infrared image data comprises a book area pointed by a user;
s200, carrying out image recognition on the infrared image data to obtain a target shape characteristic region;
s310, segmenting the infrared image data to obtain a target shape characteristic region image corresponding to each target shape characteristic region;
specifically, according to the embodiment, after the target shape feature region is identified by using the preset target detection algorithm, all the target shape feature regions in the infrared image data are segmented by using the image segmentation technology, so as to obtain the target shape feature region image corresponding to each target shape feature region. Illustratively, the infrared image data includes a user finger and a pencil, and the infrared image data is divided into a finger area image and a pencil area image. Of course, this is merely an example, and other situations are within the scope of the present invention, and will not be described in detail herein.
S320, carrying out gray level processing on the images of the characteristic regions of the target shape, converting the gray level values into temperature values, and obtaining a temperature distribution map corresponding to the characteristic regions of the target shape;
specifically, the gray level processing is performed on each target shape feature region image, and the gray level processing technology is the prior art and is not described in detail herein. The temperature is expressed by color, and the darker the color, the lower the temperature, and the whiter the color, the higher the temperature. After the target shape feature region image is grayed, since the darker the temperature is, the whiter the temperature is, the conversion coefficient of the gray value into the temperature value can be converted according to a plurality of tests or by using a gray value-temperature value conversion formula, so that the temperature distribution map corresponding to each target shape feature region can be obtained. Illustratively, the gray value-temperature value conversion formula is: TEMP — INT (GRAY (255/(t0-t1))), where TEMP is the temperature value, INT is the rounding function, GRAY is the GRAY value, t0 is the temperature minimum value, and t1 is the temperature maximum value.
S330, judging whether the temperature of the current target shape characteristic region is within a preset temperature range according to the temperature distribution map corresponding to the current target shape characteristic region;
specifically, after the temperature distribution map corresponding to the target shape feature region is obtained, one target shape feature region is randomly selected as the current target shape feature region, and of course, the first target shape feature region may also be selected as the current target shape feature region according to the sequence before and after the segmentation performed by the image segmentation technology. And judging whether the temperature of the target shape characteristic region is within a preset temperature range according to the temperature distribution map corresponding to the current target shape characteristic region.
S340, if the temperature of the current target shape characteristic region is within the preset temperature range, responding to the content of the target shape characteristic region;
s350, if the temperature of the current target shape characteristic region is out of the preset temperature range, not responding to the content of the target shape characteristic region;
s360, switching to the temperature distribution diagram corresponding to the next target shape characteristic region to continue judging until the temperature distribution diagrams corresponding to all the target shape characteristic regions are judged.
Specifically, if the temperature of the current target shape feature region is within the preset temperature range, the content indicated by the target shape feature region is responded, and if the temperature of the current target shape feature region is outside the preset temperature range, the content indicated by the target shape feature region is not responded. And after the temperature judgment of the current target shape characteristic region is finished and corresponding operation is executed, switching to a temperature distribution diagram corresponding to the next target shape characteristic region to continue judging until the temperature distribution diagrams corresponding to all the target shape characteristic regions are judged. Illustratively, a typical user's finger temperature is 30-34 °, assuming that after detecting its temperature profile through a finger sample image, the resulting preset temperature range is 30-34 °, whereas the temperature of a finger-like elongated article (including but not limited to a pencil, pen) tends to be below 30 °, and the temperature of a finger-like elongated pattern or illustration tends to be below 30 °. Therefore, if the temperature of the current target shape feature region is within the preset temperature range, it is determined that the current target shape feature region corresponds to a human finger in the real world. On the contrary, the article is a long strip similar to the finger shape.
The same parts in this embodiment as those in the above embodiments are not described in detail herein. In this embodiment, after the image is grayed, the grayed image is subjected to gray value conversion into a corresponding temperature value to obtain a temperature distribution map, so that the temperature of the target shape feature region is obtained according to the temperature distribution map, and when the temperature of the target shape feature region is determined to be within the preset temperature range, the content indicated by the target shape feature region is responded, otherwise, the response is not performed. This embodiment can combine image recognition and temperature to detect, can filter other target shape characteristic regions except that the user indicates, can filter the pattern similar with the finger shape to and the non-finger object similar with the finger shape, like pen, pencil etc. consequently, can reduce the misidentification, improve the rate of accuracy that the finger point read, thereby promote user's use and experience.
An embodiment of the present invention, as shown in fig. 4, is a method for implementing point reading, including:
s110, acquiring input information of a user, and triggering an infrared camera to shoot towards a book area pointed by the user to acquire infrared image data if the input information meets a preset triggering condition;
specifically, the input information includes, but is not limited to, voice input information and operation input information. And when the voice input information of the user is acquired, identifying the voice input information, and when the voice input information contains a preset key word for starting the infrared camera, sending a starting instruction. For example, the user says "turn on the infrared camera," and when the infrared camera detects keywords such as "turn on", "infrared camera", and the like, a turn-on instruction is generated accordingly. The keywords can be set in an early stage based on the requirements of the user, such as a series of words with similar meanings, such as "turn on", "infrared camera", and the like, which represent that the intelligent device is turned on. In addition, the related keywords should be included as comprehensively as possible in consideration of the influence of the accents, dialects, and other factors of the users in different areas and the language.
And generating a starting instruction when the operation input information accords with the preset starting operation. The specific type of the user operation input information depends on the opening mode of the infrared camera. For example, if the intelligent device body is provided with a key button and the like for controlling the opening or closing of the infrared camera, the user controls the opening or closing of the infrared camera by pressing or clicking the key button. For example, the intelligent device body is provided with a touch screen, and if the gesture track of the user on the touch screen accords with a preset opening gesture, the infrared camera is controlled to be opened.
S200, carrying out image recognition on the infrared image data to obtain a target shape characteristic region;
s300, acquiring a temperature distribution map of the target shape characteristic region, and responding to the content of the target shape characteristic region if the temperature of the target shape characteristic region is within a preset temperature range.
The same parts in this embodiment as those in the above embodiments are not described in detail herein. In this embodiment, can open infrared camera through multiple mode and shoot and acquire infrared image data, increase the scene that the user used intelligent device, promote user's use and experience.
An embodiment of the present invention, as shown in fig. 5, is a point-and-read implementation system, including:
the acquisition module 10 is used for acquiring infrared image data; the infrared image data comprises a book area pointed by a user;
specifically, a user can place a book in a reading area of the reading machine, the user points to a target to be read by using a finger, and then the infrared camera arranged on the reading machine shoots the book area pointed by the user to obtain infrared image data. Certainly, the user can place books in intelligent lamps and lanterns (including intelligent desk lamp, intelligent wall lamp etc.) below, is provided with infrared camera on the intelligent lamps and lanterns, and the user uses the finger to point to the self required object of reading, then shoots the books region that the user points through the infrared camera that sets up on the intelligent lamps and lanterns and obtain infrared image data. Certainly, the user can place books in infrared camera below, and the user uses the finger to point to the object that needs to click to read by oneself, then shoots the books region that the user points through infrared camera and obtain infrared image data.
The object to be read can be a word or a sentence, and the language of the word or the sentence can be any language form such as Chinese or English.
Of course, this is merely an example, and the user may adjust the orientation of the infrared camera toward the book area pointed by the user through any intelligent hardware (including but not limited to a point-reading machine, a computer, a tablet, and an intelligent lamp) provided with the infrared camera, so as to capture infrared image data. Certainly, the user can also directly shoot through infrared camera and obtain infrared image data, then, upload infrared image data to the server or be provided with the intelligent hardware of infrared camera not.
The image recognition module 20 is configured to perform image recognition on the infrared image data to obtain a target shape feature region;
specifically, the acquired infrared image data may include a finger pointed by the user to point the reading object, or a long strip-shaped article (including but not limited to a pencil or a pen) similar to the shape of the finger, or a long strip-shaped pattern or a picture inserted in a book pointed by the user and similar to the shape of the finger.
Since the acquired infrared image data may have many interference contents, such as a strip pattern or illustration similar to the shape of a finger, a strip article similar to the shape of a finger, and the finger of the user is also strip-shaped, the interference contents and the shape of the finger of the user are extremely similar, and the target shape is strip-shaped. Therefore, no matter what the infrared image data includes, image recognition can be performed on the infrared image data by using any image recognition technology in the prior art, so as to recognize the target shape feature region in the infrared image data, that is, recognize the strip-shaped feature region in the infrared image data.
The processing module 30 is configured to obtain a temperature distribution map of the target shape feature region, and respond to the content of the target shape feature region if the temperature of the target shape feature region is within a preset temperature range.
Specifically, a temperature distribution map of a target shape feature region, that is, a long-strip-shaped feature region, is obtained through an existing temperature distribution map extraction technology, because any natural object radiates infrared rays to the surroundings, and the heat values of the infrared rays radiated by different objects are different, the temperature range of a user's finger is set to be a preset temperature range, and then after the temperature distribution map of the target shape feature region is obtained, it is determined that the temperature of the target shape feature region belongs to the preset temperature range, where the temperature may be the temperature of all pixels or the average value of the temperatures of all pixels. Once the temperature of the target shape characteristic region is within the preset temperature range, it is indicated that the target shape characteristic region corresponds to a human finger in the real world, and therefore, the content pointed by the target shape characteristic region, that is, the content pointed by the user finger, is responded, and accurate finger touch reading is achieved.
In this embodiment, infrared image data is acquired, a target shape feature region in the infrared image data is identified, a temperature distribution map is extracted for the target shape feature region, and only when the temperature of the target shape feature region falls within a preset temperature range, the content pointed by the target shape feature region is responded, so that, in combination with image identification and temperature detection, the remaining target shape feature regions except for the fingers of the user can be filtered, i.e., patterns similar to the shapes of the fingers and non-finger objects similar to the shapes of the fingers, such as pens, pencils and the like, can be filtered, therefore, misrecognition can be reduced, the accuracy of finger point reading is improved, and the use experience of the user is improved.
Based on the foregoing embodiment corresponding to fig. 5, as shown in fig. 6, the click-to-read implementation system further includes:
a collection module 40 for acquiring a sample image dataset;
a training module 50, configured to train according to the sample image data set to obtain a finger recognition neural model;
in particular, the sample image dataset includes a finger sample image dataset and a finger-like sample image dataset. The finger sample image includes human finger portion content and does not include finger-like shaped interference content.
The finger sample image dataset is acquired in a manner comprising: the method comprises the steps of downloading a large amount of image data from a network, screening finger sample images which comprise human finger part contents and do not comprise interference contents similar to finger shapes from the image data, and screening a large amount of finger sample images to obtain a finger sample image data set. And secondly, acquiring a small amount of finger sample images which comprise the content of the finger part of the human body and do not comprise interference content similar to the shape of the finger, and then performing data amplification by taking the small amount of finger sample images as template images, namely performing pixel transformation and/or geometric transformation on the small amount of finger sample images respectively to obtain a large amount of finger sample images so as to obtain a finger sample image data set.
The acquisition mode of the finger-like sample image dataset comprises the following steps: firstly, a large amount of image data are downloaded from a network, finger-like sample images which do not include human finger part contents and include interference contents similar to finger shapes are screened out from the image data, and a large amount of finger-like sample images are screened out, so that a finger-like sample image data set is obtained. And secondly, acquiring a small amount of finger-like sample images which do not include the content of the human finger part and include interference content similar to the shape of the finger, and then performing data amplification by taking the small amount of finger-like sample images as template images, namely performing pixel transformation and/or geometric transformation on the small amount of finger-like sample images respectively to obtain a large amount of finger-like sample images so as to obtain a finger-like sample image dataset.
Wherein the pixel transformation comprises: 1. adding noise and filtering, wherein the noise is obtained by a mode including but not limited to salt and pepper noise, Gaussian noise and median filtering; 2. changing channels and adjusting the sequence of the three RBG channels; 3. adjusting contrast, brightness and saturation, and color dithering.
The geometric transformation includes: 1. turning over, for example: the face turning device is turned horizontally and vertically according to actual conditions, for example, regarding the face, the face turned up and down is changed into a face turned over, and the turning has no practical significance; 2. translating to simulate the situation that the picture in real life is not centered, and changing the position; 3. rotating; 4. black setting is carried out, and data samples which are partially shielded are simulated; 5. cutting; 6. and (4) zooming.
After the sample image data set is obtained in the above manner, if the sample image data set only includes the finger sample image data set, the finger sample image data set can be divided into a finger sample image data training set and a finger sample image data verification set, training is performed according to the finger sample images in the finger sample image data training set, verification is performed according to the finger sample images in the finger sample image data verification set, model parameters are adjusted according to verification results, verification is performed through multiple times of training, and the model parameters are adjusted, so that the finger recognition neural model is obtained.
Of course, after the sample image dataset is obtained in the above manner, if the sample image dataset includes a finger sample image dataset and a finger-like sample image dataset, the finger sample image dataset may be divided into a finger sample image data training set and a finger sample image data validation set, the finger-like sample image dataset may be divided into a finger-like sample image data training set and a finger-like sample image data validation set, a finger sample image in the finger sample image data training set and a finger-like sample image in the finger-like sample image data training set are trained according to the finger sample image in the finger sample image data training set, a finger sample image in the finger-like sample image data validation set and a finger-like sample image in the finger-like sample image data validation set are validated according to the finger sample image in the finger sample image data training set, a model parameter is adjusted according to the validation result, a model parameter is trained and, thereby obtaining the finger recognition neural model. Compared with the method for training the finger recognition neural network model only according to the finger sample image data set, the method for training the finger recognition neural network model through the positive and negative samples (namely the finger sample image and the finger-like sample image) can improve the accuracy and reliability of finger recognition of the trained finger recognition neural model.
The image recognition module 20 is further configured to input the infrared image data to the finger recognition neural model, and if the infrared image data conforms to the shape of the finger, recognize all target shape feature areas conforming to the shape of the finger in the infrared image data through a preset target detection algorithm.
Specifically, the acquired infrared image data is input into a trained finger recognition neural model, and through the trained finger recognition neural model, whether the acquired infrared image data includes a target shape characteristic region conforming to the shape of the finger is judged first, and if not, the subsequent steps are not performed. And if the target shape characteristic area is included, performing target detection on the infrared image data through a preset target detection algorithm, so as to identify all target shape characteristic areas which accord with the finger shape in the infrared image data. The preset target detection algorithm includes a YOLO target detection algorithm, a YOLO2 target detection algorithm, an R-CNN target detection algorithm, a Fast R-CNN target detection algorithm, an FPN target detection algorithm, an SSD target detection algorithm, a RetinaNet target detection algorithm, and the like.
The same parts in this embodiment as those in the above embodiments are not described in detail herein. Through this embodiment, use finger identification neural network model to carry out the infrared image data that preliminary screening accords with the finger shape, to the infrared image data that accords with the finger shape after passing preliminary screening again, use and predetermine the target detection algorithm and discern wherein all target shape characteristic regions that accord with the finger shape, can promote all the identification accuracy who accords with the target shape characteristic region of finger shape, greatly reduced the error probability of discernment. And the calculation amount and the resource occupation can be reduced through preliminary screening, and the recognition efficiency of the target shape characteristic region conforming to the finger shape can be improved.
Based on the foregoing embodiment corresponding to fig. 5, as shown in fig. 7, the processing module 30 includes:
the extraction unit 31 is configured to segment the infrared image data to obtain a target shape feature region image corresponding to each target shape feature region;
specifically, according to the embodiment, after the target shape feature region is identified by using the preset target detection algorithm, all the target shape feature regions in the infrared image data are segmented by using the image segmentation technology, so as to obtain the target shape feature region image corresponding to each target shape feature region. Illustratively, the infrared image data includes a user finger and a pencil, and the infrared image data is divided into a finger area image and a pencil area image. Of course, this is merely an example, and other situations are within the scope of the present invention, and will not be described in detail herein.
The processing unit 32 is configured to perform gray processing on the image of each target shape feature region, convert the gray value into a temperature value, and obtain a temperature distribution map corresponding to each target shape feature region;
specifically, the gray level processing is performed on each target shape feature region image, and the gray level processing technology is the prior art and is not described in detail herein. The temperature is expressed by color, and the darker the color, the lower the temperature, and the whiter the color, the higher the temperature. After the target shape feature region image is grayed, since the darker the temperature is, the whiter the temperature is, the conversion coefficient of the gray value into the temperature value can be converted according to a plurality of tests or by using a gray value-temperature value conversion formula, so that the temperature distribution map corresponding to each target shape feature region can be obtained. Illustratively, the gray value-temperature value conversion formula is: TEMP — INT (GRAY (255/(t0-t1))), where TEMP is the temperature value, INT is the rounding function, GRAY is the GRAY value, t0 is the temperature minimum value, and t1 is the temperature maximum value.
The judging unit 33 is configured to judge whether the temperature of the current target shape feature region is within a preset temperature range according to the temperature distribution map corresponding to the current target shape feature region;
specifically, after the temperature distribution map corresponding to the target shape feature region is obtained, one target shape feature region is randomly selected as the current target shape feature region, and of course, the first target shape feature region may also be selected as the current target shape feature region according to the sequence before and after the segmentation performed by the image segmentation technology. And judging whether the temperature of the target shape characteristic region is within a preset temperature range according to the temperature distribution map corresponding to the current target shape characteristic region.
The control unit 34 is configured to respond to the content of the target shape characteristic region if the temperature of the current target shape characteristic region is within the preset temperature range; if the temperature of the current target shape characteristic region is out of the preset temperature range, not responding to the content of the target shape characteristic region;
the control unit 34 is further configured to switch to the temperature distribution map corresponding to the next target shape feature region to continue the determination until the temperature distribution maps corresponding to all the target shape feature regions are determined.
Specifically, if the temperature of the current target shape feature region is within the preset temperature range, the content indicated by the target shape feature region is responded, and if the temperature of the current target shape feature region is outside the preset temperature range, the content indicated by the target shape feature region is not responded. And after the temperature judgment of the current target shape characteristic region is finished and corresponding operation is executed, switching to a temperature distribution diagram corresponding to the next target shape characteristic region to continue judging until the temperature distribution diagrams corresponding to all the target shape characteristic regions are judged. Illustratively, a typical user's finger temperature is 30-34 °, assuming that after detecting its temperature profile through a finger sample image, the resulting preset temperature range is 30-34 °, whereas the temperature of a finger-like elongated article (including but not limited to a pencil, pen) tends to be below 30 °, and the temperature of a finger-like elongated pattern or illustration tends to be below 30 °. Therefore, if the temperature of the current target shape feature region is within the preset temperature range, it is determined that the current target shape feature region corresponds to a human finger in the real world. On the contrary, the article is a long strip similar to the finger shape.
The same parts in this embodiment as those in the above embodiments are not described in detail herein. In this embodiment, after the image is grayed, the grayed image is subjected to gray value conversion into a corresponding temperature value to obtain a temperature distribution map, so that the temperature of the target shape feature region is obtained according to the temperature distribution map, and when the temperature of the target shape feature region is determined to be within the preset temperature range, the content indicated by the target shape feature region is responded, otherwise, the response is not performed. This embodiment can combine image recognition and temperature to detect, can filter other target shape characteristic regions except that the user indicates, can filter the pattern similar with the finger shape to and the non-finger object similar with the finger shape, like pen, pencil etc. consequently, can reduce the misidentification, improve the rate of accuracy that the finger point read, thereby promote user's use and experience.
Based on any one of the corresponding embodiments of fig. 5 to fig. 7, the click-to-read implementation system further includes:
the input module is used for acquiring input information of a user;
and the control module is used for triggering the infrared camera to shoot towards the book area pointed by the user to acquire infrared image data if the input information meets the preset triggering condition.
Specifically, the input information includes, but is not limited to, voice input information and operation input information. And when the voice input information of the user is acquired, identifying the voice input information, and when the voice input information contains a preset key word for starting the infrared camera, sending a starting instruction. For example, the user says "turn on the infrared camera," and when the infrared camera detects keywords such as "turn on", "infrared camera", and the like, a turn-on instruction is generated accordingly. The keywords can be set in an early stage based on the requirements of the user, such as a series of words with similar meanings, such as "turn on", "infrared camera", and the like, which represent that the intelligent device is turned on. In addition, the related keywords should be included as comprehensively as possible in consideration of the influence of the accents, dialects, and other factors of the users in different areas and the language.
And generating a starting instruction when the operation input information accords with the preset starting operation. The specific type of the user operation input information depends on the opening mode of the infrared camera. For example, if the intelligent device body is provided with a key button and the like for controlling the opening or closing of the infrared camera, the user controls the opening or closing of the infrared camera by pressing or clicking the key button. For example, the intelligent device body is provided with a touch screen, and if the gesture track of the user on the touch screen accords with a preset opening gesture, the infrared camera is controlled to be opened.
Illustratively, the point reading machine is switched to a point reading mode, the intention of a user is recognized, when the intention of the user is recognized, an infrared camera is started to collect infrared image data, the infrared image data are recognized, whether the temperature exists in a target shape characteristic region or not is judged, whether the temperature value is within a preset temperature range or not is judged, if the temperature value is out of the preset temperature range, interference content, namely patterns similar to the shape of a finger, and non-finger objects similar to the shape of the finger, such as a pen, a pencil and the like, is judged, and the interference content is removed. And if the human body characteristic is the finger of the user within the preset temperature range, responding to the area pointed by the finger of the user. For example, if the human body characteristic is the finger of the user within the preset temperature range, the response is performed on the area pointed by the finger of the user, and the area pointed by the finger of the user is 'meditation night' and the reading machine searches and plays the 'meditation night' content to the user.
One embodiment of the present invention, as shown in fig. 8, is a computer apparatus 100, comprising a processor 110, a memory 120, wherein the memory 120 is used for storing a computer program; the processor 110 is configured to execute the computer program stored in the memory 120 to implement the read-and-click implementation method in the method embodiment corresponding to any one of fig. 1 to fig. 4.
Fig. 8 is a schematic structural diagram of a computer device 100 according to an embodiment of the present invention. Referring to fig. 8, the computer device 100 includes a processor 110 and a memory 120, and may further include a communication interface 140 and a communication bus 120, and may further include an input/output interface 130, wherein the processor 110, the memory 120, the input/output interface 130 and the communication interface 140 complete communication with each other through the communication bus 120. The memory 120 stores a computer program, and the processor 110 is configured to execute the computer program stored in the memory 120 to implement the read-and-click implementation method in the method embodiment corresponding to any one of fig. 1 to fig. 4.
A communication bus 120 is a circuit that connects the described elements and enables transmission between the elements. For example, the processor 110 receives commands from other elements through the communication bus 120, decrypts the received commands, and performs calculations or data processing according to the decrypted commands. The memory 120 may include program modules such as a kernel (kernel), middleware (middleware), an Application Programming Interface (API), and applications. The program modules may be comprised of software, firmware or hardware, or at least two of the same. The input/output interface 130 relays commands or data input by a user through input/output devices (e.g., sensors, keyboards, touch screens). The communication interface 140 connects the computer device 100 to other network devices, user devices, networks. For example, the communication interface 140 may be connected to a network by wire or wirelessly to connect to external other network devices or user devices. The wireless communication may include at least one of: wireless fidelity (WiFi), Bluetooth (BT), Near Field Communication (NFC), Global Positioning Satellite (GPS) and cellular communications, among others. The wired communication may include at least one of: universal Serial Bus (USB), high-definition multimedia interface (HDMI), asynchronous transfer standard interface (RS-232), and the like. The network may be a telecommunications network and a communications network. The communication network may be a computer network, the internet of things, a telephone network. The computer device 100 may connect to a network through the communication interface 140, and protocols by which the computer device 100 communicates with other network devices may be supported by at least one of an application, an Application Programming Interface (API), middleware, a kernel, and the communication interface 140.
In an embodiment of the present invention, a storage medium stores at least one instruction, and the instruction is loaded and executed by a processor to implement the operations performed by the corresponding embodiments of the above-mentioned point-and-read implementation method. For example, the computer readable storage medium may be a read-only memory (ROM), a random-access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.
They may be implemented in program code that is executable by a computing device such that it is executed by the computing device, or separately, or as individual integrated circuit modules, or as a plurality or steps of individual integrated circuit modules. Thus, the present invention is not limited to any specific combination of hardware and software.
It should be noted that the above embodiments can be freely combined as necessary. The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A point reading implementation method is characterized by comprising the following steps:
acquiring infrared image data; the infrared image data comprises a book area pointed by a user;
carrying out image recognition on the infrared image data to obtain a target shape characteristic region;
and acquiring a temperature distribution map of the target shape characteristic region, and responding to the content pointed by the target shape characteristic region if the temperature of the target shape characteristic region is within a preset temperature range.
2. The method for realizing point reading according to claim 1, wherein the step of acquiring infrared image data comprises the steps of:
obtaining a sample image dataset;
training according to the sample image data set to obtain a finger recognition neural model;
the step of carrying out image recognition on the infrared image data to obtain a target shape characteristic region comprises the following steps:
and inputting the infrared image data into the finger recognition neural model, and recognizing all target shape characteristic areas which accord with the finger shape in the infrared image data through a preset target detection algorithm if the infrared image data accord with the finger shape.
3. The method according to claim 1, wherein the step of obtaining the temperature distribution map of the target shape feature region, and if the temperature of the target shape feature region falls within a preset temperature range, responding to the content of the target shape feature region comprises:
segmenting the infrared image data to obtain a target shape characteristic region image corresponding to each target shape characteristic region;
carrying out gray level processing on the images of the characteristic regions of each target shape, converting the gray level values into temperature values, and obtaining a temperature distribution map corresponding to each characteristic region of the target shape;
judging whether the temperature of the current target shape characteristic region is within a preset temperature range or not according to the temperature distribution map corresponding to the current target shape characteristic region;
if the temperature of the current target shape characteristic region is within the preset temperature range, responding to the content of the target shape characteristic region;
and if the temperature of the current target shape characteristic region is out of the preset temperature range, not responding to the content of the target shape characteristic region.
4. The point-reading implementation method according to any one of claims 1 to 3, wherein the acquiring infrared image data includes the steps of:
acquiring input information of a user, and triggering an infrared camera to shoot towards a book area pointed by the user to acquire infrared image data if the input information meets a preset triggering condition;
and if the operation instruction accords with a preset trigger instruction, triggering an infrared camera to shoot towards the book area pointed by the user to acquire the infrared image data.
5. A click-to-read implementation system, comprising:
the acquisition module is used for acquiring infrared image data; the infrared image data comprises a book area pointed by a user;
the image recognition module is used for carrying out image recognition on the infrared image data to obtain a target shape characteristic region;
and the processing module is used for acquiring the temperature distribution map of the target shape characteristic region, and responding to the content pointed by the target shape characteristic region if the temperature of the target shape characteristic region is within a preset temperature range.
6. The point-reading implementation system according to claim 5, further comprising:
a collection module for obtaining a sample image dataset;
the training module is used for training according to the sample image data set to obtain a finger recognition neural model;
the image recognition module is further configured to input the infrared image data to the finger recognition neural model, and if the infrared image data conforms to the shape of the finger, recognize all target shape feature areas conforming to the shape of the finger in the infrared image data through a preset target detection algorithm.
7. The point-reading implementation system according to claim 5, wherein the processing module includes:
the extraction unit is used for segmenting the infrared image data to obtain a target shape characteristic region image corresponding to each target shape characteristic region;
the processing unit is used for carrying out gray processing on the images of the characteristic regions of the target shapes and converting the gray values into temperature values to obtain temperature distribution maps corresponding to the characteristic regions of the target shapes;
the judging unit is used for judging whether the temperature of the current target shape characteristic region is within a preset temperature range according to the temperature distribution map corresponding to the current target shape characteristic region;
the control unit is used for responding to the content of the target shape characteristic region if the temperature of the current target shape characteristic region is within the preset temperature range; if the temperature of the current target shape characteristic region is out of the preset temperature range, not responding to the content of the target shape characteristic region;
and the control unit is also used for switching to the temperature distribution diagram corresponding to the next target shape characteristic region to continue judging until the temperature distribution diagrams corresponding to all the target shape characteristic regions are judged.
8. The point-reading implementation system according to any one of claims 5 to 7, further comprising:
the input module is used for acquiring input information of a user;
and the control module is used for triggering the infrared camera to shoot towards the book area pointed by the user to acquire the infrared image data if the input information meets the preset triggering condition.
9. A computer device comprising a processor, a memory, wherein the memory is configured to store a computer program; the processor is configured to execute the computer program stored in the memory to implement the operations performed by the click-to-read implementation method according to any one of claims 1 to 4.
10. A storage medium having stored therein at least one instruction, which is loaded and executed by a processor to perform an operation performed by the read-and-click implementation method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910870427.XA CN110633027A (en) | 2019-09-16 | 2019-09-16 | Point reading implementation method, system, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910870427.XA CN110633027A (en) | 2019-09-16 | 2019-09-16 | Point reading implementation method, system, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110633027A true CN110633027A (en) | 2019-12-31 |
Family
ID=68972539
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910870427.XA Pending CN110633027A (en) | 2019-09-16 | 2019-09-16 | Point reading implementation method, system, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110633027A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111353501A (en) * | 2020-02-25 | 2020-06-30 | 暗物智能科技(广州)有限公司 | Book point-reading method and system based on deep learning |
| CN111638783A (en) * | 2020-05-18 | 2020-09-08 | 广东小天才科技有限公司 | Man-machine interaction method and electronic equipment |
| CN112051953A (en) * | 2020-09-29 | 2020-12-08 | 中国银行股份有限公司 | Output control method and device for page column and electronic equipment |
| CN113721769A (en) * | 2021-08-31 | 2021-11-30 | 歌尔科技有限公司 | Projection interference detection method, device, equipment and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080112597A1 (en) * | 2006-11-10 | 2008-05-15 | Tomoyuki Asano | Registration Apparatus, Verification Apparatus, Registration Method, Verification Method and Program |
| CN103188537A (en) * | 2011-10-05 | 2013-07-03 | 索尼公司 | Input apparatus and input recognition method |
| CN105992988A (en) * | 2014-02-17 | 2016-10-05 | Metaio有限公司 | Method and device for detecting a touch between a first object and a second object |
| CN106030493A (en) * | 2013-12-18 | 2016-10-12 | 前视红外***股份公司 | Processing infrared images based on swipe gestures |
| CN106408560A (en) * | 2016-09-05 | 2017-02-15 | 广东小天才科技有限公司 | Method and device for rapidly acquiring effective image |
| CN107563494A (en) * | 2017-08-01 | 2018-01-09 | 华南理工大学 | A kind of the first visual angle Fingertip Detection based on convolutional neural networks and thermal map |
| CN108446032A (en) * | 2017-12-28 | 2018-08-24 | 安徽慧视金瞳科技有限公司 | A kind of mouse gestures implementation method in projection interactive system |
| CN207833454U (en) * | 2018-02-02 | 2018-09-07 | 深圳博林文创股份有限公司 | Multimedia interaction experiencing system |
| CN109376612A (en) * | 2018-09-27 | 2019-02-22 | 广东小天才科技有限公司 | Method and system for assisting positioning learning based on gestures |
-
2019
- 2019-09-16 CN CN201910870427.XA patent/CN110633027A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080112597A1 (en) * | 2006-11-10 | 2008-05-15 | Tomoyuki Asano | Registration Apparatus, Verification Apparatus, Registration Method, Verification Method and Program |
| CN103188537A (en) * | 2011-10-05 | 2013-07-03 | 索尼公司 | Input apparatus and input recognition method |
| CN106030493A (en) * | 2013-12-18 | 2016-10-12 | 前视红外***股份公司 | Processing infrared images based on swipe gestures |
| CN105992988A (en) * | 2014-02-17 | 2016-10-05 | Metaio有限公司 | Method and device for detecting a touch between a first object and a second object |
| CN106408560A (en) * | 2016-09-05 | 2017-02-15 | 广东小天才科技有限公司 | Method and device for rapidly acquiring effective image |
| CN107563494A (en) * | 2017-08-01 | 2018-01-09 | 华南理工大学 | A kind of the first visual angle Fingertip Detection based on convolutional neural networks and thermal map |
| CN108446032A (en) * | 2017-12-28 | 2018-08-24 | 安徽慧视金瞳科技有限公司 | A kind of mouse gestures implementation method in projection interactive system |
| CN207833454U (en) * | 2018-02-02 | 2018-09-07 | 深圳博林文创股份有限公司 | Multimedia interaction experiencing system |
| CN109376612A (en) * | 2018-09-27 | 2019-02-22 | 广东小天才科技有限公司 | Method and system for assisting positioning learning based on gestures |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111353501A (en) * | 2020-02-25 | 2020-06-30 | 暗物智能科技(广州)有限公司 | Book point-reading method and system based on deep learning |
| CN111638783A (en) * | 2020-05-18 | 2020-09-08 | 广东小天才科技有限公司 | Man-machine interaction method and electronic equipment |
| CN112051953A (en) * | 2020-09-29 | 2020-12-08 | 中国银行股份有限公司 | Output control method and device for page column and electronic equipment |
| CN112051953B (en) * | 2020-09-29 | 2021-09-14 | 中国银行股份有限公司 | Output control method and device for page column and electronic equipment |
| CN113721769A (en) * | 2021-08-31 | 2021-11-30 | 歌尔科技有限公司 | Projection interference detection method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110633027A (en) | Point reading implementation method, system, computer equipment and storage medium | |
| JP6208383B2 (en) | Image capturing parameter adjustment in preview mode | |
| CN105825109B (en) | The method and terminal of a kind of terminal unlocking | |
| CN107871001B (en) | Audio playing method and device, storage medium and electronic equipment | |
| CN107885430B (en) | Audio playing method and device, storage medium and electronic equipment | |
| CN111758116B (en) | Face image recognition system, recognizer generation device, recognition device, and face image recognition system | |
| CN106228556B (en) | image quality analysis method and device | |
| EP3001354A1 (en) | Object detection method and device for online training | |
| US11151398B2 (en) | Anti-counterfeiting processing method, electronic device, and non-transitory computer-readable storage medium | |
| CN108781252B (en) | Image shooting method and device | |
| US20150269135A1 (en) | Language identification for text in an object image | |
| CN108108117B (en) | Screen capturing method and device and terminal | |
| JP6127676B2 (en) | Image recognition apparatus, image recognition program, recording medium, and image recognition method | |
| WO2023197648A1 (en) | Screenshot processing method and apparatus, electronic device, and computer readable medium | |
| CN108021905A (en) | image processing method, device, terminal device and storage medium | |
| WO2023005813A1 (en) | Image direction adjustment method and apparatus, and storage medium and electronic device | |
| CN113221718A (en) | Formula identification method and device, storage medium and electronic equipment | |
| CN109271982A (en) | Multiple identification region recognition methods, identification terminal and readable storage medium storing program for executing | |
| CN110619325A (en) | Text recognition method and device | |
| CN103942334A (en) | User biological characteristic database updating method and device | |
| CN106485246B (en) | Character identifying method and device | |
| US11600057B2 (en) | Method for processing multimodal images, apparatus, device and storage medium | |
| CN110858291A (en) | Character segmentation method and device | |
| CN110796096B (en) | Training method, device, equipment and medium for gesture recognition model | |
| CN109977737A (en) | A kind of character recognition Robust Method based on Recognition with Recurrent Neural Network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191231 |
|
| RJ01 | Rejection of invention patent application after publication |