CN117827013B - Intelligent learning method and system for production skills based on image processing and electronic equipment - Google Patents
Intelligent learning method and system for production skills based on image processing and electronic equipment Download PDFInfo
- Publication number
- CN117827013B CN117827013B CN202410244508.XA CN202410244508A CN117827013B CN 117827013 B CN117827013 B CN 117827013B CN 202410244508 A CN202410244508 A CN 202410244508A CN 117827013 B CN117827013 B CN 117827013B
- Authority
- CN
- China
- Prior art keywords
- production equipment
- target production
- contour
- current frame
- frame image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004519 manufacturing process Methods 0.000 title claims abstract description 221
- 238000012545 processing Methods 0.000 title claims abstract description 39
- 238000000034 method Methods 0.000 title claims abstract description 36
- 230000003993 interaction Effects 0.000 claims abstract description 30
- 230000002452 interceptive effect Effects 0.000 claims description 9
- 239000011159 matrix material Substances 0.000 claims description 6
- 239000003086 colorant Substances 0.000 claims description 3
- 230000008859 change Effects 0.000 claims description 2
- 238000004590 computer program Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000012552 review Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/28—Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Evolutionary Computation (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Databases & Information Systems (AREA)
- Artificial Intelligence (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the field of general image processing or generation, in particular to an intelligent learning method and system for production skills based on image processing, and electronic equipment, wherein the method comprises the steps of obtaining a current frame image of target production equipment shot by a VR camera on head-mounted VR equipment; processing the current frame image, and extracting the outline of the target production equipment from the current frame image; determining the name of the target production equipment according to the outline of the target production equipment in the current frame image; writing the name of the target production equipment into the interior of the outline corresponding to the target production equipment; outputting a voice interaction signal related to whether to play the operation video of the target production equipment to a user, acquiring interaction voice of the user, judging whether to acquire the operation video corresponding to the target production equipment preset in the control module according to the interaction voice of the user, and judging whether to play the video in the VR equipment. The user can perform relevant learning without consulting the target equipment data by himself, and a great deal of time and cost are saved.
Description
Technical Field
The invention relates to the technical field of general image processing or generation, in particular to an intelligent learning method and system for production skills based on image processing and electronic equipment.
Background
Along with the social development, production equipment of workshops is gradually intelligent, operation is gradually complicated, and because workshop workers learn production equipment and production skills, books, words, pictures and image data which are needed are obtained by searching by people, and then the workshops are required to review a large amount of data by themselves in the prior art, so that a large amount of time is wasted for searching the data, and the learning efficiency is low.
Thus, the prior art is still to be further developed.
Disclosure of Invention
The invention aims to overcome the technical defects and provide an intelligent learning method and system for production skills based on image processing and electronic equipment so as to solve the problems in the prior art.
To achieve the above object, according to a first aspect of the present invention, there is provided an intelligent learning method for production skills based on image processing, comprising:
s100, acquiring a current frame image of a target production device shot by a VR camera on a head-mounted VR device; processing the current frame image, and extracting the outline of the target production equipment from the current frame image;
S200, determining the name of the target production equipment according to the outline of the target production equipment in the current frame image; writing the name of the target production equipment into the interior of the outline corresponding to the target production equipment;
S300, outputting a voice interaction signal related to whether to play the operation video of the target production equipment to a user, acquiring interaction voice of the user, judging whether to acquire the operation video corresponding to the target production equipment preset in the control module according to the interaction voice of the user, and judging whether to play the operation video in the head-mounted VR equipment.
Specifically, the extracting the contour of the target production device in the current frame image includes:
And carrying out binarization processing on the current frame image, converting the current frame image into an image only comprising black and white colors so as to distinguish a target production equipment area and a background area in the current frame image, extracting contour feature points from the target production equipment area of the current frame image after the binarization processing, and fitting according to the contour feature points to obtain a first contour of target production equipment to be matched.
Specifically, the determining the name of the target production device according to the outline of the target production device in the current frame image includes:
Obtaining second contours of all sample production equipment preset in a control module, calculating mass center coordinates of the first contours and the second contours, rotating and scaling the first contours after the mass centers of the first contours and the second contours are overlapped, enabling the difference of Euclidean distances of all vertexes of the first contours and the second contours to be minimum, sequentially calculating contour coincidence degree data of the first contours and the second contours of all sample production equipment at the moment, and determining names of target production equipment according to the contour coincidence degree data.
Specifically, the determining the name of the target production equipment according to the contour coincidence degree data includes:
Sequentially judging whether the contour coincidence degree data of the first contour and the second contour of each sample production device is larger than or equal to a first preset threshold value, if the contour coincidence degree data of the first contour and the second contour of a certain sample production device is larger than or equal to the first preset threshold value, judging that the name of the sample production device is the name of the target production device; if the contour coincidence ratio data of the first contour and the second contour of each sample production device are smaller than a first preset threshold value, outputting a voice prompt signal related to matching failure and requesting to adjust the shooting angle.
Specifically, the step of judging whether to acquire the operation video corresponding to the target production device preset in the control module according to the interactive voice of the user and judging whether to play the operation video in the head-mounted VR device includes:
if the acquisition result of the interactive voice of the user is yes, acquiring an operation video corresponding to the target production equipment preset in the control module, and playing the operation video in the head-mounted VR equipment;
If the acquisition result of the interactive voice of the user is no, the operation video corresponding to the target production equipment preset in the control module is not acquired.
Specifically, the writing the name of the target production equipment into the outline corresponding to the target production equipment includes:
And writing the name of the target production equipment into the current frame image through an opencv library of python, wherein the preset position corresponds to the centroid coordinate of the first contour, the preset position is the upper left corner coordinate of a text box corresponding to the name by taking the centroid coordinate of the first contour as the centroid coordinate of the first contour, and the text corresponding to the name of the target production equipment is inserted.
Specifically, the method further comprises the following steps:
Acquiring GPS coordinates of a camera by using a GPS positioning system arranged on the head-mounted VR equipment, measuring first height data of the camera from the ground by using a laser ranging system arranged on the head-mounted VR equipment, acquiring internal reference data and a holder angle of the camera, and calculating geographic coordinates for indicating target production equipment according to centroid coordinates of a first contour, the GPS coordinates of the camera, the first height data, the internal reference data and the holder angle; obtaining preset geographic coordinates corresponding to each sample production device preset in the control module, calculating actual distances for indicating the geographic coordinates of the target production device and the preset geographic coordinates corresponding to each sample production device, judging whether the shortest actual distance is smaller than or equal to a second preset threshold value among a plurality of actual distances corresponding to each sample production device, and determining the name of the target production device according to a judging result.
Specifically, the method further comprises the following steps:
if the shortest actual distance is smaller than or equal to a second preset threshold value, the name of the sample production equipment corresponding to the shortest actual distance is the name of the target production equipment;
If the shortest practical distance is larger than the second preset threshold, outputting a voice prompt signal related to the failure of matching and the adjustment of the shooting angle and the shooting position.
According to a second aspect of the present invention, there is provided an intelligent learning system for production skills based on image processing, comprising:
the acquisition module is used for acquiring a current frame image of the target production equipment shot by the VR camera on the head-mounted VR equipment;
The control module is used for processing the current frame image and extracting the outline of the target production equipment from the current frame image; or determining the name of the target production equipment according to the outline of the target production equipment in the current frame image; writing the name of the target production equipment into the interior of the outline corresponding to the target production equipment; or the method is used for outputting a voice interaction signal related to whether to play the operation video of the target production equipment to the user, acquiring the interaction voice of the user, judging whether to acquire the operation video corresponding to the target production equipment preset in the control module according to the interaction voice of the user, and judging whether to play the operation video in the head-mounted VR equipment.
According to a third aspect of the present invention, there is provided an electronic device comprising: a memory; and a processor, wherein the memory stores computer readable instructions, and the computer readable instructions realize the intelligent learning method of the production skills based on the image processing when being executed by the processor.
The beneficial effects are that:
According to the invention, the operation video and production skills of the corresponding production equipment can be learned by the head-mounted VR equipment without the need of the user to review the data, so that a large amount of time cost is saved, the learning efficiency is improved to a great extent, the modeling of a complex algorithm is not needed, the intelligent degree and usability of the invention are improved to a great extent, and the application scene of the invention is expanded to a great extent.
Drawings
FIG. 1 is a flow chart of a method for intelligent learning of production skills based on image processing provided in an embodiment of the present invention;
fig. 2 is a schematic diagram of the system components of the intelligent learning system for production skills based on image processing according to the embodiment of the invention.
Detailed Description
In order to make the technical solution of the present application better understood by those skilled in the art, the technical solution of the present application will be clearly and completely described in the following with reference to the accompanying drawings, and based on the embodiments of the present application, other similar embodiments obtained by those skilled in the art without making any inventive effort should be included in the scope of protection of the present application. In addition, directional words such as "upper", "lower", "left", "right", and the like, as used in the following embodiments are merely directions with reference to the drawings, and thus, the directional words used are intended to illustrate, not to limit, the application.
The invention will be further described with reference to the drawings and preferred embodiments.
Referring to fig. 1, the invention provides an intelligent learning method for production skills based on image processing, which comprises the following steps:
S100, acquiring a current frame image of a target production device shot by a VR camera on a head-mounted VR device; and processing the current frame image, and extracting the outline of the target production equipment from the current frame image.
Here, the step S100 includes, before:
The first preset threshold value, the second outline of each sample production device and the corresponding preset geographic coordinates of each sample production device are preset in the control module.
It can be understood that the first preset threshold value can be specifically set according to actual needs, the method does not limit the specific numerical value of the first preset threshold value, preferably, the method sets the first preset threshold value to 80%, and the fact that the first preset threshold value is set to 80% is obtained by a plurality of experiments of technicians of the method, whether the similarity of the first contour and the second contour is enough or not can be well judged, and reliability of the method is greatly improved.
It will be appreciated that the calculation of the contour overlap ratio data is a mature prior art, and the present invention is not repeated herein, and preferably, the calculation of the contour overlap ratio may be a ratio of calculating the number of pixels in the first contour area and the number of pixels in the second contour area.
It can be understood that the second preset threshold value can be specifically set according to actual needs, the specific numerical value of the second preset threshold value is not limited, preferably, the first preset threshold value is set to be 1 meter, the second preset threshold value is set to be 1 meter, and the second preset threshold value is obtained by a technician of the invention through a large number of experiments, so that whether the name of the sample production equipment corresponding to the shortest actual distance is the name of the target production equipment can be better judged, and the reliability of the invention is further improved.
Specifically, the extracting the contour of the target production device in the current frame image includes:
And carrying out binarization processing on the current frame image, converting the current frame image into an image only comprising black and white colors so as to distinguish a target production equipment area and a background area in the current frame image, extracting contour feature points from the target production equipment area of the current frame image after the binarization processing, and fitting according to the contour feature points to obtain a first contour of target production equipment to be matched.
S200, determining the name of the target production equipment according to the outline of the target production equipment in the current frame image; and writing the name of the target production equipment into the interior of the outline corresponding to the target production equipment.
Specifically, the determining the name of the target production device according to the outline of the target production device in the current frame image includes:
Obtaining second contours of all sample production equipment preset in a control module, calculating mass center coordinates of the first contours and the second contours, rotating and scaling the first contours after the mass centers of the first contours and the second contours are overlapped, enabling the difference of Euclidean distances of all vertexes of the first contours and the second contours to be minimum, sequentially calculating contour coincidence degree data of the first contours and the second contours of all sample production equipment at the moment, and determining names of target production equipment according to the contour coincidence degree data.
It can be appreciated that the method for calculating the centroid coordinates is as follows:
And respectively establishing a target search box according to the first contour and the second contour, wherein the target search box needs to comprise all contour feature points of the first contour and the second contour.
And carrying out gray level processing on the current frame image, carrying out piecewise linear change on the gray level value of the original current frame image according to the mean value and standard deviation of the gray level value of the original current frame image to obtain the gray level value of each point of the 8-bit single-channel image of the current frame image, and calculating according to the gray level value of each point of the 8-bit single-channel image of the current frame image to obtain the barycenter coordinate.
The piecewise linear variation formula is:
Wherein mu and sigma are respectively the mean value and standard deviation of the gray values of the original current frame image, x is the gray value of each point of the original current frame image, y is the gray value of each point of the 8-bit single-channel image obtained after linear transformation, and downward rounding operation is performed.
The centroid positioning calculation formula is as follows:
Wherein M and N represent the width and height of the target search box, respectively, (Mmin, nmin), and (Xmax, ymax) represent the upper left and lower right coordinates of the target search box, iij is the gray value of the pixel point in the rectangular region, i is the row position of the pixel point, j is the column position of the pixel point, and (Xc, yc) represents the centroid coordinates of the target search box, and the centroid coordinates are marked as the centroid coordinates of the first contour. The area of the target search box is the product of the height and the width of the target search box.
Specifically, the calculating, according to the centroid coordinate of the first contour, the GPS coordinate of the camera, the first height data, the internal reference data, and the pan-tilt angle, the geographic coordinate for indicating the target production device includes:
according to the first height, the internal reference data and the holder angle, calculating pixel coordinates and geographic coordinates of four vertexes and center points contained in the current frame image, according to the geographic coordinates and pixel coordinates of the four vertexes and the center points, calculating a homography matrix for converting the geographic coordinates and the pixel coordinates, and according to the centroid coordinates and the homography matrix of the first contour, calculating geographic coordinates for indicating target production equipment.
It should be noted that, the present invention is not repeated here, but the present invention calculates the homography matrix of the conversion of the geographic coordinates and the pixel coordinates, and calculates the prior art for indicating that the geographic coordinates of the target production device are mature according to the centroid coordinates and the homography matrix of the first contour.
Specifically, the determining the name of the target production equipment according to the contour coincidence degree data includes:
Sequentially judging whether the contour coincidence degree data of the first contour and the second contour of each sample production device is larger than or equal to a first preset threshold value, if the contour coincidence degree data of the first contour and the second contour of a certain sample production device is larger than or equal to the first preset threshold value, judging that the name of the sample production device is the name of the target production device; if the contour coincidence ratio data of the first contour and the second contour of each sample production device are smaller than a first preset threshold value, outputting a voice prompt signal related to matching failure and requesting to adjust the shooting angle.
It should be noted that, if the contour coincidence ratio data of the first contour and the second contour of a certain sample production device is greater than or equal to the first preset threshold value, it is proved that the coincidence ratio of the first contour and the second contour of the sample production device is large enough, and the name of the sample production device is judged to be the name of the target production device, otherwise, it is proved that the coincidence ratio of the first contour and the second contour of the sample production device is smaller, and it is not possible to judge that the name of the sample production device is the name of the target production device, and at this time, a voice prompt signal about failure in matching and requesting to adjust the shooting angle is output. The intelligent degree and usability of the invention are further improved.
S300, outputting a voice interaction signal related to whether to play the operation video of the target production equipment to a user, acquiring interaction voice of the user, judging whether to acquire the operation video corresponding to the target production equipment preset in the control module according to the interaction voice of the user, and judging whether to play the operation video in the head-mounted VR equipment.
Specifically, the step of judging whether to acquire the operation video corresponding to the target production device preset in the control module according to the interactive voice of the user and judging whether to play the operation video in the head-mounted VR device includes:
if the acquisition result of the interactive voice of the user is yes, acquiring an operation video corresponding to the target production equipment preset in the control module, and playing the operation video in the head-mounted VR equipment;
If the acquisition result of the interactive voice of the user is no, the operation video corresponding to the target production equipment preset in the control module is not acquired.
Specifically, the writing the name of the target production equipment into the outline corresponding to the target production equipment includes:
And writing the name of the target production equipment into the current frame image through an opencv library of python, wherein the preset position corresponds to the centroid coordinate of the first contour, the preset position is the upper left corner coordinate of a text box corresponding to the name by taking the centroid coordinate of the first contour as the centroid coordinate of the first contour, and the text corresponding to the name of the target production equipment is inserted.
Specifically, the method further comprises the following steps:
Acquiring GPS coordinates of a camera by using a GPS positioning system arranged on the head-mounted VR equipment, measuring first height data of the camera from the ground by using a laser ranging system arranged on the head-mounted VR equipment, acquiring internal reference data and a holder angle of the camera, and calculating geographic coordinates for indicating target production equipment according to centroid coordinates of a first contour, the GPS coordinates of the camera, the first height data, the internal reference data and the holder angle; obtaining preset geographic coordinates corresponding to each sample production device preset in the control module, calculating actual distances for indicating the geographic coordinates of the target production device and the preset geographic coordinates corresponding to each sample production device, judging whether the shortest actual distance is smaller than or equal to a second preset threshold value among a plurality of actual distances corresponding to each sample production device, and determining the name of the target production device according to a judging result.
Specifically, the method further comprises the following steps:
if the shortest actual distance is smaller than or equal to a second preset threshold value, the name of the sample production equipment corresponding to the shortest actual distance is the name of the target production equipment;
If the shortest practical distance is larger than the second preset threshold, outputting a voice prompt signal related to the failure of matching and the adjustment of the shooting angle and the shooting position.
It should be noted that, if the shortest actual distance is smaller than or equal to the second preset threshold, it is proved that the centroid coordinate corresponding to the first contour is close enough to the preset geographic coordinate corresponding to the sample production device, and the position of the target production device is proved to be the position of the sample production device, and the name of the sample production device is judged to be the name of the target production device, otherwise, it is unable to prove that the position of the target production device is the position of the sample production device, at this time, a voice prompt signal about failure in matching and adjustment of the shooting angle and shooting position is output, so that the intelligent degree and usability of the invention are further improved.
It can be understood that the present invention obtains the current frame image of the target production device photographed by the VR camera on the head-mounted VR device; processing the current frame image, and extracting the outline of the target production equipment from the current frame image; determining the name of the target production equipment according to the outline of the target production equipment in the current frame image; writing the name of the target production equipment into the interior of the outline corresponding to the target production equipment; outputting a voice interaction signal related to whether to play the operation video of the target production equipment to a user, acquiring interaction voice of the user, judging whether to acquire the operation video corresponding to the target production equipment preset in the control module according to the interaction voice of the user, and judging whether to play the operation video in the head-mounted VR equipment. The operation video and production skills of the corresponding production equipment can be automatically learned through the head-mounted VR equipment without consulting the data of the user, so that a large amount of time cost is saved, the learning efficiency is improved to a great extent, complex algorithm modeling is not needed, the intelligent degree and usability of the invention are improved to a great extent, and the application scene of the invention is expanded to a great extent.
Referring to fig. 2, another embodiment of the present invention provides an intelligent learning system for production skills based on image processing, including:
an acquiring module 100, configured to acquire a current frame image of a target production device captured by a VR camera on a head-mounted VR device;
The control module 200 is used for processing the current frame image and extracting the outline of the target production equipment from the current frame image; or determining the name of the target production equipment according to the outline of the target production equipment in the current frame image; writing the name of the target production equipment into the interior of the outline corresponding to the target production equipment; or the method is used for outputting a voice interaction signal related to whether to play the operation video of the target production equipment to the user, acquiring the interaction voice of the user, judging whether to acquire the operation video corresponding to the target production equipment preset in the control module according to the interaction voice of the user, and judging whether to play the operation video in the head-mounted VR equipment.
The method and the device have the advantages that the current frame image of the target production device, which is shot by the VR camera on the head-mounted VR device, is acquired; processing the current frame image, and extracting the outline of the target production equipment from the current frame image; determining the name of the target production equipment according to the outline of the target production equipment in the current frame image; writing the name of the target production equipment into the interior of the outline corresponding to the target production equipment; outputting a voice interaction signal related to whether to play the operation video of the target production equipment to a user, acquiring interaction voice of the user, judging whether to acquire the operation video corresponding to the target production equipment preset in the control module according to the interaction voice of the user, and judging whether to play the operation video in the head-mounted VR equipment. The operation video and production skills of the corresponding production equipment can be learned by the head-mounted VR equipment without consulting the data by a user, so that a great amount of time cost is saved, complex algorithm modeling is not needed, the intelligent degree and usability of the invention are improved to a great extent, and the application scene of the invention is expanded to a great extent.
In a preferred embodiment, the present application also provides an electronic device, including:
A memory; and a processor, wherein the memory stores computer readable instructions, and the computer readable instructions realize the intelligent learning method of the production skills based on image processing when being executed by the processor. The computer device may be broadly a server, a terminal, or any other electronic device having the necessary computing and/or processing capabilities. In one embodiment, the computer device may include a processor, memory, network interface, communication interface, etc. connected by a system bus. The processor of the computer device may be used to provide the necessary computing, processing and/or control capabilities. The memory of the computer device may include a non-volatile storage medium and an internal memory. The non-volatile storage medium may have an operating system, computer programs, etc. stored therein or thereon. The internal memory may provide an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface and communication interface of the computer device may be used to connect and communicate with external devices via a network. Which when executed by a processor performs the steps of the method of the invention.
The present invention may be implemented as a computer readable storage medium having stored thereon a computer program which, when executed by a processor, causes steps of a method of an embodiment of the present invention to be performed. In one embodiment, the computer program is distributed over a plurality of computer devices or processors coupled by a network such that the computer program is stored, accessed, and executed by one or more computer devices or processors in a distributed fashion. A single method step/operation, or two or more method steps/operations, may be performed by a single computer device or processor, or by two or more computer devices or processors. One or more method steps/operations may be performed by one or more computer devices or processors, and one or more other method steps/operations may be performed by one or more other computer devices or processors. One or more computer devices or processors may perform a single method step/operation or two or more method steps/operations.
Those of ordinary skill in the art will appreciate that the method steps of the present invention may be implemented by a computer program, which may be stored on a non-transitory computer readable storage medium, to instruct related hardware such as a computer device or a processor, which when executed causes the steps of the present invention to be performed. Any reference herein to memory, storage, database, or other medium may include non-volatile and/or volatile memory, as the case may be. Examples of nonvolatile memory include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage, hard disk, solid state disk, and the like. Examples of volatile memory include Random Access Memory (RAM), external cache memory, and the like.
It can be understood that the present invention obtains the current frame image of the target production device photographed by the VR camera on the head-mounted VR device; processing the current frame image, and extracting the outline of the target production equipment from the current frame image; determining the name of the target production equipment according to the outline of the target production equipment in the current frame image; writing the name of the target production equipment into the interior of the outline corresponding to the target production equipment; outputting a voice interaction signal related to whether to play the operation video of the target production equipment to a user, acquiring interaction voice of the user, judging whether to acquire the operation video corresponding to the target production equipment preset in the control module according to the interaction voice of the user, and judging whether to play the operation video in the head-mounted VR equipment. The operation video and production skills of the corresponding production equipment can be automatically learned through the head-mounted VR equipment without consulting the data of the user, so that a large amount of time cost is saved, the learning efficiency is improved to a great extent, complex algorithm modeling is not needed, the intelligent degree and usability of the invention are improved to a great extent, and the application scene of the invention is expanded to a great extent.
The technical features described above may be arbitrarily combined. Although not all possible combinations of features are described, any combination of features should be considered to be covered by the description provided that such combinations are not inconsistent.
The above-described embodiments of the present invention do not limit the scope of the present invention. Any other corresponding changes and modifications made in accordance with the technical idea of the present invention shall be included in the scope of the claims of the present invention.
Claims (3)
1. An intelligent learning method for production skills based on image processing, which is characterized by comprising the following steps:
s100, acquiring a current frame image of a target production device shot by a VR camera on a head-mounted VR device; processing the current frame image, and extracting the outline of the target production equipment from the current frame image;
S200, determining the name of the target production equipment according to the outline of the target production equipment in the current frame image; writing the name of the target production equipment into the interior of the outline corresponding to the target production equipment;
s300, outputting a voice interaction signal related to whether to play an operation video of a target production device to a user, acquiring interaction voice of the user, judging whether to acquire the operation video corresponding to the target production device preset in a control module according to the interaction voice of the user, and judging whether to play the operation video in a head-mounted VR device;
The extracting the outline of the target production equipment in the current frame image comprises the following steps:
Performing binarization processing on the current frame image, converting the current frame image into an image only comprising black and white colors so as to distinguish a target production equipment area and a background area in the current frame image, extracting contour feature points from the target production equipment area of the current frame image after the binarization processing, and fitting according to the contour feature points to obtain a first contour of target production equipment to be matched;
The determining the name of the target production equipment according to the outline of the target production equipment in the current frame image comprises the following steps:
Acquiring second contours of all sample production equipment preset in a control module, calculating mass center coordinates of the first contours and the second contours, rotating and scaling the first contours after the mass centers of the first contours and the second contours are overlapped, enabling the difference between Euclidean distances of all vertexes of the first contours and the second contours to be minimum, sequentially calculating contour coincidence degree data of the first contours and the second contours of all sample production equipment at the moment, and determining names of target production equipment according to the contour coincidence degree data;
The determining the name of the target production equipment according to the contour coincidence degree data comprises the following steps:
Sequentially judging whether the contour coincidence degree data of the first contour and the second contour of each sample production device is larger than or equal to a first preset threshold value, if the contour coincidence degree data of the first contour and the second contour of a certain sample production device is larger than or equal to the first preset threshold value, judging that the name of the sample production device is the name of the target production device; if the contour coincidence ratio data of the first contour and the second contour of each sample production device are smaller than a first preset threshold value, outputting a voice prompt signal related to failure in matching and adjustment of shooting angles;
Judging whether to acquire an operation video corresponding to target production equipment preset in a control module according to interactive voice of a user, and judging whether to play the operation video in head-mounted VR equipment, wherein the method comprises the following steps:
if the acquisition result of the interactive voice of the user is yes, acquiring an operation video corresponding to the target production equipment preset in the control module, and playing the operation video in the head-mounted VR equipment;
if the acquisition result of the interactive voice of the user is no, not acquiring the operation video corresponding to the target production equipment preset in the control module;
The writing the name of the target production equipment into the interior of the outline corresponding to the target production equipment comprises the following steps:
writing the name of the target production equipment into a current frame image through an opencv library of python, wherein the preset position corresponds to the centroid coordinate of the first contour, the preset position is the upper left corner coordinate of a text box corresponding to the name by taking the centroid coordinate of the first contour as the centroid coordinate of the first contour, and inserting a text corresponding to the name of the target production equipment;
The method further comprises the steps of:
Acquiring GPS coordinates of a camera by using a GPS positioning system arranged on the head-mounted VR equipment, measuring first height data of the camera from the ground by using a laser ranging system arranged on the head-mounted VR equipment, acquiring internal reference data and a holder angle of the camera, and calculating geographic coordinates for indicating target production equipment according to centroid coordinates of a first contour, the GPS coordinates of the camera, the first height data, the internal reference data and the holder angle; acquiring preset geographic coordinates corresponding to each sample production device preset in a control module, calculating actual distances for indicating the geographic coordinates of the target production device and the preset geographic coordinates corresponding to each sample production device, judging whether the shortest actual distance is smaller than or equal to a second preset threshold value among a plurality of actual distances corresponding to each sample production device, and determining the name of the target production device according to a judging result;
The method further comprises the steps of:
if the shortest actual distance is smaller than or equal to a second preset threshold value, the name of the sample production equipment corresponding to the shortest actual distance is the name of the target production equipment;
if the shortest actual distance is greater than a second preset threshold, outputting voice prompt signals related to matching failure, requesting to adjust shooting angles and shooting positions;
the method for calculating the centroid coordinates comprises the following steps:
Respectively establishing a target search box according to the first contour and the second contour, wherein the target search box needs to comprise all contour feature points of the first contour and the second contour;
Carrying out gray level processing on a current frame image, carrying out piecewise linear change on the gray level value of the original current frame image according to the mean value and standard deviation of the gray level value of the original current frame image to obtain the gray level value of each point of an 8-bit single-channel image of the current frame image, and calculating according to the gray level value of each point of the 8-bit single-channel image of the current frame image to obtain a centroid coordinate;
Specifically, the calculating, according to the centroid coordinate of the first contour, the GPS coordinate of the camera, the first height data, the internal reference data, and the pan-tilt angle, the geographic coordinate for indicating the target production device includes:
according to the first height, the internal reference data and the holder angle, calculating pixel coordinates and geographic coordinates of four vertexes and center points contained in the current frame image, according to the geographic coordinates and pixel coordinates of the four vertexes and the center points, calculating a homography matrix for converting the geographic coordinates and the pixel coordinates, and according to the centroid coordinates and the homography matrix of the first contour, calculating geographic coordinates for indicating target production equipment.
2. An intelligent learning system for production skills based on image processing, characterized in that the intelligent learning method for production skills based on image processing according to claim 1 is adopted, comprising:
the acquisition module is used for acquiring a current frame image of the target production equipment shot by the VR camera on the head-mounted VR equipment;
The control module is used for processing the current frame image and extracting the outline of the target production equipment from the current frame image; or determining the name of the target production equipment according to the outline of the target production equipment in the current frame image; writing the name of the target production equipment into the interior of the outline corresponding to the target production equipment; or the method is used for outputting a voice interaction signal related to whether to play the operation video of the target production equipment to the user, acquiring the interaction voice of the user, judging whether to acquire the operation video corresponding to the target production equipment preset in the control module according to the interaction voice of the user, and judging whether to play the operation video in the head-mounted VR equipment.
3. An electronic device, comprising:
A memory; and a processor having stored thereon computer readable instructions which when executed by the processor implement the image processing based production skills intelligent learning method of claim 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410244508.XA CN117827013B (en) | 2024-03-05 | 2024-03-05 | Intelligent learning method and system for production skills based on image processing and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410244508.XA CN117827013B (en) | 2024-03-05 | 2024-03-05 | Intelligent learning method and system for production skills based on image processing and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117827013A CN117827013A (en) | 2024-04-05 |
| CN117827013B true CN117827013B (en) | 2024-05-14 |
Family
ID=90515563
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410244508.XA Active CN117827013B (en) | 2024-03-05 | 2024-03-05 | Intelligent learning method and system for production skills based on image processing and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117827013B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111970568A (en) * | 2020-08-31 | 2020-11-20 | 上海松鼠课堂人工智能科技有限公司 | Method and system for interactive video playing |
| CN114625251A (en) * | 2022-03-11 | 2022-06-14 | 平安普惠企业管理有限公司 | Interaction method and device based on VR, computer equipment and storage medium |
| CN116009529A (en) * | 2022-11-11 | 2023-04-25 | 青岛杰瑞自动化有限公司 | Control method and system for patrol robot in petroleum exploration area and electronic equipment |
| CN116012732A (en) * | 2022-12-08 | 2023-04-25 | 亿咖通(湖北)技术有限公司 | Video generation method, device, equipment and storage medium |
| CN116744065A (en) * | 2023-05-04 | 2023-09-12 | 维沃移动通信有限公司 | Video playing method and device |
| WO2023207759A1 (en) * | 2022-04-25 | 2023-11-02 | 漳州松霖智能家居有限公司 | User posture detection method, intelligent cushion system, and related device |
| CN117336457A (en) * | 2023-10-17 | 2024-01-02 | 深圳市惠浦众嘉科技有限公司 | Intelligent multifunctional LED display screen with human screen interaction and implementation method thereof |
| CN117496984A (en) * | 2023-05-05 | 2024-02-02 | 马上消费金融股份有限公司 | Interaction method, device and equipment of target object and readable storage medium |
| CN117576778A (en) * | 2023-10-27 | 2024-02-20 | 青岛鹏海软件有限公司 | Factory abnormal behavior monitoring method and system based on video stream and electronic equipment |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2996017B1 (en) * | 2014-09-11 | 2022-05-11 | Nokia Technologies Oy | Method, apparatus and computer program for displaying an image of a physical keyboard on a head mountable display |
| CN112927259A (en) * | 2021-02-18 | 2021-06-08 | 青岛小鸟看看科技有限公司 | Multi-camera-based bare hand tracking display method, device and system |
-
2024
- 2024-03-05 CN CN202410244508.XA patent/CN117827013B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111970568A (en) * | 2020-08-31 | 2020-11-20 | 上海松鼠课堂人工智能科技有限公司 | Method and system for interactive video playing |
| CN114625251A (en) * | 2022-03-11 | 2022-06-14 | 平安普惠企业管理有限公司 | Interaction method and device based on VR, computer equipment and storage medium |
| WO2023207759A1 (en) * | 2022-04-25 | 2023-11-02 | 漳州松霖智能家居有限公司 | User posture detection method, intelligent cushion system, and related device |
| CN116009529A (en) * | 2022-11-11 | 2023-04-25 | 青岛杰瑞自动化有限公司 | Control method and system for patrol robot in petroleum exploration area and electronic equipment |
| CN116012732A (en) * | 2022-12-08 | 2023-04-25 | 亿咖通(湖北)技术有限公司 | Video generation method, device, equipment and storage medium |
| CN116744065A (en) * | 2023-05-04 | 2023-09-12 | 维沃移动通信有限公司 | Video playing method and device |
| CN117496984A (en) * | 2023-05-05 | 2024-02-02 | 马上消费金融股份有限公司 | Interaction method, device and equipment of target object and readable storage medium |
| CN117336457A (en) * | 2023-10-17 | 2024-01-02 | 深圳市惠浦众嘉科技有限公司 | Intelligent multifunctional LED display screen with human screen interaction and implementation method thereof |
| CN117576778A (en) * | 2023-10-27 | 2024-02-20 | 青岛鹏海软件有限公司 | Factory abnormal behavior monitoring method and system based on video stream and electronic equipment |
Non-Patent Citations (3)
| Title |
|---|
| InteractiveDisplay: A computer-vision-based solution to retrofit existing flat displays into interactive surfaces;Lahiru L. Priyadarshana；Kulari De Silva Lokuge;《 2011 IEEE Symposium on Computers & Informatics (ISCI)》;20110811;第622-627页 * |
| 基于树莓派的穿戴式智能眼镜***设计与实现;李旭东;陆朝铨;桂彦;张建明;;软件;20160815(08);第45-48页 * |
| 民航航班跟踪视频关键帧提取方法研究;宁煜西;周铭;李广强;王宁;;空军预警学院学报;20180615(03);第52-57页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117827013A (en) | 2024-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110568447B (en) | Visual positioning method, device and computer readable medium | |
| CN110645986B (en) | Positioning method and device, terminal and storage medium | |
| CN109960742B (en) | Local information searching method and device | |
| CN109520500B (en) | Accurate positioning and street view library acquisition method based on terminal shooting image matching | |
| CN111179358A (en) | Calibration method, device, equipment and storage medium | |
| US10839599B2 (en) | Method and device for three-dimensional modeling | |
| CN112150548B (en) | Positioning method and device, terminal and storage medium | |
| CN112541484B (en) | Face matting method, system, electronic device and storage medium | |
| CN110634138A (en) | Bridge deformation monitoring method, device and equipment based on visual perception | |
| CN110361005A (en) | Positioning method, positioning device, readable storage medium and electronic equipment | |
| CN114387199A (en) | Image annotation method and device | |
| CN117827013B (en) | Intelligent learning method and system for production skills based on image processing and electronic equipment | |
| CN113298867A (en) | Accurate positioning method and device for ground object target position based on line matching and storage medium | |
| CN112907657A (en) | Robot repositioning method, device, equipment and storage medium | |
| CN110880003B (en) | Image matching method and device, storage medium and automobile | |
| CN117253022A (en) | Object identification method, device and inspection equipment | |
| CN117132649A (en) | Ship video positioning method and device for artificial intelligent Beidou satellite navigation fusion | |
| CN115797291A (en) | Circuit terminal identification method and device, computer equipment and storage medium | |
| CN113297344B (en) | Three-dimensional remote sensing image-based ground linear matching method and device and ground object target position positioning method | |
| CN115311336A (en) | Image registration method, device and equipment of multiple cameras and storage medium | |
| CN114782692A (en) | House model repairing method and device, electronic equipment and readable storage medium | |
| CN110196638B (en) | Mobile terminal augmented reality method and system based on target detection and space projection | |
| CN116266402A (en) | Automatic object labeling method and device, electronic equipment and storage medium | |
| US20240221200A1 (en) | Method for identifying depths of images and related device | |
| CN116433769B (en) | Space calibration method, device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |