CN106636510B - It is a kind of that pricker method is moved based on machine vision certainly - Google Patents
It is a kind of that pricker method is moved based on machine vision certainly Download PDFInfo
- Publication number
- CN106636510B CN106636510B CN201610914755.1A CN201610914755A CN106636510B CN 106636510 B CN106636510 B CN 106636510B CN 201610914755 A CN201610914755 A CN 201610914755A CN 106636510 B CN106636510 B CN 106636510B
- Authority
- CN
- China
- Prior art keywords
- image
- drill steel
- target
- pricker
- detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- C—CHEMISTRY; METALLURGY
- C21—METALLURGY OF IRON
- C21B—MANUFACTURE OF IRON OR STEEL
- C21B7/00—Blast furnaces
- C21B7/12—Opening or sealing the tap holes
Landscapes
- Engineering & Computer Science (AREA)
- Chemical & Material Sciences (AREA)
- Manufacturing & Machinery (AREA)
- Materials Engineering (AREA)
- Metallurgy (AREA)
- Organic Chemistry (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
It is a kind of that pricker method is moved based on machine vision certainly, comprising the following steps: mounting industrial camera, embeded processor, the image acquisition region of industry camera cover the region of manipulator, drill steel.It is handled according to industry camera acquired image, using gaussian filtering, sharpening, edge detection, straight-line detection, separates, detect the position of drill steel from background.Whether judge drill steel " in place ".Using target detection and target tracking algorism, the position of mechanical gripping arm, and the motion state of real-time tracking manipulator are detected, judge the relative positional relationship of manipulator and drill steel, according to the coverage extent of the two, whether judge manipulator " in place ".The result can will test, analyzed is transferred to field controller, controls pricker device all parts are moved certainly.The present invention is a kind of to move pricker method based on machine vision certainly, not only meets the generation requirement of automation equipment, also improves mechanization, the automatization level of casthouse equipment.
Description
Technical field
The present invention be it is a kind of pricker method is moved based on machine vision certainly, belong to technical field of metallurgical equipment machine vision skill
Art field.
Background technique
Iron notch drill is a kind of commonly used equipment of iron-smelter, and function mainly utilizes drill rod to open blast furnace taphole.With
The requirement of the development of blast furnace enlargement and industry 4.0, the requirement to the mechanization of casthouse equipment, Automated water product is increasingly
It is high.And in current blast furnace steel-making factory, mostly still using manually changing/unload drill rod, a small number of large steel-making plants use from
Move pricker machine during use and there are problems that following 3: (1) due to the length of drill steel and the factors such as distribution of weight is unbalanced
Drill steel is caused skew occur, to cause mechanical gripping arm that cannot normally clamp drill steel.(2) it is influenced by live adverse circumstances, machine
Tool gripping arm, which may be run, not in place there is the phenomenon that uses leisure moments.(3) moving pricker machine certainly is to execute the operation of dress pricker or unload pricker operation, at present
Still using artificial judgment, it is not able to achieve comprehensive automation, intelligence.
Summary of the invention
In order to improve the labor intensity and working environment of stokehold worker, further increase from move pricker machine reliability,
Intelligence, the present invention provide it is a kind of pricker method is moved based on machine vision certainly, the generation for not only meeting automation equipment is wanted
It asks, also improves mechanization, the automatization level of casthouse equipment.
The technical scheme adopted by the invention is as follows:
Step 1: mounting industrial camera, embeded processor, the image acquisition region of industry camera cover manipulator,
The region of drill steel.
Step 2: it being handled according to industry camera acquired image, is examined using gaussian filtering, sharpening, edge
It surveys, straight-line detection, separates, detects the position of drill steel from background.
Step 3: whether judging drill steel " in place ".
Step 4: utilizing target detection and target tracking algorism, detect the position of mechanical gripping arm, and real-time tracking is mechanical
The motion state of hand judges the relative positional relationship of manipulator and drill steel, according to the coverage extent of the two, whether judges manipulator
" in place ".
Step 5: can will test, the result analyzed is transferred to field controller, be carried out to pricker device all parts are moved certainly
Control.
The present invention is a kind of to move pricker method based on machine vision certainly, and technical effect is as follows:
1: whether drill steel has been equipped with by Machine Vision Detection iron notch drill, for machinery arm controller provide " dress pricker " and
" unloading pricker " operation provides foundation.
2: whether the drill steel in detection " pricker frame " is in " in place " state；Otherwise signal is provided so as to control system driving
" positioning " component on " pricker frame " works to drill steel " in place ".
3: detect mechanical gripping arm on " pricker frame " drill steel and the relative position between " in place " drill steel, be machinery gripping arm
Controller provides the movements such as " start and stop " and " crawl " and provides foundation.
4: the present invention not only meets the generation requirement of automation equipment, also improves the mechanization, automatic of casthouse equipment
Change horizontal.
Detailed description of the invention
Fig. 1 is hardware configuration schematic diagram of the invention.
Fig. 2 is industry camera installation site and its visual field schematic diagram of the invention.
Fig. 3 is that industry camera of the invention shoots image schematic diagram.
Fig. 4 is manipulator " in place " schematic diagram of the invention.
Fig. 5 moves pricker machine working condition transition graph for of the invention certainly.
Fig. 6 is embedded software system structure chart of the invention.
Specific embodiment
It is a kind of that pricker method is moved based on machine vision certainly, comprising the following steps:
Step 1: mounting industrial camera, embeded processor, the image acquisition region of industry camera cover manipulator,
The region of drill steel.
Step 2: it being handled according to industry camera acquired image, is examined using gaussian filtering, sharpening, edge
It surveys, straight-line detection, separates, detects the position of drill steel 8 from background.
Gaussian filtering: being a kind of linear smoothing filtering, is suitable for eliminating Gaussian noise, is widely used in subtracting for image procossing
It makes an uproar process.Popular says, gaussian filtering is exactly the process being weighted and averaged to entire image, the value of each pixel, all
It is obtained after being weighted averagely by other pixel values in itself and neighborhood.The concrete operations of gaussian filtering are: with a mould
Each of plate (or convolution, mask) scan image pixel, with the weighted average gray scale of pixel in the determining neighborhood of template
Value goes the value of alternate template central pixel point.
Sharpen: using sharpening tool can quickly focus blur edge, improve image in a certain position clarity or
Focal length degree keeps the color of image specific region distincter.
Edge detection: being the basic problem in image procossing and computer vision, and the purpose is to bright in reference numbers image
Degree changes apparent point.Significant changes in image attributes usually reflect the critical event and variation of attribute, these include (i)
Discontinuous, (ii) surface direction in depth is discontinuous, the variation of (iii) material property and the variation of (iv) scene lighting.Edge inspection
Survey is a research field in image procossing and computer vision, especially in feature extraction.
Straight-line detection: detecting straight line in original image, uses Hough straight-line detection here, thought is, in original
A point under beginning image coordinate system has corresponded to the straight line in parameter coordinate system, the straight line pair of same parameter coordinate system
A point under original coordinate system has been answered, then, all the points of straight line have been presented under original coordinate system, their slope and intercept is
It is identical, so they correspond to the same point in the parametric coordinate system.It is projected in this way by each point under original coordinate system
After under to parameter coordinate system, see that, either with or without accumulation point under parameter coordinate system, such accumulation point has just corresponded to original coordinate system
Under straight line.
Step 3: whether judging drill steel " in place ".
Step 4: utilizing target detection and target tracking algorism, detect the position of mechanical gripping arm, and real-time tracking is mechanical
The motion state of hand 4 judges the relative position of manipulator Yu drill steel to be taken in shooting area, if manipulator is complete
Drill steel to be taken is blocked, then can be considered manipulator " in place ".
Target detection is also Objective extraction, refers to and mentions the foreground target of movement from background image from sequence image
It takes out.Here target detection uses frame difference method.Frame difference method mainly utilizes the variation in video sequence between two continuous frames to examine
Survey the moving target under static scene, it is assumed that fk(x, y) and fk+1(x, y) is respectively+1 frame of kth frame and kth in image sequence
The pixel value of middle pixel (x, y), then the error image of this two field pictures is as shown in following equation:
Diffk+1(x, y)=| fk(x,y)-fk+1(x,y)| (1-1)
Difference does not represent the region (background pixel passed through by the movement of moving target by 0 image-region in above formula
It is worth constant), and because of adjacent video interframe time interval very little, target position changes also very little.So the movement of moving target
The region passed through also just represents the region in present frame where moving target.Target can be extracted using this principle.Frame
The basic procedure of poor method: (1) using 1-1 formula the error image Diff of kth frame and kth+1 frame is obtainedk+1；(2) to obtained difference
It is worth image Diffk+1Binaryzation obtains Qk+1；
It (3) is the interference for eliminating small noise, the moving target made is more acurrate, to Qk+1Carry out it is necessary filtering and
Denoising, post-processing result are Mk+1。
Target following is the process of position of the determining same object in the different frame of image sequence, and tracking mode can divide
For the tracking based on segmentation and based on the tracking of model.Tracking based on segmentation is will to be partitioned into all fortune in each frame
Then moving-target is matched with tracked target, to achieve the purpose that tracking.Tracking based on model only need to be in first frame
In tracked target selected by manual or automatic mode, then analyze the color of target, Texture eigenvalue, construct target mould
Plate does not need to extract moving target again in subsequent frames, only need to go out the target template i.e. by certain decision search in the picture
Realize the tracking to target.Mean Shift is exactly a kind of tracking based on model, using color or grey level histogram
The statistical property of distribution describes target signature.To pixel all in initial frame objective area in image, it is empty to calculate feature
Between in each characteristic value probability, that is, establish target template.To there may be the candidate regions of target in later every frame image
Domain calculates characteristic value, utilizes the similitude of similarity function metric objective template and present frame candidate region.By seeking similitude
Function maxima obtains the Mean Shift vector about target, this vector is exactly that target turns from initial position to correct position
The vector of shifting can find the correct position of target according to this vector.
The target in image is described with the distribution of color of a target, target's center is located at x0, then object module
It is expressed as vectorWherein:
WhereinIt is indicated with y
The center of candidate target, then candidate family is expressed as vectorWherein:
Wherein
K [] is kernel function in two above formula, and m is the number of color characteristic in color space, and u is the spy of color characteristic
Value indicative, b (xi) it is x in imageiPixel value (color value) at point.Target following can simplify to find optimal y so that q with
P (y) is utmostly similar.
Step 5: can will test, the result analyzed is transferred to field controller PLC, to move certainly pricker device all parts into
Row control.
It is a kind of that pricker method is moved based on machine vision certainly, further includes: by being industry camera configuration in 7 position of pricker frame
One holder, after holder moves to station, by field controller control holder movement, by the shooting visual field 5 of industry camera
It is moved to the drill steel installation site of iron notch drill, sends the picture that industry camera is shot in embeded processor, passes through
Gaussian filtering, sharpening, Sobel operator edge detection and Hough straight-line detection.Image passes through after a series of processing, in figure
Existing straight line can be labeled out, and there are straight lines at iron notch drill, that is, represent and assembled drill steel 8.
Gaussian filtering: being a kind of linear smoothing filtering, is suitable for eliminating Gaussian noise, is widely used in subtracting for image procossing
It makes an uproar process.Popular says, gaussian filtering is exactly the process being weighted and averaged to entire image, the value of each pixel, all
It is obtained after being weighted averagely by other pixel values in itself and neighborhood.The concrete operations of gaussian filtering are: with a mould
Each of plate (or convolution, mask) scan image pixel, with the weighted average gray scale of pixel in the determining neighborhood of template
Value goes the value of alternate template central pixel point.
Sharpen: using sharpening tool can quickly focus blur edge, improve image in a certain position clarity or
Focal length degree keeps the color of image specific region distincter.
Sobel operator edge detection: in edge detection, common template is Sobel operator, and technically, it is one
A discrete first difference operator, for calculate brightness of image function First-order Gradient approximation.In any point of image
Use this operator, it will generate the corresponding gradient vector of point or its law vector.The operator includes the matrix of two groups of 3x3, point
Not Wei transverse direction and longitudinal direction, it is made into planar convolution with image, can obtain the brightness difference approximation of transverse direction and longitudinal direction respectively.Such as
Fruit represents original image with A, and Gx and Gy respectively represent the image through transverse direction and longitudinal direction edge detection, and formula is as follows:
The transverse direction and longitudinal direction gradient approximation of each pixel of image can be combined with formula below, to calculate gradient
Size.
Then following formula can be used to calculate gradient direction.
Hough straight-line detection: the basic principle of Hough transform is the duality using point with line, by original image sky
Between given curve negotiating curve representation form become a point of parameter space, thus given curve in original image
Test problems be converted into the spike problem found in parameter space, namely that detection overall permanence is converted into detection part is special
Property.Hough straight-line detection thought are as follows: a point under coordinates of original image coordinates system corresponded to one in parameter coordinate system it is straight
Line, the straight line of same parameter coordinate system have corresponded to a point under original coordinate system, then, under original coordinate system present straight
The all the points of line, their slope and intercept is identical, so they correspond to the same point in the parametric coordinate system.In this way
After projecting to each point under original coordinate system under parameter coordinate system, see under parameter coordinate system either with or without accumulation point, this
The accumulation point of sample has just corresponded to the straight line under original coordinate system.
Specific implementation step:
1, hardware is installed according to structure principle chart, mainly includes industry camera, embeded processor.Due to moving pricker certainly
The vision module of machine be mainly responsible for drill steel 8 on detection " pricker frame " and manipulator 4 whether " in place ", therefore image acquisition region is answered
This covers the region of manipulator 4 and drill steel as much as possible 8, as shown in Figure 2.
In Fig. 2: 1 is trolley track, and 2 be industry camera, and 3 be trolley, and 4 be manipulator, and 5 be camera view region, 6
Drill rod is taken for generation, 7 be pricker frame, and 8 be drill steel.
2, after industry camera installs, the image taken is as shown in Figure 3: according to the collected figure of industry camera
Picture, the image analysis of vision module and processing handle library using the open source computer vision based on OpenCV, using gaussian filtering,
Sharpening, Sobel operator edge detection and Hough transform straight-line detection and etc., it separated from background, detect drill steel 8
Position.
3, straight line and the angle a with reference to dotted line in Fig. 3 are calculated, wherein being that system initialisation phase detection is in reference to straight line
The drill steel 8 of " in place " state has obtained initial position.If angle a <b (wherein b is experiment threshold value), system thinks drill steel 8
" in place ", otherwise it is 0, as shown in table 1 that then setting POLE_STATUS register, which is 1,.
4, using target detection and target tracking algorism, the position of mechanical gripping arm is detected, and real-time tracking manipulator 4
Motion state judges the relative positional relationship (blocking) of manipulator 4 Yu drill steel 8, and the coverage extent according to the two judges manipulator 4
Whether " in place ", if mechanical gripping arm is in place, setting MAIN_STATUS register is 1, otherwise sets 0.Wherein mechanical arm is " just
Position " is if shown in schematic diagram 4: i.e. when d '-Δ≤d≤d ', system thinks mechanical gripping arm in place, and wherein d ' be to test to test
Optimal value, Δ are tolerance.
5, the present invention is also equipped with the function that drill steel whether has been assembled on monitoring iron notch drill.By in pricker frame 7
Position is that camera configures a holder, after walking carriage moves to station, by field controller control holder movement, will be taken the photograph
The shooting visual field of camera is moved to the drill steel installation site of iron notch drill, sends embedded processing for the picture that camera is shot
In device, by gaussian filtering, sharpening, Sobel operator edge detection and Hough straight-line detection, it whether there is in detection image
Special characteristic straight line.
6, MODBUS communication protocol:
The present invention is using MODBUS standard for Fieldbus, and the RTU data format of MODBUS bus uses 2 institute of table
Show.
1 holding register explanation of table
2 MODBUS RTU data frame format of table
7, apparatus of the present invention include two parts of hardware and software: hardware components are mainly by industry camera and embedded
Manage device two parts composition；Software section mainly includes Image Acquisition, MODBUS protocol stack, image analysis, processing, parameter setting,
Debugging and task schedule.
Claims (1)
1. a kind of move pricker method based on machine vision certainly, it is characterised in that the following steps are included:
Step 1: mounting industrial camera, embeded processor, the image acquisition region of industry camera cover manipulator (4),
The region of drill steel (8)；
Step 2: it is handled according to industry camera acquired image, using gaussian filtering, sharpening, edge detection,
Straight-line detection step separates from background, detects the position of drill steel (8)；
Step 3: judge drill steel (8) whether " in place "；
Calculate straight line and the angle a with reference to dotted line, wherein be in " in place " state with reference to dotted line for system initialisation phase detection
The obtained initial position of drill steel (8), if angle a <b, wherein b is experiment threshold value, then system thinks drill steel (8) " just
Position ", otherwise it is 0 that then setting POLE_STATUS register, which is 1,；
Step 4: utilizing target detection and target tracking algorism, detect the position of mechanical gripping arm, and real-time tracking manipulator (4)
Motion state, judge the relative positional relationship of manipulator (4) Yu drill steel (8), according to both coverage extent, judge manipulator
(4) whether " in place "；
If in place, setting MAIN_STATUS register is 1 for mechanical gripping arm, 0 is otherwise set, wherein when d '-Δ≤d≤d ', be
System thinks mechanical gripping arm in place, and wherein d ' be the optimal value that experiment is tested, and Δ is tolerance；
Target detection refers to extracts the foreground target of movement from sequence image from background image, here target detection
Using frame difference method；Frame difference method is mainly to detect the movement mesh under static scene using the variation between two continuous frames in video sequence
Mark, fk(x, y) and fk+1(x, y) is respectively the pixel value of pixel (x, y) in+1 frame of kth frame and kth in image sequence, then
The error image of this two field pictures is as shown in following equation:
Diffk+1(x, y)=| fk(x,y)-fk+1(x,y)| (1-1)
In above formula, difference does not represent the region passed through by the movement of moving target by 0 image-region, and background pixel value is not
Become, and because of adjacent video interframe time interval very little, target position changes also very little；So the movement of moving target is passed through
Region also just represent the region in present frame where moving target；Target can be extracted using this principle；
The basic procedure of frame difference method: (1) using 1-1 formula the error image Diff of kth frame and kth+1 frame is obtainedk+1；(2) to gained
The error image Diff arrivedk+1Binaryzation obtains Qk+1；
It (3) is the interference for eliminating small noise, the moving target made is more acurrate, to Qk+1Carry out necessary filtering and denoising
Processing, post-processing result are Mk+1；
Step 5: will test, the result analyzed is transferred to field controller, control pricker all parts are moved certainly；
This moves pricker method certainly further include: by being that industry camera configures a holder in pricker frame (7) position, when holder moves
To after station, by field controller control holder movement, the shooting visual field (5) of industry camera is moved to the pricker of iron notch drill
Bar (8) installation site, by industry camera shoot picture be sent in embeded processor, by gaussian filtering, sharpening,
Sobel operator edge detection and Hough straight-line detection whether there is special characteristic straight line in detection image, so that monitoring is opened
Whether drill steel is assembled on iron mouth machine.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610914755.1A CN106636510B (en) | 2016-10-20 | 2016-10-20 | It is a kind of that pricker method is moved based on machine vision certainly |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610914755.1A CN106636510B (en) | 2016-10-20 | 2016-10-20 | It is a kind of that pricker method is moved based on machine vision certainly |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106636510A CN106636510A (en) | 2017-05-10 |
| CN106636510B true CN106636510B (en) | 2019-07-30 |
Family
ID=58856892
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610914755.1A Active CN106636510B (en) | 2016-10-20 | 2016-10-20 | It is a kind of that pricker method is moved based on machine vision certainly |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106636510B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116083669A (en) * | 2023-01-03 | 2023-05-09 | 中冶南方工程技术有限公司 | Method for automatically replacing iron rod in front of blast furnace |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105759720A (en) * | 2016-04-29 | 2016-07-13 | 中南大学 | Mechanical arm tracking and positioning on-line identification and correction method based on computer vision |
| CN105930854A (en) * | 2016-04-19 | 2016-09-07 | 东华大学 | Manipulator visual system |
-
2016
- 2016-10-20 CN CN201610914755.1A patent/CN106636510B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105930854A (en) * | 2016-04-19 | 2016-09-07 | 东华大学 | Manipulator visual system |
| CN105759720A (en) * | 2016-04-29 | 2016-07-13 | 中南大学 | Mechanical arm tracking and positioning on-line identification and correction method based on computer vision |
Non-Patent Citations (1)
| Title |
|---|
| 高炉炉前结合工业机器人技术的控制设计;贾祥瑞;《冶金电气电气应用》;20160531;第35卷(第9期);第75页第7-10段，第76页第1段 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106636510A (en) | 2017-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104749184B (en) | Automatic optical detection method and system | |
| CN110315525A (en) | A kind of robot workpiece grabbing method of view-based access control model guidance | |
| JP6899189B2 (en) | Systems and methods for efficiently scoring probes in images with a vision system | |
| CN109911481B (en) | Cabin frame target visual identification and positioning method and system for metallurgical robot plugging | |
| CN110807355A (en) | Pointer instrument detection and reading identification method based on mobile robot | |
| CN105184812B (en) | A kind of pedestrian based on target following hovers detection method | |
| CN110349207A (en) | A kind of vision positioning method under complex environment | |
| CN108491851A (en) | A kind of container lockhole based on machine vision is quick to be identified and suspender method for correcting error | |
| CN108629343A (en) | A kind of license plate locating method and system based on edge detection and improvement Harris Corner Detections | |
| CN103870824B (en) | A kind of face method for catching and device during Face datection tracking | |
| CN108257185A (en) | More checkerboard angle point detection process and camera marking method | |
| CN110610150A (en) | Tracking method, device, computing equipment and medium of target moving object | |
| CN108109154A (en) | A kind of new positioning of workpiece and data capture method | |
| CN111539927B (en) | Detection method of automobile plastic assembly fastening buckle missing detection device | |
| CN109434251A (en) | A kind of weld image tracking based on particle filter | |
| CN112164050B (en) | Method and device for detecting surface defects of products on production line and storage medium | |
| CN111539259A (en) | Target object recognition method, artificial neural network training method, computer-readable storage medium, and manipulator | |
| CN107341793A (en) | A kind of target surface image processing method and device | |
| CN104749801B (en) | High Precision Automatic optical detecting method and system | |
| CN113822810A (en) | Method for positioning workpiece in three-dimensional space based on machine vision | |
| CN108582075A (en) | A kind of intelligent robot vision automation grasping system | |
| CN106381264B (en) | The a wide range of automatic positioning method of micropipette pipe needle point in the micro-injection system of robot assisted | |
| CN106636510B (en) | It is a kind of that pricker method is moved based on machine vision certainly | |
| CN109583306B (en) | Bobbin residual yarn detection method based on machine vision | |
| CN109341664A (en) | The swing state measurement method of the two-dimentional overhead crane of view-based access control model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |