CN112270329A - Accurate MARK point acquisition and identification algorithm based on multi-level algorithm fusion - Google Patents

Accurate MARK point acquisition and identification algorithm based on multi-level algorithm fusion Download PDF

Info

Publication number
CN112270329A
CN112270329A CN202011195593.3A CN202011195593A CN112270329A CN 112270329 A CN112270329 A CN 112270329A CN 202011195593 A CN202011195593 A CN 202011195593A CN 112270329 A CN112270329 A CN 112270329A
Authority
CN
China
Prior art keywords
algorithm
mark point
matching
mark
contour
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011195593.3A
Other languages
Chinese (zh)
Inventor
杨尊凯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Huawei Guochuang Electronic Technology Co ltd
Original Assignee
Beijing Huawei Guochuang Electronic Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Huawei Guochuang Electronic Technology Co ltd filed Critical Beijing Huawei Guochuang Electronic Technology Co ltd
Priority to CN202011195593.3A priority Critical patent/CN112270329A/en
Publication of CN112270329A publication Critical patent/CN112270329A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/44Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/22Matching criteria, e.g. proximity measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/25Fusion techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/80Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • HELECTRICITY
    • H05ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR
    • H05KPRINTED CIRCUITS; CASINGS OR CONSTRUCTIONAL DETAILS OF ELECTRIC APPARATUS; MANUFACTURE OF ASSEMBLAGES OF ELECTRICAL COMPONENTS
    • H05K3/00Apparatus or processes for manufacturing printed circuits
    • H05K3/30Assembling printed circuits with electric components, e.g. with resistor
    • H05K3/32Assembling printed circuits with electric components, e.g. with resistor electrically connecting electric components or wires to printed circuits
    • H05K3/34Assembling printed circuits with electric components, e.g. with resistor electrically connecting electric components or wires to printed circuits by soldering
    • H05K3/341Surface mounted components

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Manufacturing & Machinery (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Supply And Installment Of Electrical Components (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses an accurate MARK point acquisition and identification algorithm based on multi-level algorithm fusion, comprising the following steps of S1: the method comprises the steps of selecting a camera and setting light source parameters, acquiring a MARK picture through the camera, adjusting a MARK contour searching algorithm until the contour of a MARK point is clearly found out, and automatically adjusting preset parameters of a camera light source and adjusting contour searching visual algorithm parameters when all set parameters of the MARK contour searching algorithm are traversed and the clear contour of the MARK is not searched. This accurate MARK point gathers and discernment algorithm based on multi-level algorithm fusion, through from MARK point gather just prepare the various parameters when the MAKR discerns, including the camera light source, and the parameter of profile search, has resisted traditional MARK point discernment like this and has received the influence of external environment changes such as light source, through combining the high advantage of MARK point profile matching precision of image template matching rate, has improved the quality of MARK point discernment.

Description

Accurate MARK point acquisition and identification algorithm based on multi-level algorithm fusion
Technical Field
The invention relates to the technical field of SMT chip mounter equipment, in particular to an accurate MARK point acquisition and identification algorithm based on multi-level algorithm fusion.
Background
A mounter, also known as a "mounter" or a "surface mount system", is an apparatus that is disposed behind a dispenser or a screen printer and accurately places surface mount components on PCB pads by moving a mounting head in a production line. The method is divided into manual operation and full-automatic operation. The device is used for realizing high-speed and high-precision component placement, and is the most critical and complex device in the whole SMT and production. The chip mounter is a chip mounting device to be used in SMT production, and the chip mounter is developed from an early low-speed mechanical chip mounter to a high-speed optical centering chip mounter and is developed towards multifunctional and flexible connection modularization.
The chip mounter is in the first step of mounting chip component for the PCB and will position the true position of PCB, because there is slight deviation in the position of advancing the board each time, the chip mounter on the market basically all uses the way of MARK point identification, fixes the actual position and the angle deviation of PCB through two MARK points, two kinds of existing MARK point identification algorithms are specifically:
the first is an algorithm based on a template picture matching mode, a small template picture is directly stored when a picture is collected, and the template picture is used for directly matching the picture currently taken when the picture is actually matched. The disadvantage of this algorithm is that the matching accuracy is low, resulting in random overall shift of the mounted component position.
The second is based on finding dots of a specified size, since Mark points are mostly small dots. The disadvantage of this method is that the matching rate is low, and often no MARK point can be found, which results in reduced efficiency of production.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides an accurate MARK point acquisition and identification algorithm based on multi-level algorithm fusion, and solves the problems that the position of a mounted component is randomly and integrally deviated due to low identification rate and identification precision of MARK points in the prior art, and the production efficiency is reduced.
In order to achieve the purpose, the invention is realized by the following technical scheme: a multi-level algorithm fusion-based accurate MARK point acquisition and identification algorithm specifically comprises the following steps:
s1, MARK point acquisition: selecting a camera and setting light source parameters, acquiring a MARK picture through the camera, adjusting a MARK contour searching algorithm until the contour of a MARK point is clearly found out, automatically adjusting preset parameters of the camera light source and adjusting contour searching visual algorithm parameters when all set parameters of the MARK contour searching algorithm are traversed and the clear contour of the MARK is not searched, and searching the contour of the MARK point again until the clear contour of the MARK point is found, at the moment, saving the current light source and camera parameters, saving a template picture of the MARK point and contour searching parameters of the MARK point;
s2, matching and identifying MARK points: a1, in actual work, firstly moving a paster head of a chip mounter to be close to a MARK point, acquiring an image through a camera, judging whether the MARK point is close to a central point, performing rough matching by adopting a template image matching algorithm when the MARK point is not close to the central point, and obtaining XY deviation and moving if the matching is successful; a2, when the MARK point is close to the central point, adopting a MARK point contour matching recognition algorithm to perform accurate matching to obtain XY offset, and acquiring the image again to perform the next round of matching after moving the head.
Preferably, in step S2, if the XY offset finally converges to the tolerance range of the central point, the MARK point is considered to be recognized and the algorithm is ended.
Preferably, the template image matching algorithm in step S2 is a second matching error algorithm, where matching is performed twice in the second matching error algorithm, and the first matching is a rough matching. Taking interlaced array data of the template, namely quarter of the template data, carrying out interlaced array scanning matching on the searched image, namely matching in a quarter of the range of the original image, wherein the matching speed is obviously improved and the error threshold value E is greatly reduced due to the fact that the data volume is greatly reduced0
Figure BDA0002753924560000031
Wherein e0The average maximum error of each point is generally 40-50, m and n are respectively the length and width of the template, and the second matching is precise matching. At the 1 st error minimum point (i)min,jmin) In the field of (i), i.e. at a diagonal point of (i)min-1,jmin-1),(imin+1,jmin+1) And searching and matching are carried out in the rectangle to obtain the final result. Preferably, in step S2, a MARK point template image matching algorithm and a MARK point contour matching algorithm are hierarchically fused, and dynamic matching and convergence are performed to the MARK center point.
Preferably, the algorithm needs to be integrated into the software of a control system of an upper computer of the chip mounter for use.
Preferably, before the system is used, a MARK point setting page needs to be opened through a software man-machine interaction interface of a chip mounter control system, an initial light source camera parameter is adjusted, and therefore a button for starting customizing a corresponding function of a MARK point is clicked.
Advantageous effects
The invention provides an accurate MARK point acquisition and identification algorithm based on multi-level algorithm fusion. Compared with the prior art, the method has the following beneficial effects:
(1) the accurate MARK point acquisition and recognition algorithm based on multi-level algorithm fusion is characterized in that various parameters including a camera light source and a contour search parameter during the identification of the MAKR are prepared from the beginning of MARK point acquisition, so that the influence of external environment changes such as the light source on the traditional MARK point recognition is resisted.
(2) The accurate MARK point acquisition and identification algorithm based on multi-level algorithm fusion improves the quality of MARK point identification, improves the production efficiency and also improves the mounting precision by combining the advantages of high matching rate of image templates and high matching precision of MARK point profiles.
(3) According to the accurate MARK point acquisition and identification algorithm based on multi-level algorithm fusion, the corresponding algorithm is used for searching and matching when identification is carried out in the MARK point acquisition stage, so that a user can quickly know whether the set MARK point is effective or not in the acquisition stage, and the existing method usually knows whether the initially set MARK point is effective or not at all in the real production time, so that the algorithm can more efficiently and quickly find out the reasonable MARK point compared with the existing method.
Drawings
FIG. 1 is a flow chart of the MARK point acquisition system framework of the present invention;
FIG. 2 is a flow chart of the MARK point acquisition algorithm body of the present invention;
FIG. 3 is a flow chart of the MARK point identification system framework of the present invention;
FIG. 4 is a flowchart of the main body of the MARK point matching identification algorithm of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1-4, the present invention provides a technical solution: a multi-level algorithm fusion-based accurate MARK point acquisition and identification algorithm specifically comprises the following steps:
s1, MARK point acquisition: selecting a camera and setting light source parameters, acquiring a MARK picture through the camera, adjusting a MARK contour searching algorithm until the contour of a MARK point is clearly found out, automatically adjusting preset parameters of the camera light source and adjusting contour searching visual algorithm parameters when all set parameters of the MARK contour searching algorithm are traversed and the clear contour of the MARK is not searched, and searching the contour of the MARK point again until the clear contour of the MARK point is found, at the moment, saving the current light source and camera parameters, saving a template picture of the MARK point and contour searching parameters of the MARK point;
s2, matching and identifying MARK points: a1, in actual work, firstly moving a paster head of a chip mounter to be close to a MARK point, acquiring an image through a camera, judging whether the MARK point is close to a central point, performing rough matching by adopting a template image matching algorithm when the MARK point is not close to the central point, and obtaining XY deviation and moving if the matching is successful; a2, when the MARK point is close to the central point, using the MARK point contour matching recognition algorithm to perform accurate matching to obtain XY offset, moving the head, and then collecting the image again to perform the next round of matching. Meanwhile, when one algorithm cannot be identified under certain conditions, the other algorithm is automatically switched to, and the robustness of the identification algorithm is guaranteed.
In the present invention, if the XY deviation finally converges within the tolerance range of the central point in step S2, it is determined that the MARK point has been identified, and the algorithm ends.
In the invention, the template image matching algorithm in the step S2 is a secondary matching error algorithm, the matching in the secondary matching error algorithm is carried out twice, and the first matching is rough matching. Taking interlaced array data of the template, namely quarter of the template data, carrying out interlaced array scanning matching on the searched image, namely matching in a quarter of the range of the original image, wherein the matching speed is obviously improved and the error threshold value E is greatly reduced due to the fact that the data volume is greatly reduced0
Figure BDA0002753924560000051
Wherein e0The average maximum error of each point is generally 40-50, m and n are respectively the length and width of the template, and the second matching is precise matching. At the 1 st error minimum point (i)min,jmin) In the field of (i), i.e. at a diagonal point of (i)min-1,jmin-1),(imin+1,jmin+1) to obtain the final result.
In the invention, a MARK point template picture matching algorithm and a MARK point contour matching algorithm are hierarchically fused in step S2, and dynamic matching and convergence are carried out to the MARK center point.
In the invention, the algorithm needs to be integrated in the upper computer control system software of the chip mounter for use.
In the invention, before the system is used, a MARK point setting page is opened through a software man-machine interaction interface of a chip mounter control system, and an initial light source camera parameter is adjusted, so that a button for starting customizing a corresponding function of the MARK point is clicked.
And those not described in detail in this specification are well within the skill of those in the art.
When the system is used, a MARK point setting page is opened through a man-machine interaction interface of a chip mounter control system software, an initial light source camera parameter is adjusted, a button corresponding to the MARK point is clicked to start customizing, the system can search the MARK profile, the clear MARK profile is known to stop, corresponding algorithm parameters, camera light source parameters, coordinate information and template pictures are stored, when the system is in actual production work, the head is attached to the position close to the MARK point coordinate, algorithm matching is carried out, the algorithm comprises multi-stage fusion of a template picture matching algorithm and a profile matching algorithm, and finally the position of the current MARK point is accurately located.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (6)

1. An accurate MARK point acquisition and identification algorithm based on multi-level algorithm fusion is characterized in that: the method specifically comprises the following steps:
s1, MARK point acquisition: selecting a camera and setting light source parameters, acquiring a MARK picture through the camera, adjusting a MARK contour searching algorithm until the contour of a MARK point is clearly found out, automatically adjusting preset parameters of the camera light source and adjusting contour searching visual algorithm parameters when all set parameters of the MARK contour searching algorithm are traversed and the clear contour of the MARK is not searched, and searching the contour of the MARK point again until the clear contour of the MARK point is found, at the moment, saving the current light source and camera parameters, saving a template picture of the MARK point and contour searching parameters of the MARK point;
s2, matching and identifying MARK points: a1, in actual work, firstly moving a paster head of a chip mounter to be close to a MARK point, acquiring an image through a camera, judging whether the MARK point is close to a central point, performing rough matching by adopting a template image matching algorithm when the MARK point is not close to the central point, and obtaining XY deviation and moving if the matching is successful; a2, when the MARK point is close to the central point, adopting a MARK point contour matching recognition algorithm to perform accurate matching to obtain XY offset, and acquiring the image again to perform the next round of matching after moving the head.
2. The multi-hierarchy algorithm fusion-based accurate MARK point acquisition and recognition algorithm of claim 1, wherein: if the XY deviation finally converges within the tolerance range of the central point in step S2, it is determined that the MARK point is recognized and the algorithm is ended.
3. The multi-hierarchy algorithm fusion-based accurate MARK point acquisition and recognition algorithm of claim 1, wherein: the template image matching algorithm in the step S2 is a secondary matching error algorithm, in which matching is performed twice, and the first matching is rough matching. Taking interlaced array data of the template, namely quarter of the template data, carrying out interlaced array scanning matching on the searched image, namely matching in a quarter of the range of the original image, wherein the matching speed is obviously improved and the error threshold value E is greatly reduced due to the fact that the data volume is greatly reduced0
Figure FDA0002753924550000021
Wherein e0The average maximum error of each point is generally 40-50, m and n are respectively the length and width of the template, and the second matching is precise matching. Error minimization at time 1Point (i)min,jmin) In the field of (i), i.e. at a diagonal point of (i)min-1,jmin-1),(imin+1,jmin+1) to obtain the final result.
4. The multi-hierarchy algorithm fusion-based accurate MARK point acquisition and recognition algorithm of claim 1, wherein: in the step S2, a MARK point template image matching algorithm and a MARK point contour matching algorithm are hierarchically fused, and dynamic matching and convergence are performed to the MARK center point.
5. The multi-hierarchy algorithm fusion-based accurate MARK point acquisition and recognition algorithm of claim 1, wherein: the algorithm needs to be integrated into the software of a chip mounter upper computer control system for use.
6. The multi-hierarchy algorithm fusion-based accurate MARK point acquisition and recognition algorithm of claim 1, wherein: before the system is used, a MARK point setting page is opened through a man-machine interaction interface of a chip mounter control system software, an initial light source camera parameter is adjusted, and a button for starting customizing a corresponding function of the MARK point is clicked.
CN202011195593.3A 2020-10-30 2020-10-30 Accurate MARK point acquisition and identification algorithm based on multi-level algorithm fusion Pending CN112270329A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011195593.3A CN112270329A (en) 2020-10-30 2020-10-30 Accurate MARK point acquisition and identification algorithm based on multi-level algorithm fusion

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011195593.3A CN112270329A (en) 2020-10-30 2020-10-30 Accurate MARK point acquisition and identification algorithm based on multi-level algorithm fusion

Publications (1)

Publication Number Publication Date
CN112270329A true CN112270329A (en) 2021-01-26

Family

ID=74345093

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011195593.3A Pending CN112270329A (en) 2020-10-30 2020-10-30 Accurate MARK point acquisition and identification algorithm based on multi-level algorithm fusion

Country Status (1)

Country Link
CN (1) CN112270329A (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102521560A (en) * 2011-11-14 2012-06-27 上海交通大学 Instrument pointer image identification method of high-robustness rod
CN107316315A (en) * 2017-05-04 2017-11-03 佛山市南海区广工大数控装备协同创新研究院 A kind of object recognition and detection method based on template matches
CN107545566A (en) * 2017-07-27 2018-01-05 深圳市易飞扬通信技术有限公司 Visible detection method and system
CN108243600A (en) * 2017-01-06 2018-07-03 广东华志珹智能科技有限公司 A kind of screening cover control system of chip mounting machine and method
CN109300161A (en) * 2018-10-24 2019-02-01 四川阿泰因机器人智能装备有限公司 A kind of localization method and device based on binocular vision
CN110111312A (en) * 2019-04-20 2019-08-09 东莞中科蓝海智能视觉科技有限公司 The MARK point visible detection method and device of transparent flexible glue paper
CN110838149A (en) * 2019-11-25 2020-02-25 创新奇智(广州)科技有限公司 Camera light source automatic configuration method and system
CN111311618A (en) * 2018-12-11 2020-06-19 长春工业大学 Circular arc workpiece matching and positioning method based on high-precision geometric primitive extraction
CN111474184A (en) * 2020-04-17 2020-07-31 河海大学常州校区 AOI character defect detection method and device based on industrial machine vision
CN111536872A (en) * 2020-05-12 2020-08-14 河北工业大学 Two-dimensional plane distance measuring device and method based on vision and mark point identification device

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102521560A (en) * 2011-11-14 2012-06-27 上海交通大学 Instrument pointer image identification method of high-robustness rod
CN108243600A (en) * 2017-01-06 2018-07-03 广东华志珹智能科技有限公司 A kind of screening cover control system of chip mounting machine and method
CN107316315A (en) * 2017-05-04 2017-11-03 佛山市南海区广工大数控装备协同创新研究院 A kind of object recognition and detection method based on template matches
CN107545566A (en) * 2017-07-27 2018-01-05 深圳市易飞扬通信技术有限公司 Visible detection method and system
CN109300161A (en) * 2018-10-24 2019-02-01 四川阿泰因机器人智能装备有限公司 A kind of localization method and device based on binocular vision
CN111311618A (en) * 2018-12-11 2020-06-19 长春工业大学 Circular arc workpiece matching and positioning method based on high-precision geometric primitive extraction
CN110111312A (en) * 2019-04-20 2019-08-09 东莞中科蓝海智能视觉科技有限公司 The MARK point visible detection method and device of transparent flexible glue paper
CN110838149A (en) * 2019-11-25 2020-02-25 创新奇智(广州)科技有限公司 Camera light source automatic configuration method and system
CN111474184A (en) * 2020-04-17 2020-07-31 河海大学常州校区 AOI character defect detection method and device based on industrial machine vision
CN111536872A (en) * 2020-05-12 2020-08-14 河北工业大学 Two-dimensional plane distance measuring device and method based on vision and mark point identification device

Similar Documents

Publication Publication Date Title
CN107529011B (en) A kind of motorized zoom lens control method
CN106855520B (en) Workpiece defect detection method based on machine vision
CN110449783B (en) Robot welding device and method for acquiring welding seam track
EP1388738A1 (en) Inspection data producing method and board inspection apparatus using the method
CN103500337A (en) Quadrangular side frame identifying method and device applied to intelligent wireless communication terminal
CN207223199U (en) A kind of laser marking system of online vision guide contraposition
WO2020083014A1 (en) Electromagnetic interference control method and related apparatus
CN109165422B (en) XY coordinate system-based linear endpoint coordinate labeling method and system
CN114176290A (en) Template-free full-automatic sole glue spraying method and device and storage medium
CN110187262B (en) Selection method of surface copper thickness detection test point
CN109447941B (en) Automatic registration and quality detection method in welding process of laser soldering system
CN112270329A (en) Accurate MARK point acquisition and identification algorithm based on multi-level algorithm fusion
CN114720473A (en) Defect detection method of vehicle-mounted screen backboard based on machine vision
CN113137932B (en) Portable surface clearance measuring device and measuring method
CN105787435A (en) Indication method and apparatus for iris acquisition
JP3974445B2 (en) Mark recognition method and apparatus
CN117314859A (en) Target object grabbing angle determining method
CN111596425A (en) Automatic light focusing system for collimator to focus light
CN108520533B (en) Workpiece positioning-oriented multi-dimensional feature registration method
CN107579028B (en) Method and device for determining edge of incomplete wafer and scribing device
JP2003097916A (en) Method and apparatus for recognizing substrate mark
TWI549097B (en) A method of detecting images of appearance of electronic components and computer readable media thereof
CN111162035A (en) Positioning method, positioning device, assembling method, positioning structure and chip
CN106127731B (en) Weld joint recognition method based on touch-control mark and image procossing
JP2000056210A (en) Automatic focusing device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination