JP2017150927A - Inspection device, mounted device, inspection method, and program - Google Patents

Inspection device, mounted device, inspection method, and program Download PDF

Info

Publication number
JP2017150927A
JP2017150927A JP2016033073A JP2016033073A JP2017150927A JP 2017150927 A JP2017150927 A JP 2017150927A JP 2016033073 A JP2016033073 A JP 2016033073A JP 2016033073 A JP2016033073 A JP 2016033073A JP 2017150927 A JP2017150927 A JP 2017150927A
Authority
JP
Japan
Prior art keywords
mounting
inspection
component
image
area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
JP2016033073A
Other languages
Japanese (ja)
Other versions
JP6745117B2 (en
Inventor
仁彦 横瀬
Yoshihiko Yokose
仁彦 横瀬
今野 貴史
Takashi Konno
貴史 今野
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Juki Corp
Original Assignee
Juki Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Juki Corp filed Critical Juki Corp
Priority to JP2016033073A priority Critical patent/JP6745117B2/en
Priority to CN201710102785.7A priority patent/CN107124865B/en
Publication of JP2017150927A publication Critical patent/JP2017150927A/en
Application granted granted Critical
Publication of JP6745117B2 publication Critical patent/JP6745117B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H05ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR
    • H05KPRINTED CIRCUITS; CASINGS OR CONSTRUCTIONAL DETAILS OF ELECTRIC APPARATUS; MANUFACTURE OF ASSEMBLAGES OF ELECTRICAL COMPONENTS
    • H05K13/00Apparatus or processes specially adapted for manufacturing or adjusting assemblages of electric components
    • H05K13/08Monitoring manufacture of assemblages
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20212Image combination
    • G06T2207/20224Image subtraction

Landscapes

  • Engineering & Computer Science (AREA)
  • Operations Research (AREA)
  • Manufacturing & Machinery (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Supply And Installment Of Electrical Components (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

PROBLEM TO BE SOLVED: To conduct a precise inspection with simple processing to determine whether a component is equipped to a base board.SOLUTION: An inspection device (60) is for inspection to determine whether a component is equipped to a base board from images of an equipment surface of the base board before and after a component is equipped. The inspection device includes a region setting section (63) for setting division regions formed by dividing the inspection region for the images; and a determination section (65) for determining whether a component is equipped by comparing the images before and after a component is equipped with the division regions.SELECTED DRAWING: Figure 4

Description

本発明は、基板に対する部品の搭載有無を検査する検査装置、実装装置、検査方法及びプログラムに関する。   The present invention relates to an inspection apparatus, a mounting apparatus, an inspection method, and a program for inspecting whether a component is mounted on a substrate.

実装装置として、実装ヘッドに搭載された撮像装置を用いて、基板に対する部品の搭載有無を検査する検査装置を備えたものが知られている(例えば、特許文献1参照)。特許文献1に記載の検査装置では、基板に対する搭載面を部品の搭載前後で撮像して、搭載前画像と搭載後画像との輝度差分を求めることで、基板の搭載領域に部品が搭載されているか否かが検査される。このような搭載前後の輝度差分を利用した検査では、部品以外の箇所の輝度差分が影響を与える場合がある。このため、通常は部品の周辺ノイズを除去するために撮像画像に対してノイズカットが施されている。   As a mounting apparatus, an apparatus including an inspection apparatus that inspects whether or not a component is mounted on a substrate using an imaging apparatus mounted on a mounting head is known (see, for example, Patent Document 1). In the inspection apparatus described in Patent Document 1, a part is mounted in a mounting region of a board by imaging a mounting surface on the board before and after mounting the component and obtaining a luminance difference between the image before mounting and the image after mounting. It is inspected whether it exists. In such an inspection using the luminance difference before and after mounting, the luminance difference in a portion other than the part may be affected. For this reason, normally, a noise cut is applied to the captured image in order to remove noise around the component.

特開2014−110335号公報JP, 2014-110335, A

ところで、部品のボディ部分が基板と同程度の輝度に映る場合には、部品の搭載前画像と搭載後画像の輝度差分が小さい。この場合、必要な輝度差分を出すために部品のボディに印字された文字や金属箇所の光沢等が重要になっている。しかしながら、撮像画像の全体に対して文字や光沢等の割合が小さいと、撮像画像にノイズカットが施されることで文字や光沢等による輝度差分が小さくなってしまう。このため、搭載前画像と搭載後画像の十分な輝度差分が得られず、基板の搭載面に部品が搭載されているにも関わらず、部品が搭載されていないと誤判定される可能性があった。   By the way, when the body part of a component appears to have the same luminance as the board, the luminance difference between the pre-mounting image and the post-mounting image is small. In this case, in order to obtain a necessary luminance difference, characters printed on the body of the component, gloss of the metal portion, and the like are important. However, if the ratio of characters, gloss, etc. is small with respect to the entire captured image, the luminance difference due to the characters, gloss, etc. will be reduced by applying noise cut to the captured image. For this reason, there is a possibility that a sufficient luminance difference between the pre-mounting image and the post-mounting image cannot be obtained, and it is erroneously determined that the component is not mounted even though the component is mounted on the mounting surface of the board. there were.

本発明はかかる点に鑑みてなされたものであり、簡易な処理により、基板に対する部品の搭載有無を高精度に検査することができる検査装置、実装装置、検査方法及びプログラムを提供することを目的とする。   The present invention has been made in view of the above points, and an object thereof is to provide an inspection apparatus, a mounting apparatus, an inspection method, and a program capable of inspecting whether a component is mounted on a substrate with high accuracy by simple processing. And

本発明の検査装置は、基板の搭載面に対する部品の搭載前後を撮像した前記部品の搭載前画像と搭載後画像から前記部品の搭載有無を検査する検査装置であって、前記搭載前画像と前記搭載後画像に対する検査領域を複数に分割した分割領域を設定する領域設定部と、前記分割領域に対して前記搭載前画像と前記搭載後画像を比較することで前記部品の搭載有無を判定する判定部とを備えたことを特徴とする。   The inspection apparatus of the present invention is an inspection apparatus that inspects whether or not the component is mounted from the pre-mounting image and post-mounting image of the component obtained by imaging before and after mounting of the component on the mounting surface of the substrate, An area setting unit that sets a divided area obtained by dividing the inspection area for the post-mounting image into a plurality of areas, and a determination for determining whether the component is mounted by comparing the pre-mounting image and the post-mounting image with respect to the divided area And a section.

本発明の検査方法は、基板の搭載面に対する部品の搭載前後を撮像した前記部品の搭載前画像と搭載後画像から前記部品の搭載有無を検査する検査方法であって、前記搭載前画像と前記搭載後画像に対する検査領域を複数に分割した分割領域を設定するステップと、前記分割領域に対して前記搭載前画像と前記搭載後画像を比較することで前記部品の搭載有無を判定するステップとを有することを特徴とする。   The inspection method of the present invention is an inspection method for inspecting the presence / absence of mounting of the component from the pre-mounting image and post-mounting image of the component obtained by imaging before and after mounting of the component on the mounting surface of the substrate. Setting a divided area obtained by dividing an inspection area for a post-mounting image into a plurality of areas, and determining whether the component is mounted by comparing the pre-mounting image and the post-mounting image with respect to the divided area. It is characterized by having.

これらの構成によれば、検査領域を複数に分割した各分割領域で搭載前画像と搭載後画像が比較されるため、各分割領域における搭載前画像と搭載後画像の差分箇所が強調される。よって、検査領域全体では搭載前画像と搭載後画像の差分箇所の違いが曖昧であっても、分割領域毎に搭載前画像と搭載後画像を比較することで、搭載前画像と搭載後画像の差分箇所が強調されて部品の搭載有無を高精度に検査することができる。   According to these configurations, the pre-mounting image and the post-mounting image are compared in each divided region obtained by dividing the inspection region into a plurality of regions, and thus the difference between the pre-mounting image and the post-mounting image in each divided region is emphasized. Therefore, even if the difference in the difference between the pre-mounting image and the post-mounting image is ambiguous in the entire inspection area, comparing the pre-mounting image and the post-mounting image for each divided area, The difference location is emphasized, and the presence / absence of component mounting can be inspected with high accuracy.

上記の検査装置において、前記搭載前画像と前記搭載後画像から差分画像を生成する差分画像生成部を備え、前記領域設定部が、前記搭載前画像と前記搭載後画像の差分画像に対して前記複数の分割領域を設定し、前記判定部が、前記差分画像の前記複数の分割領域のいずれかで輝度が分割領域用の判定閾値以上か否かに基づいて前記部品の有無を判定する。この構成によれば、搭載前画像と搭載後画像の差分画像における各分割領域の輝度から部品の搭載有無を高精度に検査することができる。   The inspection apparatus includes a difference image generation unit that generates a difference image from the pre-mounting image and the post-mounting image, and the region setting unit performs the difference image between the pre-mounting image and the post-mounting image. A plurality of divided areas are set, and the determination unit determines the presence / absence of the component based on whether the luminance is equal to or higher than a determination threshold for the divided areas in any of the plurality of divided areas of the difference image. According to this configuration, it is possible to inspect whether or not a component is mounted with high accuracy from the luminance of each divided region in the difference image between the pre-mounting image and the post-mounting image.

上記の検査装置において、前記部品の搭載有無の検査は、前記差分画像の前記検査領域の輝度が前記検査領域用の判定閾値より小さい場合の再検査である。この構成によれば、検査領域に対する初期検査で基板の搭載面に部品が搭載されていないと判定された場合に、分割領域毎に部品の搭載有無を再検査することができる。   In the inspection apparatus, the inspection of whether or not the component is mounted is a re-inspection when the luminance of the inspection area of the difference image is smaller than a determination threshold for the inspection area. According to this configuration, when it is determined that the component is not mounted on the mounting surface of the substrate in the initial inspection for the inspection region, it is possible to re-inspect whether the component is mounted for each divided region.

上記の検査装置において、前記部品の搭載有無の検査は、前記検査領域用の判定閾値を上限値とした所定範囲に、前記差分画像の前記検査領域の輝度が含まれる場合の再検査である。この構成によれば、検査領域に対する初期検査で、輝度が判定閾値付近であるにも関わらず基板の搭載面に部品が搭載されていないと判定された場合にのみ、分割領域毎に部品の搭載有無を再検査することができる。よって、基板の載置面に明らかに部品が搭載されていない場合には再検査されず、基板の載置面に部品が搭載されている可能性がある場合にのみ再検査させることができる。   In the inspection apparatus, the inspection of whether or not the component is mounted is a re-inspection when the luminance of the inspection area of the difference image is included in a predetermined range with the determination threshold for the inspection area as an upper limit value. According to this configuration, the component is mounted for each divided region only when it is determined that the component is not mounted on the mounting surface of the board in the initial inspection for the inspection region even though the luminance is near the determination threshold value. Existence can be rechecked. Therefore, when a component is clearly not mounted on the substrate mounting surface, re-inspection is not performed, and only when there is a possibility that a component is mounted on the substrate mounting surface, re-inspection can be performed.

上記の検査装置において、前記分割領域用の判定閾値が、前記検査領域用の判定閾値を前記分割領域の分割数で除算した除算結果にマージンを設けた値である。この構成によれば、分割領域の大きさに適した判定閾値によって部品の搭載有無を検査することができる。また、分割領域毎に差分箇所が強調されることで輝度が増加傾向にあるため、マージンによって判定閾値を高くしてバランスを調整することができる。   In the inspection apparatus, the determination threshold value for the divided area is a value obtained by providing a margin to a division result obtained by dividing the determination threshold value for the inspection area by the number of divisions of the divided area. According to this configuration, it is possible to inspect whether or not a component is mounted based on a determination threshold suitable for the size of the divided area. Further, since the luminance tends to increase by enhancing the difference portion for each divided region, the balance can be adjusted by increasing the determination threshold value by the margin.

本発明の実装装置は、上記の検査装置と、前記搭載面に対して前記部品を搭載する実装ヘッドと、前記実装ヘッドによる部品の搭載前後を撮像する撮像装置とを備えたことを特徴とする。この構成によれば、撮像装置によって実装ヘッドによる部品の搭載前後の実装面が撮像され、搭載前画像と搭載後画像から実装ヘッドによる部品の搭載ミスを即座に検出することができる。   A mounting apparatus of the present invention includes the above-described inspection apparatus, a mounting head for mounting the component on the mounting surface, and an imaging device for imaging before and after mounting of the component by the mounting head. . According to this configuration, the mounting surface before and after the mounting of the component by the mounting head is imaged by the imaging device, and a mounting error of the component by the mounting head can be immediately detected from the pre-mounting image and the post-mounting image.

本発明のプログラムは、上記の検査方法を検査装置に実行させることを特徴とする。この構成によれば、検査装置にプログラムをインストールすることで、部品の搭載有無の高精度な検査機能を検査装置に追加することができる。   A program according to the present invention causes an inspection apparatus to execute the above-described inspection method. According to this configuration, by installing a program in the inspection apparatus, it is possible to add a high-precision inspection function for whether or not a component is mounted to the inspection apparatus.

本発明によれば、検査領域を複数の分割領域に分割することで、各分割領域における搭載前画像と搭載後画像の差分箇所を強調した状態で比較して部品の搭載有無を高精度に検査することができる。   According to the present invention, by dividing the inspection area into a plurality of divided areas, the difference between the pre-mounting image and the post-mounting image in each divided area is compared in a highlighted state, and the presence / absence of mounting of the component is inspected with high accuracy. can do.

本実施の形態の実装装置全体を示す模式図である。It is a schematic diagram which shows the whole mounting apparatus of this Embodiment. 本実施の形態の実装ヘッド周辺を示す模式図である。It is a schematic diagram which shows the mounting head periphery of this Embodiment. 比較例の部品の差分画像の一例を示す図である。It is a figure which shows an example of the difference image of the components of a comparative example. 本実施の形態の検査装置のブロック図である。It is a block diagram of the inspection apparatus of this Embodiment. 本実施の形態の検査処理の一例を示すフローチャートである。It is a flowchart which shows an example of the test | inspection process of this Embodiment. 本実施の形態の部品の搭載前画像及び搭載後画像の一例を示す図である。It is a figure which shows an example of the image before mounting of the component of this Embodiment, and the image after mounting. 本実施の形態の差分画像の一例を示す図である。It is a figure which shows an example of the difference image of this Embodiment. 本実施の形態の領域設定処理の一例を示す図である。It is a figure which shows an example of the area | region setting process of this Embodiment. 本実施の形態の判定処理の一例を示す図である。It is a figure which shows an example of the determination process of this Embodiment.

以下、添付図面を参照して本実施の形態の実装装置について説明する。図1は、本実施の形態の実装装置全体を示す模式図である。図2は、本実施の形態の実装ヘッド周辺を示す模式図である。図3は、比較例の部品の差分画像の一例を示す図である。なお、本実施の形態の実装装置は一例に過ぎず、適宜変更が可能である。   Hereinafter, the mounting apparatus of the present embodiment will be described with reference to the accompanying drawings. FIG. 1 is a schematic diagram showing the entire mounting apparatus of the present embodiment. FIG. 2 is a schematic diagram showing the periphery of the mounting head of the present embodiment. FIG. 3 is a diagram illustrating an example of a difference image of parts in a comparative example. In addition, the mounting apparatus of this Embodiment is only an example, and can be changed suitably.

図1に示すように、実装装置1は、テープフィーダ等の部品供給ユニット20から供給された部品P(図2参照)を、一対の実装ヘッド40によって基板Wの載置面に搭載するように構成されている。基板Wの表面には配線パターンや電極パッド等が設けられており、配線パターンや電極パッド等には後段のリフロー工程で部品Pの端子に接合するための半田ペーストが付着されている。なお、基板Wは部品Pが搭載可能なものであればよく、形状や種類は特に限定されない。また、部品PとしてICチップを例示して説明するが、基板Wに搭載される部品であれば、特に電子部品に限られない。   As shown in FIG. 1, the mounting apparatus 1 mounts a component P (see FIG. 2) supplied from a component supply unit 20 such as a tape feeder on the mounting surface of the substrate W by a pair of mounting heads 40. It is configured. A wiring pattern, an electrode pad, and the like are provided on the surface of the substrate W, and a solder paste for bonding to the terminal of the component P in a subsequent reflow process is attached to the wiring pattern, the electrode pad, and the like. The substrate W only needs to be capable of mounting the component P, and the shape and type are not particularly limited. Further, although an IC chip is exemplified as the component P, the component P is not limited to an electronic component as long as it is a component mounted on the substrate W.

実装装置1には、X軸方向に基板Wを搬送する基板搬送ユニット10が配設されている。基板搬送ユニット10は、基板Wを搬送する一対のコンベアベルト11と各コンベアベルト11に沿って基板Wの搬送をガイドする一対のガイドレール12とによって搬送路を形成している。コンベアベルト11は、X軸方向の一端側から部品搭載前の基板Wを実装ヘッド40の下方に搬入して位置決めし、部品搭載後の基板WをX軸方向の他端側に搬出している。一対のガイドレール12の上部は内向きに屈曲している(図2参照)。昇降機構(不図示)により、この屈曲部分に向けて、基板Wと一対のコンベアベルト11が上昇して、基板Wが位置決めされる。   The mounting apparatus 1 is provided with a substrate transport unit 10 that transports the substrate W in the X-axis direction. The substrate transport unit 10 forms a transport path by a pair of conveyor belts 11 that transport the substrate W and a pair of guide rails 12 that guide the transport of the substrate W along each conveyor belt 11. The conveyor belt 11 carries the substrate W before component mounting from one end side in the X axis direction under the mounting head 40 and positions it, and carries the substrate W after component mounting to the other end side in the X axis direction. . The upper parts of the pair of guide rails 12 are bent inward (see FIG. 2). A lift mechanism (not shown) raises the substrate W and the pair of conveyor belts 11 toward the bent portion, and positions the substrate W.

部品供給ユニット20にはテープリール21が着脱自在に装着され、テープリール21には多数の部品Pをパッケージングしたキャリアテープが巻回されている。各部品供給ユニット20は、テープリール21の回転によって実装ヘッド40にピックアップされる受け渡し位置に向けて、順番に部品Pを繰り出している。実装ヘッド40の受け渡し位置では、キャリアテープから表面のカバーテープが剥離され、キャリアテープのポケット内の部品Pが外部に露出される。なお、本実施の形態では、部品供給ユニット20としてテープフィーダを例示したが、ボールフィーダ等の他の部品供給ユニット20で構成されていてもよい。   A tape reel 21 is detachably mounted on the component supply unit 20, and a carrier tape that packages a large number of components P is wound around the tape reel 21. Each component supply unit 20 feeds the components P in order toward the delivery position picked up by the mounting head 40 by the rotation of the tape reel 21. At the delivery position of the mounting head 40, the cover tape on the surface is peeled from the carrier tape, and the component P in the pocket of the carrier tape is exposed to the outside. In the present embodiment, the tape feeder is exemplified as the component supply unit 20, but the component supply unit 20 may be composed of another component supply unit 20 such as a ball feeder.

一対の実装ヘッド40は、移動機構30によって実装ヘッド40をX軸方向、Y軸方向に移動させて、部品供給ユニット20からピックアップした部品Pを基板W上の搭載面に実装している。移動機構30は、Y軸方向に延びる一対のY軸駆動部31上に、X軸方向に延びる一対のX軸駆動部32が両持ちで支持されている。一対のY軸駆動部31には一対のX軸駆動部32がY軸方向に移動可能に設置されており、各X軸駆動部32には実装ヘッド40がX軸方向に移動可能に設置されている。一対の実装ヘッド40は、X軸駆動部32とY軸駆動部31とによって、部品供給ユニット20と基板Wとの間を往復移動される。   The pair of mounting heads 40 mount the component P picked up from the component supply unit 20 on the mounting surface by moving the mounting head 40 in the X-axis direction and the Y-axis direction by the moving mechanism 30. In the moving mechanism 30, a pair of X-axis drive units 32 extending in the X-axis direction are supported by both ends on a pair of Y-axis drive units 31 extending in the Y-axis direction. A pair of X-axis drive units 32 is installed in the pair of Y-axis drive units 31 so as to be movable in the Y-axis direction, and a mounting head 40 is installed in each X-axis drive unit 32 so as to be movable in the X-axis direction. ing. The pair of mounting heads 40 are reciprocated between the component supply unit 20 and the substrate W by the X-axis drive unit 32 and the Y-axis drive unit 31.

図2に示すように、実装ヘッド40は、X軸駆動部32に支持されたヘッド本体41に複数のノズル42(本実施の形態では1つのみ図示)を設けて構成されている。各ノズル42は、ノズル駆動部43を介してヘッド本体41に支持されており、ノズル駆動部43によってZ軸方向に上下動すると共にノズル42をZ軸回りに回転する。各ノズル42は吸引源(不図示)に接続されており、吸引源からの吸引力によって部品Pを吸着保持する。ノズル42にはコイルバネが設けられており、コイルバネを収縮させながらノズル42に吸着された部品Pを基板Wに搭載している。   As shown in FIG. 2, the mounting head 40 is configured by providing a plurality of nozzles 42 (only one is shown in the present embodiment) in a head main body 41 supported by the X-axis drive unit 32. Each nozzle 42 is supported by the head main body 41 via a nozzle drive unit 43, and moves up and down in the Z-axis direction by the nozzle drive unit 43 and rotates the nozzle 42 about the Z-axis. Each nozzle 42 is connected to a suction source (not shown) and sucks and holds the component P by a suction force from the suction source. The nozzle 42 is provided with a coil spring, and the component P adsorbed by the nozzle 42 is mounted on the substrate W while the coil spring is contracted.

ヘッド本体41には、基板Wからの高さを検出する高さセンサ(不図示)や、ノズル42で吸着した部品Pの姿勢を検出する姿勢認識部45が設けられている。高さセンサでは、基板Wからノズル42までの距離が検出され、検出結果に基づいてノズル42の上下方向の移動量が制御される。姿勢認識部45では、横一列に並んだ発光素子46と受光素子47とを水平方向で対向させ、各発光素子46からのレーザー光が部品Pに遮られて各受光素子47の受光状態が変わることで部品Pの吸着姿勢が認識される。この姿勢認識部45の認識結果によってノズル42の吸着位置や吸着向きが補正される。   The head body 41 is provided with a height sensor (not shown) for detecting the height from the substrate W and a posture recognition unit 45 for detecting the posture of the component P sucked by the nozzle 42. In the height sensor, the distance from the substrate W to the nozzle 42 is detected, and the amount of movement of the nozzle 42 in the vertical direction is controlled based on the detection result. In the posture recognition unit 45, the light emitting elements 46 and the light receiving elements 47 arranged in a horizontal row are opposed to each other in the horizontal direction, and the laser light from each light emitting element 46 is blocked by the component P and the light receiving state of each light receiving element 47 changes. Thus, the suction posture of the component P is recognized. The suction position and suction direction of the nozzle 42 are corrected based on the recognition result of the posture recognition unit 45.

ヘッド本体41には、基板W上のBOCマークを真上から撮像する基板撮像部(不図示)と、ノズル42による部品Pの搭載動作を斜め上方から撮像する部品撮像部48(撮像装置)とが設けられている。基板撮像部では、BOCマークの撮像画像に基づいて基板Wの位置、反り等が認識され、これらの認識結果に基づいて基板Wに対する部品Pの搭載位置が補正される。部品撮像部48では、部品供給ユニット20に対する部品Pの吸着前後が撮像される他、基板Wの載置面に対する部品Pの搭載前後が撮像される。これら撮像画像によって、ノズル42による部品Pの吸着有無、基板Wにおける部品Pの搭載有無が検査される。   The head main body 41 includes a substrate imaging unit (not shown) that images the BOC mark on the substrate W from directly above, and a component imaging unit 48 (imaging device) that images the mounting operation of the component P by the nozzle 42 from obliquely above. Is provided. The board imaging unit recognizes the position, warpage, and the like of the substrate W based on the captured image of the BOC mark, and corrects the mounting position of the component P on the board W based on the recognition result. The component imaging unit 48 images before and after the component P is attracted to the component supply unit 20 and also images before and after the component P is mounted on the mounting surface of the substrate W. With these captured images, whether or not the component P is attracted by the nozzle 42 and whether or not the component P is mounted on the substrate W are inspected.

また、実装装置1には、装置各部を統括制御する制御装置50と、ノズル42による部品Pの吸着有無や基板Wに対する部品Pの搭載有無を検査する検査装置60とが設けられている。これらの装置は、各種処理を実行するプロセッサやメモリ等により構成されている。メモリは、用途に応じてROM(Read Only Memory)、RAM(Random Access Memory)等の一つ又は複数の記憶媒体で構成されており、実装装置1の制御プログラムや、検査装置60に検査方法を実行させるためのプログラム等、部品Pの搭載有無の判定閾値等の各種パラメータが記憶されている。   In addition, the mounting apparatus 1 is provided with a control device 50 that performs overall control of each part of the device, and an inspection device 60 that inspects whether or not the component P is adsorbed by the nozzle 42 and whether or not the component P is mounted on the substrate W. These apparatuses are configured by a processor, a memory, and the like that execute various processes. The memory is composed of one or a plurality of storage media such as a ROM (Read Only Memory) and a RAM (Random Access Memory) depending on the application, and the control program of the mounting apparatus 1 and the inspection method are applied to the inspection apparatus 60. Various parameters such as a determination threshold for determining whether or not the component P is mounted, such as a program to be executed, are stored.

このように構成された実装装置1では、実装ヘッド40を部品供給ユニット20(図1参照)まで移動させて、部品供給ユニット20から供給された部品Pをノズル42でピックアップして、基板Wの所望の搭載面に部品Pを搭載している。このノズル42による部品Pの搭載動作では、基板Wに対する部品Pの搭載が失敗して、基板W上に部品Pが搭載されていない場合がある。このため、上記したように、基板Wの搭載面に対する部品Pの搭載前後がモノクロ画像で撮像され、検査装置60によって部品Pの搭載前画像と搭載後画像から基板W上の部品Pの搭載有無が検査されている。   In the mounting apparatus 1 configured as described above, the mounting head 40 is moved to the component supply unit 20 (see FIG. 1), the component P supplied from the component supply unit 20 is picked up by the nozzle 42, and the substrate W The component P is mounted on a desired mounting surface. In the mounting operation of the component P by the nozzle 42, the mounting of the component P on the substrate W may fail and the component P may not be mounted on the substrate W. Therefore, as described above, before and after the mounting of the component P on the mounting surface of the substrate W is captured as a monochrome image, and whether or not the component P is mounted on the substrate W from the pre-mounting image and the post-mounting image of the component P by the inspection device 60. Has been inspected.

部品Pの搭載有無を検査する際には、搭載前画像と搭載後画像から差分画像が生成され、差分画像に対して設定されたウインドウ内の輝度(二乗和)に基づいて部品Pの搭載有無が検査される。しかしながら、実装装置1の振動等によって搭載前画像と搭載後画像にズレが生じる場合がある。この場合、基板W上のシルク等のように輝度のコントラストが明確な箇所にズレが生じていると、差分画像のウインドウの中で部品P以外の箇所の影響が大きくなる。このため、通常はウインドウ全体にノイズカットが施されて搭載前画像と搭載後画像のズレの影響が抑えられている。   When inspecting whether or not the component P is mounted, a difference image is generated from the pre-mounting image and the post-mounting image, and whether or not the component P is mounted based on the luminance (sum of squares) in the window set for the difference image. Is inspected. However, there may be a difference between the pre-mounting image and the post-mounting image due to vibration of the mounting apparatus 1 or the like. In this case, if there is a deviation in a portion where the brightness contrast is clear, such as silk on the substrate W, the influence of the portion other than the component P in the window of the difference image becomes large. For this reason, noise cut is usually applied to the entire window to suppress the influence of the deviation between the pre-mounting image and the post-mounting image.

また、比較的大きな部品Pを基板Wに搭載する場合にはウインドウが広く設定された分だけシルクのズレ等のようにコントラストが明確なノイズが入り込み易い。このため、上記のウインドウ内に対するノイズカットに加えて、差分画像に対する輝度判定の際の判定閾値が高く設定されている。ノイズカットによって搭載前画像と搭載後画像の差分画像の輝度が小さくなると共に高い判定閾値によって基板Wに対する部品Pの搭載有無が判定されるため、部品Pが搭載されているにも関わらず、部品Pが搭載されていないと誤判定されるおそれがある。   Further, when a relatively large component P is mounted on the substrate W, noise with a clear contrast is likely to enter, such as a silk shift, by the amount of a wide window. For this reason, in addition to the above-described noise cut in the window, the determination threshold for the luminance determination for the difference image is set high. Since the luminance of the difference image between the pre-mounting image and the post-mounting image is reduced by the noise cut, and whether or not the component P is mounted on the substrate W is determined by the high determination threshold, the component P is mounted even though the component P is mounted. If P is not mounted, there is a risk of erroneous determination.

例えば、図3に示すように、モノクロ画像では部品Pのボディ部分が基板Wと同程度の輝度に映る場合があり、このような場合には、金属製の取付穴の光沢78、リードの光沢79等の差分箇所の輝度が重要になる。しかしながら、差分画像に対してノイズカットが施されているため、取付穴の光沢78やリードの光沢79による輝度が小さくなる。このように、目視では明らかに搭載前画像と搭載後画像に差分があるものであっても、ノイズカットや判定閾値によっては搭載前画像と搭載後画像に差分がないと判定されてしまう。   For example, as shown in FIG. 3, in the monochrome image, the body portion of the component P may appear as bright as the substrate W. In such a case, the gloss 78 of the metal mounting hole, the gloss of the lead The luminance of the difference portion such as 79 becomes important. However, since the noise reduction is applied to the difference image, the luminance due to the gloss 78 of the mounting hole and the gloss 79 of the lead is reduced. Thus, even if there is a clear difference between the pre-mounting image and the post-mounting image visually, it is determined that there is no difference between the pre-mounting image and the post-mounting image depending on the noise cut or the determination threshold.

そこで、本実施の形態の検査装置60では、ウインドウ内の検査領域を複数の分割領域に分割することで、各分割領域における搭載前画像と搭載後画像の差分箇所を強調した状態で比較して、部品Pの搭載有無を高精度に検査するようにしている。これにより、検査領域全体に対する検査では搭載前画像と搭載後画像に差分がないと判定される場合であっても、分割領域のいずれかでは搭載前画像と搭載後画像の差分箇所が強調されて部品の搭載有無を高精度に検査することが可能になっている。   Therefore, in the inspection apparatus 60 according to the present embodiment, the inspection area in the window is divided into a plurality of divided areas, so that the difference between the pre-mounting image and the post-mounting image in each divided area is compared and emphasized. The presence / absence of the component P is inspected with high accuracy. As a result, even if it is determined that there is no difference between the pre-mounting image and the post-mounting image in the inspection for the entire inspection region, the difference between the pre-mounting image and the post-mounting image is emphasized in any of the divided regions. It is possible to inspect the presence / absence of components with high accuracy.

以下、部品の搭載有無の検査処理について説明する。図4は、本実施の形態の検査装置のブロック図である。図5は、本実施の形態の検査処理の一例を示すフローチャートである。図6は、本実施の形態の部品の搭載前画像及び搭載後画像の一例を示す図である。図7は、本実施の形態の差分画像の一例を示す図である。図8は、本実施の形態の領域設定処理の一例を示す図である。図9は、本実施の形態の判定処理の一例を示す図である。   Hereinafter, the inspection process for the presence / absence of component mounting will be described. FIG. 4 is a block diagram of the inspection apparatus of the present embodiment. FIG. 5 is a flowchart illustrating an example of the inspection process according to the present embodiment. FIG. 6 is a diagram illustrating an example of a pre-mounting image and a post-mounting image of the component according to the present embodiment. FIG. 7 is a diagram illustrating an example of a difference image according to the present embodiment. FIG. 8 is a diagram illustrating an example of area setting processing according to the present embodiment. FIG. 9 is a diagram illustrating an example of the determination process according to the present embodiment.

先ず、図4を参照して、検査装置60の制御構成について簡単に説明する。検査装置60は、部品撮像部48に接続されており、部品撮像部48から基板Wの載置面に対する部品Pの搭載前画像及び搭載後画像を取得して部品Pの搭載有無を検査している。この場合、先ずウインドウ全体を検査領域にした初期検査が実施され、初期検査で部品Pの搭載ミスと判定された場合に、検査領域を複数に分割した各分割領域で再検査が実施される。検査装置60は、マッチング部61、差分画像生成部62、領域設定部63、算出部64、判定部65を備えている。   First, a control configuration of the inspection apparatus 60 will be briefly described with reference to FIG. The inspection device 60 is connected to the component imaging unit 48, acquires images before and after mounting the component P on the mounting surface of the substrate W from the component imaging unit 48, and inspects whether or not the component P is mounted. Yes. In this case, first, an initial inspection is performed in which the entire window is set as an inspection region. When it is determined in the initial inspection that the component P is mounted incorrectly, re-inspection is performed in each divided region obtained by dividing the inspection region into a plurality of regions. The inspection device 60 includes a matching unit 61, a difference image generation unit 62, a region setting unit 63, a calculation unit 64, and a determination unit 65.

マッチング部61では、部品撮像部48から取得した搭載前画像と搭載後画像を位置合わせするマッチング処理が実施される。差分画像生成部62では、搭載前画像と搭載後画像から差分画像の生成処理が実施される。領域設定部63では、初期検査時に差分画像に対してウインドウが設定される他、再検査時に差分画像に対してウインドウ内の検査領域を複数に分割した複数の分割領域が設定される。算出部64では、初期検査時に検査領域で各画素の輝度の二乗和が算出される他、再検査時に各分割領域で各画素の輝度の二乗和が算出される。   In the matching unit 61, matching processing for aligning the pre-mounting image and the post-mounting image acquired from the component imaging unit 48 is performed. In the difference image generation unit 62, a difference image generation process is performed from the pre-mounting image and the post-mounting image. In the area setting unit 63, a window is set for the difference image at the time of the initial inspection, and a plurality of divided areas are set by dividing the inspection area in the window into the difference image for the re-inspection. The calculation unit 64 calculates the sum of squares of the brightness of each pixel in the inspection area at the time of the initial inspection, and calculates the sum of squares of the brightness of each pixel in each divided area at the time of the re-examination.

判定部65では、初期検査時にウインドウ内の検査領域における各画素の輝度の二乗和とウインドウの大きさに対応した判定閾値とが比較されて、検査領域の二乗和が検査領域用の判定閾値以上か否かで部品Pの搭載有無の判定処理が実施される。また、判定部65では、再検査時に各分割領域における各画素の二乗和と分割領域の大きさに対応した判定閾値とが比較されて、複数の分割領域のいずれかの二乗和が分割領域用の判定閾値以上か否かで部品Pの搭載有無の判定処理が実施される。なお、図4のブロック図には、検査装置60が簡略化して記載されているが、検査装置60が通常備える構成については備えているものとする。   In the determination unit 65, the square sum of the luminance of each pixel in the inspection region in the window is compared with the determination threshold corresponding to the size of the window at the time of the initial inspection, and the square sum of the inspection region is equal to or greater than the determination threshold for the inspection region. Whether or not the component P is mounted is determined depending on whether or not the component P is mounted. Further, the determination unit 65 compares the square sum of each pixel in each divided region with the determination threshold corresponding to the size of the divided region at the time of re-examination, and the square sum of any of the plurality of divided regions is used for the divided region. Whether or not the component P is mounted is determined depending on whether or not it is equal to or greater than the determination threshold. In the block diagram of FIG. 4, the inspection apparatus 60 is described in a simplified manner, but the configuration that the inspection apparatus 60 normally includes is assumed to be included.

続いて、図5から図9を参照して、検査装置の検査処理の流れについて説明する。なお、図5のフローチャートの説明では、図4の各ブロックに付された符号を適宜使用して説明する。図5に示すように、検査装置60には、部品撮像部48から搭載前画像及び搭載後画像が入力される(ステップS01)。図6Aに示すように、搭載前画像は、部品Pの搭載前に部品撮像部48で斜め上方から基板Wを撮像したモノクロ画像である。搭載前画像には、基板Wに印刷されたシルク71や、シルク71の内側の載置面72が映されている。載置面72には、半田73が載った電極や金属製の取付穴74が設けられている。   Next, the flow of the inspection process of the inspection apparatus will be described with reference to FIGS. In the description of the flowchart in FIG. 5, the description will be made by appropriately using the reference numerals attached to the respective blocks in FIG. 4. As shown in FIG. 5, the pre-mounting image and the post-mounting image are input to the inspection apparatus 60 from the component imaging unit 48 (step S01). As shown in FIG. 6A, the pre-mounting image is a monochrome image obtained by picking up an image of the substrate W from diagonally above with the component image pickup unit 48 before the component P is mounted. In the pre-mounting image, the silk 71 printed on the substrate W and the mounting surface 72 inside the silk 71 are shown. The mounting surface 72 is provided with an electrode on which the solder 73 is mounted and a metal mounting hole 74.

図6Bに示すように、搭載後画像は、部品Pの搭載後に部品撮像部48で斜め上方から基板Wを撮像したモノクロ画像である。搭載後画像には、基板Wの載置面72(図6A参照)上に搭載された部品Pが映されている。部品Pのボディ部分75には文字77が付され、部品Pのリード76が半田73上に設置されている。なお、モノクロ画像とは、無彩色又は有彩色の単色の濃淡で表された画像であり、グレースケールを含むものである。次に、マッチング部61にてマッチング処理が実施される(ステップS02)。マッチング処理では、部品Pの搭載時に生じる基板Wの反りや撓みを考慮して、搭載前画像と搭載後画像が位置合わせされる。   As shown in FIG. 6B, the post-mounting image is a monochrome image obtained by picking up the substrate W from obliquely above with the component imaging unit 48 after the component P is mounted. In the post-mounting image, the component P mounted on the mounting surface 72 (see FIG. 6A) of the substrate W is shown. The body part 75 of the part P is marked with a letter 77, and the lead 76 of the part P is placed on the solder 73. Note that a monochrome image is an image represented by a single shade of achromatic or chromatic color, and includes a gray scale. Next, matching processing is performed in the matching unit 61 (step S02). In the matching process, the pre-mounting image and the post-mounting image are aligned in consideration of the warpage and bending of the substrate W that occurs when the component P is mounted.

次に、差分画像生成部62にて差分画像の生成処理が実施される(ステップS03)。図7に示すように、差分画像の生成処理ではマッチング後の搭載前画像と搭載後画像の輝度差分の絶対値を取って差分画像が生成される。差分画像には、シルク71の一部、金属製の取付穴74の光沢78、リード76の光沢79等が搭載前画像と搭載後画像の特徴的な差分箇所として現れている。しかしながら、差分画像にはノイズカットが施されているため、ボディ部分の文字77(図6B参照)が除去されると共に、これら差分箇所の輝度が小さくなっている。   Next, the difference image generation unit 62 performs a difference image generation process (step S03). As shown in FIG. 7, in the difference image generation process, the difference image is generated by taking the absolute value of the luminance difference between the pre-mounting image after the matching and the post-mounting image. In the difference image, a part of the silk 71, the gloss 78 of the metal mounting hole 74, the gloss 79 of the lead 76, and the like appear as characteristic differences between the pre-mounting image and the post-mounting image. However, since the noise cut is applied to the difference image, the body portion character 77 (see FIG. 6B) is removed, and the luminance of these difference portions is reduced.

次に、領域設定部63にて差分画像に対する検査領域81の設定処理が実施される(ステップS04)。図8Aに示すように、検査領域81の設定処理では、部品Pの大きさに応じたウインドウによって差分画像に検査領域81が設定される。次に、算出部64にて、差分画像に設定された検査領域81に対して算出処理が実施される(ステップS05)。検査領域81に対する算出処理では、算出部64によって検査領域81の各画素の輝度の二乗の合計値が初期検査用の算出結果V1として算出される。このように、本実施の形態では差分画像の検査領域81の輝度として、検査領域81の輝度を強調した値を用いている。   Next, the setting process of the inspection area 81 for the difference image is performed by the area setting unit 63 (step S04). As shown in FIG. 8A, in the setting process of the inspection area 81, the inspection area 81 is set in the difference image by a window corresponding to the size of the component P. Next, the calculation unit 64 performs a calculation process on the inspection area 81 set in the difference image (step S05). In the calculation process for the inspection area 81, the calculation unit 64 calculates the total square value of the luminance of each pixel in the inspection area 81 as the calculation result V1 for the initial inspection. As described above, in this embodiment, a value in which the luminance of the inspection area 81 is emphasized is used as the luminance of the inspection area 81 of the difference image.

次に、領域設定部63にて差分画像に対する分割領域82の設定処理が実施される(ステップS06)。図8Bに示すように、分割領域82の設定処理では、ウインドウ内の検査領域81を4分割した分割領域82が差分画像に設定される。次に、算出部64にて、差分画像に設定された分割領域82毎に算出処理が実施される(ステップS07)。各分割領域82に対する算出処理では、算出部64によって分割領域82の各画素の輝度の二乗の合計値が再検査用の算出結果V2として算出される。このように、本実施の形態では差分画像の各分割領域82の輝度として、各分割領域82の輝度を強調した値を用いている。   Next, the area setting unit 63 performs a setting process of the divided area 82 for the difference image (step S06). As shown in FIG. 8B, in the setting process of the divided area 82, the divided area 82 obtained by dividing the inspection area 81 in the window into four is set as the difference image. Next, calculation processing is performed for each divided region 82 set in the difference image by the calculation unit 64 (step S07). In the calculation process for each divided region 82, the calculation unit 64 calculates the total square value of the luminance of each pixel in the divided region 82 as the calculation result V2 for reexamination. As described above, in the present embodiment, a value in which the luminance of each divided region 82 is emphasized is used as the luminance of each divided region 82 of the difference image.

なお、分割領域82は、ウインドウ内の検査領域81を4分割した領域に限られず、適宜変更することが可能である。ただし、検査領域81の分割数を増やして分割領域82を狭くすることで、部品Pの搭載前後の差分箇所が強調されるが、分割数を増やし過ぎると逆に差分箇所が曖昧になる。例えば、図8Cに示すように、検査領域81を4分割した分割領域A−Dでは輝度の偏りが大きくなって差分箇所が強調されるが、検査領域81を9分割した分割領域E−Mでは輝度の偏りが小さくなって差分箇所が殆ど強調されない。このため、分割領域82の輝度の偏りが一定以上になるように検査領域81の分割数が決定されている。   The divided area 82 is not limited to the area obtained by dividing the inspection area 81 in the window into four, and can be changed as appropriate. However, by increasing the number of divisions of the inspection area 81 and narrowing the division area 82, the difference points before and after the mounting of the component P are emphasized. However, if the number of divisions is increased too much, the difference points become vague. For example, as shown in FIG. 8C, in the divided areas A to D in which the inspection area 81 is divided into four, the luminance deviation becomes large and the difference portion is emphasized. In the divided areas E to M in which the inspection area 81 is divided into nine, The difference in luminance is reduced and the difference portion is hardly emphasized. For this reason, the number of divisions of the inspection area 81 is determined so that the luminance deviation of the division area 82 becomes a certain level or more.

次に、初期検査として判定部65にてウインドウ内の検査領域81で部品Pの搭載有無を判定する第1の判定処理が実施される。第1の判定処理では、差分画像の検査領域81での算出結果V1(検査領域81の輝度)と検査領域81用の判定閾値Th1とが比較され、算出結果V1が判定閾値Th1以上か否かが判定される(ステップS08)。算出結果V1が判定閾値Th1以上の場合(ステップS08でYES)、「部品あり」と判定される(図9A参照)。一方で、算出結果V1が判定閾値Th1よりも小さい場合(ステップS08でNO)、算出結果V1が判定閾値Th1を上限値Hとする所定範囲内か否かが判定される。   Next, a first determination process for determining whether or not the component P is mounted in the inspection area 81 in the window is performed by the determination unit 65 as an initial inspection. In the first determination process, the calculation result V1 of the difference image in the inspection area 81 (luminance of the inspection area 81) is compared with the determination threshold value Th1 for the inspection area 81, and whether or not the calculation result V1 is greater than or equal to the determination threshold value Th1. Is determined (step S08). If the calculation result V1 is greater than or equal to the determination threshold Th1 (YES in step S08), it is determined that “parts are present” (see FIG. 9A). On the other hand, when the calculation result V1 is smaller than the determination threshold value Th1 (NO in step S08), it is determined whether or not the calculation result V1 is within a predetermined range in which the determination threshold value Th1 is the upper limit value H.

所定範囲は再検査が必要か否かを示す範囲であり、上記の判定閾値Th1が上限値Hであるため、ここでは所定範囲の下限値L以上か否かが判定される(ステップS09)。算出結果V1が下限値Lよりも小さい場合(ステップS09でNO)、算出結果V1が所定範囲に含まれないため、部品Pの搭載有無の再検査が実施されることなく「部品なし」と判定される(図9A参照)。一方で、算出結果V1が下限値L以上の場合(ステップS09でYES)、算出結果V1が所定範囲に含まれるため、検査領域81を複数に分割した各分割領域82で部品Pの搭載有無が再検査される(図9A参照)。   The predetermined range is a range indicating whether or not re-examination is necessary. Since the determination threshold Th1 is the upper limit value H, it is determined here whether or not the lower limit value L of the predetermined range is exceeded (step S09). When the calculation result V1 is smaller than the lower limit value L (NO in step S09), the calculation result V1 is not included in the predetermined range, and therefore it is determined that there is no component without performing the re-inspection of whether or not the component P is mounted. (See FIG. 9A). On the other hand, if the calculation result V1 is equal to or greater than the lower limit L (YES in step S09), the calculation result V1 is included in the predetermined range, and therefore whether or not the component P is mounted in each divided area 82 obtained by dividing the inspection area 81 into a plurality of areas. Re-inspected (see FIG. 9A).

このように、ウインドウ内の検査領域81に対する初期検査で、算出結果V1が判定閾値Th1付近であるにも関わらず基板Wの搭載面に部品Pが搭載されていないと判定された場合にのみ、分割領域82毎に部品Pの搭載有無が再検査される。よって、基板Wの載置面に明らかに部品Pが搭載されていない場合には再検査されず、基板Wの載置面に部品Pが搭載されている可能性がある場合にのみ再検査させることができる。なお、所定範囲は、例えば判定閾値Th1の100%の上限値Hと判定閾値Th1の85%の下限値Lで設定されることが好ましい。   In this way, only when it is determined in the initial inspection for the inspection area 81 in the window that the component P is not mounted on the mounting surface of the substrate W even though the calculation result V1 is near the determination threshold Th1. The presence or absence of the component P is re-inspected for each divided area 82. Therefore, when the component P is clearly not mounted on the mounting surface of the substrate W, the re-inspection is not performed, but only when there is a possibility that the component P is mounted on the mounting surface of the substrate W. be able to. The predetermined range is preferably set with, for example, an upper limit value H of 100% of the determination threshold value Th1 and a lower limit value L of 85% of the determination threshold value Th1.

初期検査用の算出結果V1が所定範囲に含まれると、再検査として判定部65にて各分割領域82で部品Pの搭載有無を判定する第2の判定処理が実施される。第2の判定処理では、差分画像の各分割領域82の算出結果V2(各分割領域82の輝度)と分割領域82用の判定閾値Th2とが比較され、差分画像の各分割領域82のいずれかで算出結果V2が判定閾値Th2以上か否かが判定される(ステップS10)。各分割領域82のいずれかで算出結果V2が判定閾値Th2以上の場合には(ステップS10でYES)、いずれかの分割領域82で部品Pの搭載前画像と搭載後画像の差分が明確だとして「部品あり」と判定される(図9B参照)。   When the calculation result V1 for the initial inspection is included in the predetermined range, a second determination process for determining whether or not the component P is mounted in each divided region 82 is performed by the determination unit 65 as re-inspection. In the second determination process, the calculation result V2 (the luminance of each divided region 82) of each divided region 82 of the difference image is compared with the determination threshold value Th2 for the divided region 82, and one of the divided regions 82 of the difference image is selected. In step S10, it is determined whether the calculation result V2 is greater than or equal to the determination threshold Th2. If the calculation result V2 is greater than or equal to the determination threshold Th2 in any of the divided regions 82 (YES in step S10), it is assumed that the difference between the pre-mounting image and the post-mounting image of the component P is clear in any of the divided regions 82. It is determined that “there is a part” (see FIG. 9B).

一方で、全ての分割領域82で算出結果V2が判定閾値Th2よりも小さい場合には(ステップS10でNO)、全ての分割領域82で部品Pの搭載前画像と搭載後画像に差分が無いとして「部品なし」と判定される(図9B参照)。分割領域82の判定閾値Th2は、ウインドウ内の検査領域81に対応した判定閾値Th1を分割領域82の分割数(本実施の形態では4分割)で除算した除算結果を1.5倍して、除算結果にマージンを設けた値である。これにより、分割領域82の大きさに適した判定閾値Th2によって部品Pの搭載有無を検査することができる。また、分割領域82毎に差分箇所が強調されることで輝度が増加傾向にあるため、マージンによって判定閾値Th2を高くしてバランスを調整することができる。   On the other hand, if the calculation result V2 is smaller than the determination threshold Th2 in all the divided regions 82 (NO in step S10), it is assumed that there is no difference between the pre-mounting image and the post-mounting image of the component P in all the divided regions 82. It is determined that “no parts” (see FIG. 9B). The determination threshold value Th2 of the divided region 82 is 1.5 times the division result obtained by dividing the determination threshold value Th1 corresponding to the inspection region 81 in the window by the number of divisions of the divided region 82 (four divisions in this embodiment). A value obtained by providing a margin for the division result. Thereby, it is possible to inspect whether or not the component P is mounted using the determination threshold Th2 suitable for the size of the divided region 82. Further, since the luminance tends to increase by emphasizing the difference points for each divided region 82, the balance can be adjusted by increasing the determination threshold Th2 by the margin.

なお、除算結果に対するマージンの設け方は特に限定されないが、除算結果よりも高くなるようにマージンが設けられればよい。また、上記の説明では、差分画像の検査領域81の輝度を算出した後に続けて、差分画像の各分割領域の輝度を算出する構成について説明したが、差分画像の各分割領域82の輝度の算出タイミングは検査領域81の輝度の算出直後に限定されない。差分画像の各分割領域82の輝度の算出は、再検査時に実施されてもよいし、差分画像の検査領域81の輝度の算出と同時並行で実施されてもよい。上記のフローチャートは、あくまでも一例を示しており、適宜順番を入れ替えることは可能である。   Note that a method of providing a margin for the division result is not particularly limited, but a margin may be provided so as to be higher than the division result. Further, in the above description, the configuration in which the luminance of each divided region of the difference image is calculated following the calculation of the luminance of the inspection region 81 of the difference image has been described. However, the luminance of each divided region 82 of the difference image is calculated. The timing is not limited to immediately after the luminance of the inspection area 81 is calculated. The calculation of the luminance of each divided region 82 of the difference image may be performed at the time of re-examination, or may be performed simultaneously with the calculation of the luminance of the inspection region 81 of the difference image. The above flowchart shows an example to the last, and the order can be changed as appropriate.

以上のように、本実施の形態の検査装置60は、ウインドウ内の検査領域81を複数に分割した各分割領域82で搭載前画像と搭載後画像が比較されるため、各分割領域82における搭載前画像と搭載後画像の差分箇所が強調される。よって、検査領域81全体では搭載前画像と搭載後画像の差分箇所の違いが曖昧であっても、分割領域82毎に搭載前画像と搭載後画像を比較することで、搭載前画像と搭載後画像の差分箇所が強調されて部品Pの搭載有無を高精度に検査することができる。   As described above, in the inspection apparatus 60 according to the present embodiment, the pre-mounting image and the post-mounting image are compared in each divided region 82 obtained by dividing the inspection region 81 in the window into a plurality of parts. The difference between the front image and the post-mount image is emphasized. Therefore, even if the difference in the difference between the pre-mounting image and the post-mounting image is vague in the entire inspection area 81, the pre-mounting image and the post-mounting image are compared for each divided region 82 by comparing the pre-mounting image and the post-mounting image. The difference portion of the image is emphasized, and the presence / absence of the component P can be inspected with high accuracy.

なお、本発明は上記実施の形態に限定されず、種々変更して実施することが可能である。上記実施の形態において、添付図面に図示されている大きさや形状などについては、これに限定されず、本発明の効果を発揮する範囲内で適宜変更することが可能である。その他、本発明の目的の範囲を逸脱しない限りにおいて適宜変更して実施することが可能である。   In addition, this invention is not limited to the said embodiment, It can change and implement variously. In the above-described embodiment, the size, shape, and the like illustrated in the accompanying drawings are not limited to this, and can be appropriately changed within a range in which the effect of the present invention is exhibited. In addition, various modifications can be made without departing from the scope of the object of the present invention.

例えば、本実施の形態において、判定部65は、差分画像の複数の分割領域82のいずれかで輝度が分割領域用の判定閾値Th2以上か否かに基づいて部品Pの有無を判定したが、この構成に限定されない。判定部65は、分割領域82毎に搭載前画像と搭載後画像を比較することで、部品Pの搭載有無を判定すればよい。例えば、例えば、搭載前画像と搭載後画像のそれぞれの分割領域のマッチング処理にて算出された相関値によって部品Pの搭載有無が検査されてもよい。また、分割領域82に優先順序を設けて、一つの分割領域で部品Pの搭載有りが判定できれば、残された他の分割領域での部品搭載の有無の判定を省略しても良い。   For example, in the present embodiment, the determination unit 65 determines the presence / absence of the component P based on whether the luminance is equal to or higher than the determination threshold Th2 for the divided region in any of the plurality of divided regions 82 of the difference image. It is not limited to this configuration. The determination unit 65 may determine whether or not the component P is mounted by comparing the pre-mounting image and the post-mounting image for each divided region 82. For example, the presence / absence of the component P may be inspected based on the correlation value calculated in the matching process of the divided areas of the pre-mounting image and the post-mounting image. In addition, if the priority order is provided in the divided area 82 and it can be determined that the component P is mounted in one divided area, the determination of whether or not the component is mounted in the other divided areas may be omitted.

また、本実施の形態において、検査装置60が実装装置1に備えられている構成について説明したが、この構成に限定されない。検査装置60は、部品の搭載有無の検査が必要な加工装置に備えられていればよい。また、検査装置60は、実装装置1とは別体の検査専用の装置でもよい。   Moreover, in this Embodiment, although the structure with which the inspection apparatus 60 was equipped in the mounting apparatus 1 was demonstrated, it is not limited to this structure. The inspection device 60 only needs to be provided in a processing device that requires inspection of whether or not a component is mounted. Further, the inspection device 60 may be a device dedicated to inspection separate from the mounting device 1.

また、本実施の形態において、撮像装置としての部品撮像部48が斜め上方から基板Wを撮像する構成にしたが、この構成に限定されない。撮像装置は搭載前画像と搭載後画像を比較可能に撮像できればよく、例えば、真上から基板Wを撮像してもよい。また上述した、ヘッド本体41に設けた基板W上のBOCマークを真上から撮像する基板撮像部を用いて撮像してもよい。なお、斜め上方から基板Wを撮像する部品撮像部48は、ヘッド本体41が移動することなく、基板に搭載された部品Pの周辺画像を撮像することができるようにその撮影領域が設定されている。   Moreover, in this Embodiment, although the components imaging part 48 as an imaging device was set as the structure imaged the board | substrate W from diagonally upward, it is not limited to this structure. The imaging device only needs to be able to capture the pre-mounting image and the post-mounting image in a comparable manner. For example, the substrate W may be imaged from directly above. Moreover, you may image using the board | substrate imaging part which images the BOC mark on the board | substrate W provided in the head main body 41 mentioned above from right above. Note that the imaging region of the component imaging unit 48 that images the substrate W from diagonally above is set so that the peripheral image of the component P mounted on the substrate can be captured without the head body 41 moving. Yes.

また、本実施の形態において、判定部65が差分画像のウインドウ内の検査領域81の輝度として、検査領域81の各画素の輝度の二乗和に基づいて部品Pの搭載有無を判定する構成にしたが、この構成に限定されない。判定部65は、検査領域81の輝度に基づいて部品Pの搭載有無を判定する構成であればよい。例えば、判定部65は、検査領域81の各画素の輝度値の合計で部品Pの搭載有無を判定してもよいし、検査領域81の各画素の輝度の三乗和に基づいて部品Pの搭載有無を判定してもよい。   In the present embodiment, the determination unit 65 is configured to determine whether or not the component P is mounted based on the square sum of the luminance of each pixel in the inspection area 81 as the luminance of the inspection area 81 in the window of the difference image. However, it is not limited to this configuration. The determination unit 65 may be configured to determine whether or not the component P is mounted based on the luminance of the inspection area 81. For example, the determination unit 65 may determine whether or not the component P is mounted based on the sum of the luminance values of the pixels in the inspection region 81, or the component P based on the sum of the cubes of the luminances of the pixels in the inspection region 81. The presence or absence of mounting may be determined.

また、本実施の形態において、判定部65が差分画像の各分割領域82の輝度として、各分割領域82の各画素の輝度の二乗和に基づいて部品Pの搭載有無を判定する構成にしたが、この構成に限定されない。判定部65は、各分割領域82の輝度に基づいて部品Pの搭載有無を判定する構成であればよい。例えば、判定部65は、各分割領域82の各画素の輝度値の合計で部品Pの搭載有無を判定してもよいし、各分割領域82の各画素の輝度の三乗和に基づいて部品Pの搭載有無を判定してもよい。   In the present embodiment, the determination unit 65 determines whether or not the component P is mounted based on the sum of squares of the luminance of each pixel in each divided region 82 as the luminance of each divided region 82 of the difference image. The configuration is not limited to this. The determination unit 65 may be configured to determine whether or not the component P is mounted based on the luminance of each divided region 82. For example, the determination unit 65 may determine whether or not the component P is mounted based on the sum of the luminance values of the pixels in each divided region 82, or the component based on the sum of the cubes of the luminance of the pixels in each divided region 82. Whether or not P is mounted may be determined.

また、本実施の形態において、基板Wは、プリント基板に限定されず、治具基板上に載せられたフレキシブル基板であってもよい。   Moreover, in this Embodiment, the board | substrate W is not limited to a printed circuit board, The flexible substrate mounted on the jig | tool board | substrate may be sufficient.

また、本実施の形態において、検査装置60は、検査領域81で部品Pの搭載有無を判定する初期検査後の再検査として、分割領域82毎に部品Pの搭載有無を判定する構成にしたが、この構成に限定されない。検査装置60は、初期検査時に分割領域82毎に部品Pの搭載有無を判定してもよい。すなわち、検査装置60は、検査プログラムのリトライ処理を実施しなくてもよい。   In the present embodiment, the inspection apparatus 60 is configured to determine whether the component P is mounted for each divided region 82 as a re-inspection after the initial inspection for determining whether the component P is mounted in the inspection region 81. The configuration is not limited to this. The inspection device 60 may determine whether or not the component P is mounted for each divided region 82 during the initial inspection. That is, the inspection device 60 may not perform the inspection program retry process.

また、本実施の形態において、検査領域81用の判定閾値Th1を上限値とした所定範囲に、差分画像の検査領域81の輝度が含まれる場合に再検査が実施される構成にしたが、この構成に限定されない。差分画像の検査領域81の輝度が検査領域81用の判定閾値Th1より小さい場合に再検査が実施されてもよい。これにより、検査領域81に対する初期検査で基板Wの搭載面に部品Pが搭載されていないと判定された場合に、分割領域82毎に部品Pの搭載有無を再検査することができる。   In the present embodiment, the re-inspection is performed when the luminance of the inspection area 81 of the difference image is included in the predetermined range with the determination threshold value Th1 for the inspection area 81 as an upper limit. It is not limited to the configuration. The re-inspection may be performed when the brightness of the inspection area 81 of the difference image is smaller than the determination threshold Th1 for the inspection area 81. Thereby, when it is determined in the initial inspection for the inspection region 81 that the component P is not mounted on the mounting surface of the substrate W, it is possible to re-inspect whether the component P is mounted for each divided region 82.

さらに、本発明のプログラムは、上記の検査方法を検査装置に実行させている。プログラムには、初期検査用のプログラムとして上記の検査方法が組み込まれてもよいし、再検査用のプログラムとして上記の検査方法が組み込まれてもよい。   Furthermore, the program of the present invention causes the inspection apparatus to execute the above inspection method. In the program, the above inspection method may be incorporated as a program for initial inspection, or the above inspection method may be incorporated as a program for reinspection.

以上説明したように、本発明は、簡易な処理により、基板に対する部品の搭載有無を高精度に検査することができるという効果を有し、特に、リードが設けられた比較的大きな部品の搭載有無を検査する検査装置、実装装置、検査方法及びプログラムに有用である。   As described above, the present invention has an effect that the presence / absence of mounting of a component on a substrate can be inspected with high accuracy by a simple process, and in particular, mounting / dismounting of a relatively large component provided with a lead. This is useful for an inspection apparatus, a mounting apparatus, an inspection method, and a program for inspecting the above.

1 実装装置
40 実装ヘッド
48 部品撮像部
60 検査装置
61 マッチング部
62 差分画像生成部
63 領域設定部
64 算出部
65 判定部
81 検査領域
82 分割領域
P 部品
W 基板
DESCRIPTION OF SYMBOLS 1 Mounting apparatus 40 Mounting head 48 Component imaging part 60 Inspection apparatus 61 Matching part 62 Difference image generation part 63 Area setting part 64 Calculation part 65 Judgment part 81 Inspection area 82 Divided area P Component W Board | substrate

Claims (8)

基板の搭載面に対する部品の搭載前後を撮像した前記部品の搭載前画像と搭載後画像から前記部品の搭載有無を検査する検査装置であって、
前記搭載前画像と前記搭載後画像に対する検査領域を複数に分割した分割領域を設定する領域設定部と、
前記分割領域に対して前記搭載前画像と前記搭載後画像を比較することで前記部品の搭載有無を判定する判定部とを備えたことを特徴とする検査装置。
An inspection apparatus for inspecting the presence / absence of mounting of the component from the pre-mounting image and post-mounting image of the component obtained by imaging before and after mounting of the component on the mounting surface of the substrate,
An area setting unit for setting a divided area obtained by dividing the inspection area for the pre-mounting image and the post-mounting image into a plurality of areas;
An inspection apparatus comprising: a determination unit that determines whether the component is mounted by comparing the pre-mounting image and the post-mounting image with respect to the divided area.
前記搭載前画像と前記搭載後画像から差分画像を生成する差分画像生成部を備え、
前記領域設定部が、前記搭載前画像と前記搭載後画像の差分画像に対して前記複数の分割領域を設定し、
前記判定部が、前記差分画像の前記複数の分割領域のいずれかで輝度が分割領域用の判定閾値以上か否かに基づいて前記部品の有無を判定することを特徴とする請求項1に記載の検査装置。
A difference image generation unit that generates a difference image from the pre-mounting image and the post-mounting image;
The region setting unit sets the plurality of divided regions for the difference image between the pre-mounting image and the post-mounting image;
The said determination part determines the presence or absence of the said component based on whether the brightness | luminance is more than the determination threshold value for division areas in any of the said some division area of the said difference image. Inspection equipment.
前記部品の搭載有無の検査は、前記差分画像の前記検査領域の輝度が前記検査領域用の判定閾値より小さい場合の再検査であることを特徴とする請求項2に記載の検査装置。   The inspection apparatus according to claim 2, wherein the inspection of whether or not the component is mounted is a re-inspection when the luminance of the inspection area of the difference image is smaller than a determination threshold for the inspection area. 前記部品の搭載有無の検査は、前記検査領域用の判定閾値を上限値とした所定範囲に、前記差分画像の前記検査領域の輝度が含まれる場合の再検査であることを特徴とする請求項3に記載の検査装置。   The inspection of whether or not the component is mounted is a re-inspection when the luminance of the inspection area of the difference image is included in a predetermined range with the determination threshold for the inspection area as an upper limit. 3. The inspection apparatus according to 3. 前記分割領域用の判定閾値が、前記検査領域用の判定閾値を前記分割領域の分割数で除算した除算結果にマージンを設けた値であることを特徴とする請求項2から請求項4のいずれかに記載の検査装置。   5. The division threshold value for the divided area is a value obtained by providing a margin to a division result obtained by dividing the determination threshold value for the inspection area by the number of divisions of the divided area. Crab inspection device. 請求項1から請求項5のいずれかに記載の検査装置と、
前記搭載面に対して前記部品を搭載する実装ヘッドと、
前記実装ヘッドによる部品の搭載前後を撮像する撮像装置とを備えたことを特徴とする実装装置。
The inspection apparatus according to any one of claims 1 to 5,
A mounting head for mounting the component on the mounting surface;
A mounting apparatus comprising: an imaging device that images before and after mounting of a component by the mounting head.
基板の搭載面に対する部品の搭載前後を撮像した前記部品の搭載前画像と搭載後画像から前記部品の搭載有無を検査する検査方法であって、
前記搭載前画像と前記搭載後画像に対する検査領域を複数に分割した分割領域を設定するステップと、
前記分割領域に対して前記搭載前画像と前記搭載後画像を比較することで前記部品の搭載有無を判定するステップとを有することを特徴とする検査方法。
An inspection method for inspecting the presence / absence of mounting of the component from the pre-mounting image and the post-mounting image of the component obtained by imaging before and after mounting of the component on the mounting surface of the substrate,
Setting a divided region obtained by dividing a plurality of inspection regions for the pre-mounting image and the post-mounting image;
And a step of determining whether or not the component is mounted by comparing the pre-mounting image and the post-mounting image with respect to the divided region.
請求項7に記載の検査方法を検査装置に実行させることを特徴とするプログラム。   A program for causing an inspection apparatus to execute the inspection method according to claim 7.
JP2016033073A 2016-02-24 2016-02-24 Inspection device, mounting device, inspection method, and program Active JP6745117B2 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2016033073A JP6745117B2 (en) 2016-02-24 2016-02-24 Inspection device, mounting device, inspection method, and program
CN201710102785.7A CN107124865B (en) 2016-02-24 2017-02-24 Inspection apparatus, mounting apparatus, and inspection method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2016033073A JP6745117B2 (en) 2016-02-24 2016-02-24 Inspection device, mounting device, inspection method, and program

Publications (2)

Publication Number Publication Date
JP2017150927A true JP2017150927A (en) 2017-08-31
JP6745117B2 JP6745117B2 (en) 2020-08-26

Family

ID=59717200

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2016033073A Active JP6745117B2 (en) 2016-02-24 2016-02-24 Inspection device, mounting device, inspection method, and program

Country Status (2)

Country Link
JP (1) JP6745117B2 (en)
CN (1) CN107124865B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPWO2021124386A1 (en) * 2019-12-16 2021-06-24
JP2022011358A (en) * 2020-06-30 2022-01-17 ヤマハ発動機株式会社 Confirmation device and surface mounter
WO2023149192A1 (en) * 2022-02-02 2023-08-10 パナソニックIpマネジメント株式会社 Processing system, inspection system, processing method, and program

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112005624B (en) * 2018-05-31 2022-04-26 雅马哈发动机株式会社 Recognition parameter optimization device, component mounting system, and recognition parameter optimization method
JP7094366B2 (en) * 2018-07-19 2022-07-01 株式会社Fuji Inspection setting device and inspection setting method

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0431976A (en) * 1990-05-28 1992-02-04 Seikosha Co Ltd Image monitoring method
US20040075067A1 (en) * 2002-10-16 2004-04-22 Mcauliffe Robert E. Method for detecting missing components at electrical board test using optoelectronic fixture-mounted sensors
JP2008051781A (en) * 2006-08-28 2008-03-06 I-Pulse Co Ltd Visual examination method of substrate and visual examination device of substrate
JP2012039096A (en) * 2010-07-13 2012-02-23 Fuji Mach Mfg Co Ltd Device and method for determining presence/absence of component
JP2015079933A (en) * 2013-09-10 2015-04-23 Juki株式会社 Inspection method, mounting method, and mounting device

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1979137A (en) * 2005-01-11 2007-06-13 欧姆龙株式会社 Substrate inspection device, method and device for setting inspection logic
JP2007101415A (en) * 2005-10-05 2007-04-19 Shigeki Kobayashi Inspection device
JP2011082243A (en) * 2009-10-05 2011-04-21 Panasonic Corp Component mounting device, and mounting-state inspection method in the same
KR101175595B1 (en) * 2011-09-26 2012-08-24 주식회사 미르기술 Apparatus and method for noncontact inspection for components
JP6021560B2 (en) * 2012-09-28 2016-11-09 Juki株式会社 Parts inspection method and apparatus

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0431976A (en) * 1990-05-28 1992-02-04 Seikosha Co Ltd Image monitoring method
US20040075067A1 (en) * 2002-10-16 2004-04-22 Mcauliffe Robert E. Method for detecting missing components at electrical board test using optoelectronic fixture-mounted sensors
JP2008051781A (en) * 2006-08-28 2008-03-06 I-Pulse Co Ltd Visual examination method of substrate and visual examination device of substrate
JP2012039096A (en) * 2010-07-13 2012-02-23 Fuji Mach Mfg Co Ltd Device and method for determining presence/absence of component
JP2015079933A (en) * 2013-09-10 2015-04-23 Juki株式会社 Inspection method, mounting method, and mounting device

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPWO2021124386A1 (en) * 2019-12-16 2021-06-24
WO2021124386A1 (en) * 2019-12-16 2021-06-24 株式会社Fuji Component mounter
JP7439129B2 (en) 2019-12-16 2024-02-27 株式会社Fuji parts mounting machine
JP2022011358A (en) * 2020-06-30 2022-01-17 ヤマハ発動機株式会社 Confirmation device and surface mounter
JP7340493B2 (en) 2020-06-30 2023-09-07 ヤマハ発動機株式会社 Confirmation equipment, surface mount machine
WO2023149192A1 (en) * 2022-02-02 2023-08-10 パナソニックIpマネジメント株式会社 Processing system, inspection system, processing method, and program

Also Published As

Publication number Publication date
CN107124865A (en) 2017-09-01
JP6745117B2 (en) 2020-08-26
CN107124865B (en) 2021-02-05

Similar Documents

Publication Publication Date Title
CN107124865B (en) Inspection apparatus, mounting apparatus, and inspection method
JP6389651B2 (en) Inspection method, mounting method, and mounting apparatus
JP6571185B2 (en) Component mounting apparatus and component mounting method
KR102534983B1 (en) Apparatus and method for detecting attitude of electronic component
CN109040537B (en) Inspection device, mounting device, and inspection method
WO2007033349A1 (en) Pick and place machine with improved component pick image processing
JP6706717B2 (en) Transfer status inspection system and component mounter
JP2017152651A (en) Component inspection device and component mounting device
CN106455478B (en) Inspection apparatus, mounting apparatus, and inspection method
JP4883636B2 (en) Electronic component orientation inspection apparatus, electronic component orientation inspection method, and electronic component placement machine
JP2006019380A (en) Method and apparatus for inspecting liquid agent transfer condition and surface mounting apparatus
JPWO2017081773A1 (en) Image processing apparatus and image processing method for substrate
JP6466960B2 (en) Mounting machine and electronic component suction posture inspection method using mounting machine
EP3937598A1 (en) Correction amount calculating device and correction amount calculating method
JP5297913B2 (en) Mounting machine
KR20040096277A (en) A inspection system for the metal mask and the inspecting method thereof
KR101126759B1 (en) Method of teaching for electronic parts information in chip mounter
EP3941176A1 (en) Object determination method and object determination device
JP2013011569A (en) Displacement amount identification device, displacement amount identification method, and displacement amount identification program
JP6418739B2 (en) Evaluation device, surface mounter, evaluation method
JP2018014516A (en) Component mounting device and component mounting method
KR101779020B1 (en) Method for mounting components
JP6751567B2 (en) Electronic component inspection method, electronic component mounting method, and electronic component mounting device
WO2019207730A1 (en) Component mounting device and inspection method
JP2020004923A (en) Image processing apparatus, mounting apparatus, image processing method, and program

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20190121

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20191009

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20191203

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20200131

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20200421

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20200622

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20200714

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20200803

R150 Certificate of patent or registration of utility model

Ref document number: 6745117

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150