EP0954387A1 - Optisch-geleitetes zeichenabtastsystem - Google Patents

Optisch-geleitetes zeichenabtastsystem

Info

Publication number
EP0954387A1
EP0954387A1 EP98902706A EP98902706A EP0954387A1 EP 0954387 A1 EP0954387 A1 EP 0954387A1 EP 98902706 A EP98902706 A EP 98902706A EP 98902706 A EP98902706 A EP 98902706A EP 0954387 A1 EP0954387 A1 EP 0954387A1
Authority
EP
European Patent Office
Prior art keywords
scanner
conveyor
parcel
indicia reader
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP98902706A
Other languages
English (en)
French (fr)
Other versions
EP0954387B1 (de
Inventor
Henri Bonnet
Johannes A. S. Bjorner
Bennett Pardee
Catherine Alexander
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
United Parcel Service of America Inc
United Parcel Service Inc
Original Assignee
United Parcel Service of America Inc
United Parcel Service Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US08/787,974 external-priority patent/US5920056A/en
Priority claimed from US08/788,852 external-priority patent/US5923017A/en
Application filed by United Parcel Service of America Inc, United Parcel Service Inc filed Critical United Parcel Service of America Inc
Priority to EP03020263A priority Critical patent/EP1371424B1/de
Publication of EP0954387A1 publication Critical patent/EP0954387A1/de
Application granted granted Critical
Publication of EP0954387B1 publication Critical patent/EP0954387B1/de
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B07SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07CPOSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C3/00Sorting according to destination
    • B07C3/10Apparatus characterised by the means used for detection ofthe destination
    • B07C3/14Apparatus characterised by the means used for detection ofthe destination using light-responsive detecting means

Definitions

  • the present invention relates to image processing and more particularly relates to over- the-belt optical character recognition (OCR) systems.
  • OCR optical character recognition
  • the invention relates to an indicia reader system that includes a projected optical guide to assist the positioning of parcels on a conveyor.
  • OCR optical character recognition
  • Parcel delivery companies such as United Parcel Service, ship millions of parcels every day. These parcel delivery companies make extensive use of OCR systems to read the destination address labels on parcels to facilitate sorting and routing the parcels to their proper destinations.
  • the fundamental physical components of an OCR system are a scanner and a character recognition system including a central processing unit (CPU), a computer memory, and a sophisticated character recognition program module.
  • the scanner is typically an optical camera, such as a charge-coupled device (CCD) array, that captures an image of the destination address on the parcels as they travel past the scanner on the conveyor.
  • CCD charge-coupled device
  • a continuous video image of the conveyor carrying the parcels is captured by the scanner, which video image is converted into digital format and transmitted to the character recognition system.
  • the OCR system therefore, must have some way to identify the portions of the video image that need to be processed by the character recognition system.
  • Data compression is one technique for reducing the memory storage requirement.
  • the video data may be compressed for storage using any of a variety of well known data compression methods, such as run length encoding. These data compression techniques, however, alter the bit-map format of the data. This is undesirable because it is advantageous for the character recognition program module to operate on bit maps that allow easy access to information regarding neighborhoods around individual pixels.
  • the compressed data must therefore be uncompressed, typically into a frame buffer, for processing by the character recognition program module. Compressing the video data for storage, and then uncompressing the video data for processing, burdens the CPU and slows the character recognition process.
  • Real-time extraction of the desired portions of the video data is another technique for reducing the memory storage requirement.
  • real-time data extraction is a very effective technique because most of the video data created by the continuously running scanner is a useless image of the conveyor and the non-indicia bearing areas of the parcels moving along the conveyor; only a small percentage of the data includes the destination addresses of the parcels to be shipped. Therefore, extracting only small portions of the video data, such as relatively small areas covering the destination addresses, greatly reduces the memory storage requirement and speeds up the character recognition process.
  • Tonkin U.S. Patent No. 4,742,555
  • Tonkin U.S. Patent No. 4,742,555
  • a mechanical limit switch, optical sensor, or magnetic sensor that triggers a video system to capture and store an image of a parcel as the parcel reaches a predetermined location along a conveyor.
  • Tonkin would have a significant drawback if applied to a parcel shipping system. This is because the system described by Tonkin captures an image of the entire parcel; is not operative for capturing only a specific portion of the image, such as the destination address.
  • the destination address In a parcel shipping system, the destination address must be captured for sorting and routing purposes, but other indicia on the parcel, such as the return address, is not needed to route the parcel to its proper destination. It is therefore advantageous to identify the destination address prior to storing the image of the parcel, so that only the portion of the image containing the destination address may be stored in the computer memory.
  • the destination addresses may vary in size, and may be in different locations on different parcels.
  • the parcels themselves may vary in size, shape, and position on the conveyor.
  • Tonkin the exact position of a destination address on a parcel cannot be determined by simply detecting the edge of the parcel using a hmit switch or sensor, as described by Tonkin.
  • Kizu et al. U.S. Patent No. 4,516,265
  • the system includes a low resolution prescanner that coarsely scans the surface of the envelope.
  • the position of the destination address block is determined from the coarse scan, and the coordinates of the destination address block with respect to the leading edge of the envelope are then passed to a second, high-resolution camera system.
  • the second camera system stores an image of the destination address block by first detecting the leading edge of the envelope.
  • the second camera system begins storing an image of the destination address block when the block reaches the second camera, and stops storing the image when the block moves past the second camera.
  • a postal code reader subsequently processes the high-resolution scan to read the postal code.
  • Another example is disclosed in Morton et al., U.S. Patent No. 5,642,442. This patent describes a two-camera system that reads the destination addresses on parcels traveling on a conveyor. A fluorescent ink fiduciary mark is superimposed relative to the destination address on a parcel. A first camera captures an image of the fiduciary mark, the position and orientation of which is ascertained.
  • the position and orientation of the fiduciary mark is then used to extract an image of the destination address from a video data signal created by a second camera, which is positioned downstream from the first camera.
  • the image of the destination address is stored in a computer memory for subsequent processing by a character recognition system.
  • the two-camera systems described above are very effective at minimizing the amount of video data that must be stored in an OCR system. They are, however, rather expensive systems that are best suited for very high-speed parcel handling systems. The cost associated with these systems may not be justified for many lower-speed parcel handling systems. There is, therefore, a need for a less expensive system for minimizing the amount of video data that must be stored in an OCR system. In particular, there is a need for an inexpensive indicia reader system that is suited to low- to medium-speed parcel handling systems.
  • the invention seeks to provide a low-cost system for minimizing the amount of video data that must be stored in an OCR system.
  • the invention seeks to provide an inexpensive indicia reader system suited to low- to medium-speed parcel handling systems.
  • these objects are accomplished in an indicia reader system that includes an optical guide to assist the positioning of parcels on a conveyor.
  • the optical guide may include a static elongated strip of light projected toward the conveyor from an illumination source positioned above the conveyor.
  • An operator positions a parcel on the conveyor so that indicia to be imaged, such as the destination address on the parcel, is within static elongated strip of light.
  • the parcel may also be positioned on the conveyor so that other indicia that is not to be imaged, such as the return address on the parcel, is not within the static elongated strip of light. Projecting the optical guide onto the parcel avoids parallax- related alignment errors that could otherwise occur with tall parcels in systems using other types of optical guides, such as reference marks placed on the conveyor itself.
  • the optically-guided indicia reader system may also include a proximity sensor, such as a photo detector, for detecting the arrival of the parcel at the scanner.
  • the computer memory and the scanner may be operated so as to store an image of a region having a width approximately equal to the width of the area defined by the illumination source, and a length approximately equal to the length of the parcel in the direction of conveyor travel.
  • the optically-guided indicia reader system may also include a reflectivity sensor located upstream of the scanner and positioned to determine reflectivity data associated with the parcel.
  • an optically-guided indicia reader system may include a height sensor located above the conveyor and upstream of the scanner and positioned to determine height data associated with the parcel at the location of the destination address.
  • a communication link transmits the height data from the height sensor to the scanner, and the scanner is focused in response to the height data.
  • an operator positions a parcel on the conveyor so that indicia to be imaged, such as the destination address on the parcel, coincides with a moving spot light defined by an illumination source.
  • a scanner located downstream of the illumination source captures an image of a region that is defined with respect to the spot defined by the illumination source.
  • the optical guide may include a moving light, such as a narrow-beam spot light, that is projected onto the conveyor, and that travels at the same speed as the conveyor. Positioning a parcel on the conveyor so that the spot light is located in the center of the city and state address lines of the destination address allows a scanner to efficiently capture an image of the destination address for processing by a character recognition reader.
  • the moving-light indicia reader system includes a conveyor for transporting a parcel from an upstream location of the conveyor to a downstream location of the conveyor.
  • a moving-light system which is preferably positioned above the conveyor, includes an illumination source for defining a spot that moves at the same speed as the conveyor to assist in positioning the parcel on the conveyor.
  • a scanner which is located downstream from the moving-light illumination source, and a processing module are operated so as to store an image of a region of the parcel defined with respect to the spot defined by the moving-light illumination source.
  • the illumination source may define a spot that is substantially smaller than the region to be imaged by the scanner of the indicia reader system. An operator may then position a parcel so that the spot defined by the moving-light illumination source is located approximately in the center of the city and state lines of the destination address on the parcel.
  • a moving-light indicia reader system may also include a reflectivity sensor located upstream of the scanner and positioned to determine reflectivity data associated with the parcel. Reflectivity data from the reflectivity sensor is obtained for the spot defined by the moving-light illumination source so that reflectivity data is collected regarding the parcel at the region to be captured and stored for subsequent processing by a character recognition system. A communication link transmits this reflectivity data from the reflectivity sensor to the scanner, and the gain of the scanner is adjusted in response to the reflectivity data.
  • a moving-light indicia reader system may include a height sensor located above and upstream of the scanner and positioned to determine height data associated with the parcel at the location of the destination address.
  • Height data from the height sensor is obtained for the spot defined by the moving-light illumination source so that height data is collected regarding the parcel at the region to be captured and stored for subsequent processing by the character recognition system.
  • a communication link transmits this height data from the height sensor to the scanner, and the scanner is focused in response to the height data.
  • a multi-conveyor indicia reader system includes a plurality of moving-light indicia reader systems, wherein each moving-light indicia reader system includes an illumination source for defining a spot that moves at the same speed as its respective conveyor to assist in positioning parcels on the conveyor.
  • the processing module and the scanner of each moving-light indicia reader system are operated so as to store an image of a region of the parcel defined with respect to the spot defined by the illumination source.
  • the illumination sources are operated so as to time-division multiplex the storage of the images generated by the several moving-light indicia reader systems.
  • FIG. 1 is a diagram of an optically-guided indicia reader system.
  • FIG. 2 including FIGS. 2A-2E, illustrates a moving-light indicia reader system.
  • FIG. 3 illustrates a parcel with the spot defined by the moving-light system located approximately in the center of the city and state address lines of the destination address.
  • FIG. 4 illustrates a multi-conveyor indicia reader system.
  • FIG. 1 is a diagram of an optically-guided indicia reader system 10 that includes a conveyer 12 carrying a parcel 14 from an upstream location 16 to a downstream location 18 of the conveyor 12.
  • the parcel 14 includes indicia to be read by the optically- guided indicia reader system 10, such as a destination address 20.
  • the parcel 14 may include other indicia, such as the return address, that the indicia reader system 10 preferably avoids reading.
  • An illumination source 24 is positioned to define a static area 26 to assist in positioning the parcel 14 on the conveyor 12.
  • the area 26 is static in that it does not travel along with the conveyor 12, but remains stationary with respect to an operator station that is located along side the conveyor 12. Thus, the conveyor 12 travels through the area 26, which is defined by light projected by the illumination source 24.
  • the illumination source 24 may define the area 26 many different ways. For example, illumination source 24 may illuminate the interior of area 26, or the border of the area 26, or two spaced-apart parallel lines in the direction of conveyor travel, etc.
  • the illumination source 24 is positioned a sufficient distance above the conveyor 12 so that the parcel 14 may be positioned on the conveyor 12 to pass beneath the illumination source 24.
  • An operator may therefore view the area 26, which is defined by light projected by the illumination source 24, directly on the parcel 14 as the operator positions the parcel 14 on the conveyor 12.
  • the area 26 is defined by light projected by the illumination source 24, directly on the parcel 14 as the operator positions the parcel 14 on the conveyor 12.
  • a sensor assembly 28, including a height sensor and reflectivity sensor, my be located toward the downstream end of the illumination source 24.
  • a communication link 30 functionally connects the sensor assembly 28 to a scanner 32 that is located downstream from the sensor assembly 28.
  • the scanner 32 is focused in response to height data from the height sensor, and the gain of the scanner 32 is adjusted in response to reflectivity data from the reflectivity sensor, so that the scanner 32 generates a clear image of the top of the parcel 14 as the parcel passes beneath the scanner 32.
  • the scanner 32 is preferably positioned so that the scan line 34 is oriented across a portion of the conveyor 12 that is aligned with the area 26 in the cross-machine direction.
  • a belt encoder 36 measures the displacement of the conveyor 12.
  • the height data from the sensor assembly 28 indicates the presence of a parcel 14 on a particular location of the conveyor 12. Thus, the height data and the signal from the belt encoder 36 may be used to determine when a parcel 14 is present at the scanner 32.
  • a proximity sensor such as a photo sensor 45
  • a communication link 48 functionally connects the scanner 32 to the photo sensor 45.
  • the photo sensor includes a light source 46a that projects a columnar beam of light 50 toward a photo detector 46b.
  • the photo sensor 45 detects the presence of the parcel 14 as it passes by the photo sensor 45 because the parcel breaks the beam of light 50.
  • Many other types of proximity sensors such as a mechanical or a magnetic sensor, may equivalently be used to detect the location of the parcel 14 on the conveyor 12.
  • the belt encoder 36 is a standard belt-driven, opto-mechanical encoder that provides a signal indicating the linear displacement of the conveyor 12.
  • the CCD array of the scanner 32 is cycled in response to the signal from the belt encoder 26 to generate a series of analog images of the scan line 34 that are transmitted to an analog-to-digital converter within the scanner 32.
  • the analog-to-digital converter of the scanner 32 uses a standard thresholding or similar process to convert the analog signal produced by the CCD array of the scanner 32 into an eight-bit digital video signal that is transmitted via the communication link 44 to the character recognition system 40, which stores the video data in the computer memory 42 for subsequent processing.
  • the region to be stored in the computer memory 42 may include all or part of the scan line 34. This may be accomplished by only storing the output of all or only a portion of the cells of the scanner 32.
  • the region to be stored in the computer memory 42 is preferably aligned with and has a length that is approximately equal to the width of the area 26 so that the region stored in the computer memory 42 corresponds to, but is downstream from, the area 26 defined by the illumination source 24. This assists an operator in orienting a parcel 14 so that the destination address 20 can be effectively scanned by the indicia reader system 10 as configured. Alternatively, the operator may determine that the parcel 14 cannot be oriented so that the destination address 20 can be effectively scanned by the indicia reader system 10 as configured.
  • the scanner 32 may run continuously, so that region generated by the scanner 32 and stored in the computer memory 42 of the character recognition system 40 is a continuous strip having a width approximately equal to the width of the area 26 defined by the illumination source 24.
  • the size of the region may be further reduced by using the height data from the sensor assembly 28 or the signal from the photo sensor 45 to trigger the storage of video data generated by the scanner 32.
  • the video data generated by the scanner 32 may be stored in the computer memory 42 only when the beam 50 of the photo sensor 45 is broken.
  • a time delay may be imposed to account for the distance between the beam 50 or the sensor assembly 28 and the scan line 34.
  • an image of a strip of the top of the parcel 14 including the destination address 20 may be stored in the computer memory 42. That is, an image of a region of the parcel 14 having a width approximately equal to the width of the area 26 defined by the illumination source 24, and a length approximately equal to the length of the parcel 14 in the direction of conveyor travel, may be stored in the computer memory 42 of the character recognition system 40.
  • Triggering the storage of the image of the region in the computer memory 42 may be accomplished in several different ways.
  • the scanner 32 may be toggled on and off by the signal from the photo sensor 45 or the sensor assembly 28 (with an appropriate time delay).
  • the scanner 32 may run continuously, and the signal from the photo sensor 45 or the sensor assembly 28 may be used to latch a control line to an input buffer of the character recognition system 40.
  • the signal from the photo sensor 25 or the sensor assembly 28 may be used as an input to a software-based algorithm running on the processing unit 41, which triggers the storage of video data from the scanner 32 in the computer memory 42.
  • Many other means known to those skilled in the art may equivalently be employed to operate the computer memory 42 and the scanner 32 so as to store an image of a region in the computer memory 42.
  • an operator positions the parcel 14 on the conveyor 12 so that the destination address 20 is within the area 26 defined by the illumination source 24.
  • the operator may also position the parcel 14 on the conveyor 12 so that other indicia on the parcel 14, such as the return address 22, is not within the area 26 defined by the illumination source 24.
  • many other types of indicia may be placed within, or excluded from, the area 26, such a barcode, a two-dimensional code, a hologram, etc.
  • the area 26 is typically a rectangular strip that is significantly narrower than the conveyor 12, sufficiently long to allow an operator to easily position a parcel on the conveyor 12 using the optical guide, and approximately in the center of the conveyor 12.
  • the area 26 may be approximately 4 inches (10 cm) in the cross-machine direction and approximately 12 (30 cm) to 36 inches (91 cm) in the direction of conveyor travel.
  • projected illumination rather than an area painted on the conveyor 12, allows the operator to view the area 26 defined by the illumination source 24 directly on the top of the parcel 14. Thus, there is no displacement between the area 26 and the top of the parcel 14 that could cause parallax-related alignment errors with tall parcels.
  • the belt encoder 36 is a standard belt-driven, opto-mechanical encoder that provides a signal indicating the linear displacement of the conveyor 12.
  • the CCD array of the scanner 32 is cycled in response to the signal from the belt encoder 36 to generate a series of analog images of the scan line 34 that are transmitted to an analog-to-digital converter within the scanner 32.
  • the analog-to-digital converter of the scanner 32 uses a standard thresholding or similar process to convert the analog signal produced by the CCD array of the scanner 32 into an eight-bit digital video signal that is transmitted via the communication link 44 to the character recognition system 40, which is operable for storing the video data in the computer memory 42 for subsequent processing.
  • the scanner 32 is preferably a monochrome, 4,096 pixel line-scan type CCD array such as one using a Thompson TH7833A CCD chip.
  • the resolution of the image created by the scanner 32 is approximately 256 pixels or "dots" per inch (DPI) (101 dots per cm) across the field of view of the scanner 32.
  • the belt encoder 36 preferably triggers the CCD array of the scanner 32 at a rate of approximately 256 cycles per inch (101 cycles per cm) so that the resolution of the image created by the scanner 32 is approximately 256 pixels or "dots" per inch (DPI) (101 dots per cm) in the direction of conveyor travel.
  • a digital image with a correct aspect ratio may be generated by the scanner 32 and stored in the computer memory 42 of the character recognition system 40 by synchronizing the cycling rate of the scanner 32 with the linear speed of the conveyor 12. See, for example, Shah et al., U.S. Patent No. 5,291,564, which is incorporated by reference.
  • the conveyor 12 may be approximately 24 inches (61 cm) wide and travel at linear speeds up to 20 inches per second or 100 feet per minute (51 cm per second or 30 meters per minute) or more.
  • the scanner 32 is preferably mounted to have an optical path of approximately 120 inches (304 cm) to the conveyor 12 with a 16 inch (41 cm) field of view at the conveyor 12.
  • the scanner 32 is positioned approximately 30 inches (76 cm) above the center of conveyer 12 and is pointed towards a complex of mirrors (not shown) that increases the optical path from the scanner 32 to the conveyor 12 to approximately 120 inches (305 cm).
  • These parameters may be varied somewhat without unduly affecting the performance of the disclosed embodiment of the present invention. See also, Smith et al., U.S. Patent No. 5,308,960, and Bjorner, et al., U.S. Patent No. 5,485,263, which are incorporated by reference.
  • the scan line 34 may be longer than the width of the region stored in the computer memory 42.
  • the scanner 32 may be positioned to have a field of view (i.e. the scan line 34) equal to approximately 16 inches (41 cm) at the conveyor 12.
  • the region stored in the computer memory 42 may only be approximately 4 inches (10 cm), which preferably corresponds to the width of the area 26 defined by the illumination source 24. This may be accomplished by only storing the output of a portion of the cells of the scanner 32 (e.g., the center 1,024 pixels of a 4,096 pixel scanner) in the computer memory 42.
  • the optically- guided indicia reader system 10 reduces the amount of video data that must be stored in the computer memory 42 of the character recognition system 40.
  • the use of projected illumination allows the operator to view the area 30 defined by the illumination source 24 directly on the top of the parcel 14. Thus, there is no displacement between the area 26 and the top of the parcel 14 that could cause parallax-related alignment errors with tall parcels.
  • the optically-guided indicia reader system 10 allows the angle of the field of view of the scanner 32 to be relatively narrow so that the scanner 32 generates a sharp image of the top of the parcel 14.
  • FIGS. 2A-2E illustrate another embodiment of the invention, a single-conveyor moving- light indicia reader system in which a moving-light iUrrnination source defines a spot that moves at the same speed as a conveyor to assist the positioning of a parcel on the conveyor.
  • FIG. 3 illustrates a parcel in this moving-light indicia reader system with the spot defined by the moving-light illumination source located approximately in the center of the region to be captured. More specifically, the parcel is preferably positioned on the conveyor so that the center of the spot defined by the moving-light illumination source is approximately in the center of the city and state lines of the destination address.
  • FIG. 1 illustrates a single-conveyor moving- light indicia reader system in which a moving-light iUrrnination source defines a spot that moves at the same speed as a conveyor to assist the positioning of a parcel on the conveyor.
  • FIG. 3 illustrates a parcel in this moving-light indicia reader system with the spot defined by the
  • FIG. 4 illustrates a multi-conveyor indicia reader system, in which the illumination sources of a plurality of moving-light indicia reader systems are operated so as to time-division multiplex the storage of the images generated by the several moving-light indicia reader systems.
  • FIGS. 2A-E illustrate a moving-light indicia reader system 200 including a conveyer
  • the moving-light indicia reader system 200 includes indicia to be read by the moving-light indicia reader system 200, such as a destination address 220.
  • the parcel 214 may include other indicia, such as the return address, that the moving-light indicia reader system 200 preferably avoids reading.
  • the moving-light indicia reader system 200 includes a moving-light illumination source 224 that includes a plurality of discrete illumination sources 226a through 226n, such as light- emitting diodes (LEDs), that project columnar beams of light represented by the beam 228.
  • LEDs light- emitting diodes
  • the illumination source 224 is positioned a sufficient distance above the conveyor 212 so that the parcel 214 may be positioned on the conveyor 212 to pass beneath the moving-light illumination source 224.
  • An operator may therefore view the spot 230, which is defined by light projected by the moving-light illumination source 224, directly on the parcel 214 as the operator positions the parcel 214 on the conveyor 212.
  • the spot 230 thus provides an optical guide to assist the operator in positioning the parcel 214 on the conveyor 212.
  • a sensor assembly 232 including a height sensor and reflectivity sensor, is located toward the downstream end of the illumination source 224.
  • a communication link 234 functionally connects the sensor assembly 232 to a scanner 236 that is located downstream from the sensor assembly 232.
  • the scanner 236 includes internal components 233 well known to those skilled in the art to automatically focus the scanner 236 in response to height data from the height sensor, and to automatically adjust the gain of the scanner 236 in response to reflectivity data from the reflectivity sensor, so that the scanner 236 generates a clear image of the top of the parcel 214 at the region to be captured and stored for subsequent processing by a character recognition system 240 as the parcel 244 passes beneath the scanner 236.
  • the scanner 236 is aligned with the spot 230 so that the scanner may be operated to capture an image of the destination address 220 on the parcel 214.
  • a belt encoder 238 measures the displacement of the conveyor 212.
  • a communication link 240 functionally connects the belt encoder 238 to the scanner 236 and to a character recognition system 240 that includes a processing module 241 and a computer memory 242.
  • a second communication link 244 functionally connects the character recognition system 240 to the scanner 236, and a third communication link 246 functionally connects the character recognition system 240 to the moving-light illumination source 224.
  • the signal from the belt encoder 238 is used to determine the speed of the conveyor 212, which is used to synchronize the operation of the moving-light illumination source 224, the scanner 236, and the character recognition system 240 so that an image of a region 250 defined with respect to the spot 230 is stored in the computer memory 242.
  • the height data from the sensor assembly 232 indicates the presence of a parcel 214 in association with a spot 230 so that an image of a region 250 is only stored in the computer memory 242 when a parcel 214 is present in association with a spot 230 defined by the moving-light illumination source 224.
  • an operator positions the parcel 214 on the conveyor 212 so that the spot 230 defined by the moving-light illumination source 224 is centered with respect to the destination address 220 on the parcel 214.
  • FIG. 2A illustrates the parcel 214 positioned so that the spot 230 defined by the first discrete illumination source 226a is centered with respect to the destination address 220.
  • the parcel 214 travels on the conveyor 212, and the spot 230 travels at the same speed as the parcel 214, so that the spot 230 remains stationary relative to the parcel 214.
  • the parcel 214 is later positioned so that the spot 230' defined by the third discrete illumination source 226c is centered with respect to the destination address 220.
  • the parcel 214 is positioned so that the spot 230" defined by the last discrete illumination source 226n is centered with respect to the destination address 220.
  • FIGS. 2D and 2E illustrate the scanning of the parcel 214 by the scanner 236, which includes a CCD array that repeatedly generates an image of a scan line 252 to generate a video signal.
  • the operation of the scanner 236 and the processing module 241 of the character recognition system 240 are synchronized with the movement of the spot 230 so as to store in the computer memory 242 an image of the region 250, which is defined with respect to the spot 230.
  • the processing module 241 causes the computer memory 242 of the character recognition system 240 to begin storing the video data generated by the scanner 236.
  • the video data generated by the scanner 236 continues to be stored until the region 250 passes the scan line 252, which happens shortly before the parcel 214 is in the position shown in FIG. 2E.
  • FIGS. 2D-E illustrate the spot 230 when the parcel 214 is positioned as shown in FIGS. 2D-E, the spot 230 is not visible to an operator because the parcel 214 is not under the moving-light illumination source 224. Nevertheless, the character recognition system 240 uses the signal from the belt encoder 238 to keep track of the spot 230 after the parcel 214 travels past the moving-light illumination source 224. Thus, an image of the region 250, which is defined with respect to the spot 230, is stored in the computer memory 242 of the character recognition system 240.
  • FIG. 3 illustrates a parcel 214 with the spot 230 defined by the moving-light illumination source 224.
  • the spot 230 is typically a round or oval area that is somewhat smaller that than the region 250 to be imaged by the scanner 236.
  • the area associated with the spot 230 may be approximately one 1 inch (2.5 cm) across, whereas the region 250 may be approximately 4 inches (10 cm) by 4 inches (10 cm).
  • the parcel 214 is preferably positioned so that the center of the spot 230 is approximately in the center of the city and state address lines of the destination address 220. This allows the scanner 236 to capture an image of the destination address 220 by imaging the region 250.
  • the spot 230 may have virtually any size or configuration, and that multiple spots may be used to identify indicia on the parcel, such as four spots defining the corners of a rectangular region to be imaged.
  • the spot 230 may be defined by an illuminated area, or by an illuminated border, or by two illuminated spaced- apart parallel lines, etc.
  • the spot 230 could be configured to correspond to the width of the region 250 to be stored in the computer memory 224. This would assist an operator in orienting a parcel 214 so that the destination address 220 can be effectively scanned by the indicia reader system 200 as configured.
  • the operator may determine that the parcel 214 cannot be oriented so that the destination address 220 can be effectively scanned by the indicia reader system 200 as configured. This may happen if the destination address 220 is larger than the region 250 to be stored in the computer memory 242. In this case, the operator can divert the parcel 214 for hand sorting or imaging using a differently configured indicia reader system.
  • the character recognition system 240 is operative to selectively trigger the storage of an image in the computer memory 242. Triggering the storage of the image of the region 250 in the computer memory 242 may be accomplished in several different ways.
  • the scanner 236 may be toggled on and off by the processing module 241 in response to the signal from the belt encoder 238.
  • the scanner 236 may run continuously, and the processing module 241 may respond to the signal from the belt encoder 238 by latching a control line to an input buffer of the character recognition system 240.
  • the signal from the belt encoder 238 may be used as an input to a software- based algorithm running on the processing module 241, which triggers the storage of video data from the scanner 236 in the computer memory 242.
  • Many other means known to those skilled in the art may equivalently be employed to operate the character recognition system 240 and the scanner 236 so as to store an image of the region 250 in the computer memory 242.
  • the belt encoder 238 is a standard belt-driven, opto-mechanical encoder that provides a signal indicating the linear displacement of the conveyor 212.
  • the CCD array of the scanner 236 is cycled in response to the signal from the belt encoder 238 to generate a series of analog images of the scan line 252 that are transmitted to an analog-to-digital converter within the scanner 236.
  • the analog-to-digital converter of the scanner 236 uses a standard thresholding or similar process to convert the analog signal produced by the CCD array of the scanner 236 into an eight-bit digital video signal that is transmitted via the communication link 246 to the character recognition system 240, which is operable for storing the video data in the computer memory 242 for subsequent processing.
  • the scanner 236 is preferably a monochrome, 4,096 pixel line-scan type CCD array such as one using a Thompson TH7833A CCD chip.
  • the resolution of the image created by the scanner 232 is approximately 256 pixels or "dots" per inch (DPI) (101 dots per cm) across the field of view of the scanner 236.
  • the belt encoder 238 preferably triggers the CCD array of the scanner 236 at a rate of approximately 256 cycles per inch (2.54 cm) so that the resolution of the image created by the scanner 232 is approximately 256 pixels or "dots" per inch (DPI) (101 dots per cm) in the direction of conveyor travel.
  • a digital image with a correct aspect ratio may be generated by the scanner 236 and stored in the computer memory 242 of the character recognition system 240 by synchronizing the cycling rate of the scanner 236 with the linear speed of the conveyor 212.
  • the conveyor 212 may be approximately 24 inches (61 cm) wide and travel at linear speeds up to 20 inches per second or 100 feet per minute (51 cm per second or 30 meters per minute) or more.
  • the moving-light illumination source 224 is preferably positioned approximately 18 inches (46 cm) above conveyor 212 and defines a spot 230 that is approximately 1 inch (2.5 cm) wide and 1 inch (2.5 cm) long at the conveyor 212.
  • the moving-light illumination source 224 may be operated so that successive moving spots 230 are spaced virtually any distance apart.
  • acceptable performance is experienced when the moving-light indicia reader system 200 is operated with the conveyor 212 traveling at 50 feet per minute (25 cm per second or 15 meters per minute), and with the moving spots 230 spaced 22 inches (56 cm) apart, which allows the moving-light indicia reader system 200 to handle approximately 1,636 parcels per hour if the operator places a parcel under each moving spot.
  • the scanner 236 is preferably mounted to have an optical path of approximately 120 inches (305 cm) to the conveyor 212, with a 16 inch (41 cm) field of view at the conveyor 212. To save space, the scanner 236 is positioned approximately 30 inches (76 cm) above the center of conveyer 212 and is pointed towards a complex of mirrors (not shown) that increases the optical path from the scanner 236 to the conveyor 212 to approximately 120 inches (305 cm). These parameters may be varied somewhat without unduly affecting the performance of the disclosed embodiment of the present invention.
  • the scan line 234 may be longer than the width of the region stored in the computer memory 242.
  • the scanner 232 may be positioned to have a field of view (i.e., the scan line 234) equal to approximately 16 inches (41 cm) at the conveyor 212.
  • the region stored in the computer memory 242, however, may only be approximately 4 inches (10 cm) wide. This may be accomphshed by only storing the output of a portion of the cells of the scanner 232 (e.g., the center 1,024 pixels of a 4,096 pixel scanner) in the computer memory 242.
  • the moving-light illumination source 224 should be long enough to allow an operator to position the parcel 214 on the conveyor 212 while the spot 230 travels from the upstream end to the downstream end of the moving-light illumination source
  • a moving-light illumination source 224 having a length of 36 inches (91 cm) and 72 LEDs spaced 1/2 inch (1.3 cm) apart is appropriate for the conveyor 212 traveling at
  • the LEDs 226a-n of the moving-light illumination source 224 may be any of a variety of commercially available LEDs, such as a model AND190W0P manufactured by
  • the sensor assembly 232 may include any of a variety of commercially available height sensors, such as a model NR-40 manufactured by Innova Labs, Inc.
  • FIG. 4 is a diagram of a multi-conveyor indicia reader system 400 that includes a plurality of moving-light indicia reader systems 210a through 210n, which are virtually identical to those described above with respect to FIGS. 2A-E.
  • Each of the moving-light indicia reader systems 210a through 210n are synchronized by, and provide their video data to, a single character recognition system 240.
  • the character recognition system 240 synchronizes the moving spots 230a through 230n of the moving-light indicia reader systems 210a through 210n so as to time-division multiplex the storage of the regions 250a through 250n from the several scanners 236a through 236n.
  • the spots 230a through 230n are spaced relative to each other so that only one of the regions 250a through 250n captured by the scanners 236a through 236n needs to be stored in the computer memory 242 of the character recognition system 240 at any time.
  • the moving-light indicia reader system 400 allows the video data stored in the computer memory 242 of the character recognition system 240 to be reduced to a standard-sized region that is only large enough to capture the text of the destination addresses 220a-220n on the various parcels carried on the conveyors 212a- 212n.
  • the use of projected illumination allows the operator to view the spot 230a-230n defined by each moving-light illumination source 224a-224n directly on the top of the parcels 214a-214n.
  • the moving-light indicia reader system 400 allows the angle of the field of view of the scanners 236a-236n to be relatively narrow so that the scanners generates sharp images of the top of the parcels 214a-214n.

Landscapes

  • Character Input (AREA)
  • Sorting Of Articles (AREA)
  • Optical Communication System (AREA)
  • Aerials With Secondary Devices (AREA)
  • Holo Graphy (AREA)
  • Control Of Driving Devices And Active Controlling Of Vehicle (AREA)
  • Control Of Temperature (AREA)
  • Sowing (AREA)
  • Traffic Control Systems (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
  • Mechanical Optical Scanning Systems (AREA)
EP98902706A 1997-01-23 1998-01-23 Optisch-geleitetes zeichenabtastsystem Expired - Lifetime EP0954387B1 (de)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP03020263A EP1371424B1 (de) 1997-01-23 1998-01-23 Optisch geleitetes Zeichenabtastsystem

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US08/787,974 US5920056A (en) 1997-01-23 1997-01-23 Optically-guided indicia reader system for assisting in positioning a parcel on a conveyor
US787974 1997-01-23
US08/788,852 US5923017A (en) 1997-01-23 1997-01-23 Moving-light indicia reader system
US788852 1997-01-23
PCT/US1998/001366 WO1998032545A1 (en) 1997-01-23 1998-01-23 Optically-guided indicia reader system

Related Child Applications (1)

Application Number Title Priority Date Filing Date
EP03020263A Division EP1371424B1 (de) 1997-01-23 1998-01-23 Optisch geleitetes Zeichenabtastsystem

Publications (2)

Publication Number Publication Date
EP0954387A1 true EP0954387A1 (de) 1999-11-10
EP0954387B1 EP0954387B1 (de) 2004-05-26

Family

ID=27120718

Family Applications (2)

Application Number Title Priority Date Filing Date
EP03020263A Expired - Lifetime EP1371424B1 (de) 1997-01-23 1998-01-23 Optisch geleitetes Zeichenabtastsystem
EP98902706A Expired - Lifetime EP0954387B1 (de) 1997-01-23 1998-01-23 Optisch-geleitetes zeichenabtastsystem

Family Applications Before (1)

Application Number Title Priority Date Filing Date
EP03020263A Expired - Lifetime EP1371424B1 (de) 1997-01-23 1998-01-23 Optisch geleitetes Zeichenabtastsystem

Country Status (8)

Country Link
EP (2) EP1371424B1 (de)
JP (1) JP3476836B2 (de)
AT (2) ATE267649T1 (de)
CA (1) CA2271063C (de)
DE (2) DE69835840T2 (de)
DK (1) DK0954387T3 (de)
ES (2) ES2221148T3 (de)
WO (1) WO1998032545A1 (de)

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4251312B2 (ja) * 2002-03-08 2009-04-08 日本電気株式会社 画像入力装置
DE60322048D1 (de) 2002-05-16 2008-08-21 United Parcel Service Inc Systemen und verfahren für sortierung und lieferung von paketen unter verwendung von rfid technik
US6878896B2 (en) 2002-07-24 2005-04-12 United Parcel Service Of America, Inc. Synchronous semi-automatic parallel sorting
JP2005064633A (ja) * 2003-08-08 2005-03-10 Hitachi Ltd 帳票読取装置
EP3153243B1 (de) * 2010-05-06 2021-07-07 Eurosort B.V. Förderer zum transportieren von gegenständen
WO2014028204A1 (en) * 2012-08-13 2014-02-20 Laitram, L.L.C. Parcel handling methods
CN104307758B (zh) * 2014-09-25 2016-09-21 浙江海洋学院 一种快递件分拣机
US10471478B2 (en) 2017-04-28 2019-11-12 United Parcel Service Of America, Inc. Conveyor belt assembly for identifying an asset sort location and methods of utilizing the same
CN108480222A (zh) * 2018-03-14 2018-09-04 郑州工程技术学院 一种物流分拣***和方法
DE102019128814B4 (de) * 2019-10-25 2021-05-20 Sick Ag Kamera zur Erfassung eines Objektstroms und Verfahren zur Bestimmung der Höhe von Objekten
JP7423979B2 (ja) 2019-10-28 2024-01-30 日本電気株式会社 運搬制御装置、運搬制御方法、プログラム
CN110795953B (zh) * 2019-11-19 2020-09-25 浙江华眼视觉科技有限公司 一种寄件扫描与物流信息及时更新***
CN111301985A (zh) * 2020-03-09 2020-06-19 湖南城市学院 一种基于光纤激光器的光纤传感***

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB1270801A (en) * 1968-03-06 1972-04-19 Emi Ltd Improvements relating to input apparatus for pattern recognition devices
FR2657982B1 (fr) * 1990-02-02 1992-11-27 Cga Hbs Procede de localisation d'adresse sur des articles a trier, etiquette d'adressage et dispositif de mise en óoeuvre du procede.
US5202557A (en) * 1992-04-06 1993-04-13 Electrocom Automation L.P. Method and apparatus for detecting overlapping products in a singulated product stream
IL107265A0 (en) * 1993-10-12 1994-01-25 Galai Lab Ltd Parcel sorting system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO9832545A1 *

Also Published As

Publication number Publication date
ATE267649T1 (de) 2004-06-15
JP2000509655A (ja) 2000-08-02
EP1371424A2 (de) 2003-12-17
EP1371424B1 (de) 2006-09-06
WO1998032545A1 (en) 1998-07-30
ES2221148T3 (es) 2004-12-16
DE69824149T2 (de) 2004-10-28
DE69835840T2 (de) 2007-04-26
EP0954387B1 (de) 2004-05-26
DK0954387T3 (da) 2004-09-27
JP3476836B2 (ja) 2003-12-10
ES2271447T3 (es) 2007-04-16
EP1371424A3 (de) 2004-04-28
DE69824149D1 (de) 2004-07-01
DE69835840D1 (de) 2006-10-19
ATE338589T1 (de) 2006-09-15
CA2271063C (en) 2003-05-20
CA2271063A1 (en) 1998-07-30

Similar Documents

Publication Publication Date Title
US5923017A (en) Moving-light indicia reader system
US5920056A (en) Optically-guided indicia reader system for assisting in positioning a parcel on a conveyor
EP1371424B1 (de) Optisch geleitetes Zeichenabtastsystem
CA2231450C (en) System and method for reading package information
AU635607B2 (en) Process for distributing packets or the like
EP2126781B1 (de) System zur bilderfassung
EP0529966B1 (de) Sortiervorrichtung für Poststücke
US7632367B2 (en) Parcel labeling, conveying, and sorting method and apparatus
KR102010494B1 (ko) 광전자 코드 판독기 및 광학 코드 판독 방법
US7809158B2 (en) Method and apparatus for detecting doubles in a singulated stream of flat articles
US6236735B1 (en) Two camera system for locating and storing indicia on conveyed items
US20040016684A1 (en) Synchronous semi-automatic parallel sorting
CN110153565B (zh) 一种激光打码方法及装置
CN1052322C (zh) 标记定位图象处理方法及包裹分拣方法
US11667474B1 (en) Increasing scan rate of parcels within material handling facility
JP3028510B2 (ja) 画像処理装置,小包から経路指定情報を抽出するための装置,画像処理方法及び小包仕分けの方法
JPH01219968A (ja) 宛名部判定装置
JPH06206051A (ja) 水平方向および垂直方向に配置されたアドレス情報の混在する郵便物を処理する方法および装置
JPH08155397A (ja) 郵便物区分装置およびバーコード印刷装置
JPS59216281A (ja) 光学式文字読取装置
JPH0550589A (ja) 紙葉類区分装置
JP2001516281A (ja) 郵便小荷物の表面の記号および文字を認識する方法
JPH096896A (ja) 光学式画像情報読み取り装置

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 19990618

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE CH DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE

17Q First examination report despatched

Effective date: 20030417

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE CH DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20040526

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20040526

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: CH

Ref legal event code: NV

Representative=s name: BRAUN & PARTNER PATENT-, MARKEN-, RECHTSANWAELTE

Ref country code: IE

Ref legal event code: FG4D

REF Corresponds to:

Ref document number: 69824149

Country of ref document: DE

Date of ref document: 20040701

Kind code of ref document: P

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20040826

REG Reference to a national code

Ref country code: SE

Ref legal event code: TRGR

REG Reference to a national code

Ref country code: DK

Ref legal event code: T3

ET Fr: translation filed
REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2221148

Country of ref document: ES

Kind code of ref document: T3

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20050123

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20050124

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20050131

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20050301

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20070103

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: SE

Payment date: 20070104

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DK

Payment date: 20070115

Year of fee payment: 10

Ref country code: CH

Payment date: 20070115

Year of fee payment: 10

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20041026

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: DK

Ref legal event code: EBP

EUG Se: european patent has lapsed
NLV4 Nl: lapsed or anulled due to non-payment of the annual fee

Effective date: 20080801

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080801

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080131

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080124

Ref country code: DK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080131

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20120116

Year of fee payment: 15

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20130207

Year of fee payment: 16

Ref country code: DE

Payment date: 20130116

Year of fee payment: 16

Ref country code: GB

Payment date: 20130123

Year of fee payment: 16

Ref country code: FR

Payment date: 20130204

Year of fee payment: 16

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: BE

Payment date: 20130114

Year of fee payment: 16

BERE Be: lapsed

Owner name: *UNITED PARCEL SERVICE OF AMERICA INC.

Effective date: 20140131

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 69824149

Country of ref document: DE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20140123

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140801

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20140930

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 69824149

Country of ref document: DE

Effective date: 20140801

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140123

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140131

REG Reference to a national code

Ref country code: ES

Ref legal event code: FD2A

Effective date: 20150330

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140124

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140123