EP2725781B1 - Procédé de réglage de la focalisation d'une caméra vidéo numérique et caméra vidéo numérique associée - Google Patents
Procédé de réglage de la focalisation d'une caméra vidéo numérique et caméra vidéo numérique associée Download PDFInfo
- Publication number
- EP2725781B1 EP2725781B1 EP12189977.7A EP12189977A EP2725781B1 EP 2725781 B1 EP2725781 B1 EP 2725781B1 EP 12189977 A EP12189977 A EP 12189977A EP 2725781 B1 EP2725781 B1 EP 2725781B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- pixels
- digital video
- video camera
- subset
- focus
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims description 22
- 238000012545 processing Methods 0.000 claims description 28
- 230000008569 process Effects 0.000 claims description 10
- 239000006059 cover glass Substances 0.000 claims description 4
- 230000003287 optical effect Effects 0.000 description 50
- 238000001514 detection method Methods 0.000 description 15
- 230000009471 action Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000004091 panning Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
- H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
- H01L27/144—Devices controlled by radiation
- H01L27/146—Imager structures
- H01L27/14601—Structural or functional details thereof
- H01L27/14609—Pixel-elements with integrated switching, control, storage or amplification elements
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
- H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
- H01L27/144—Devices controlled by radiation
- H01L27/146—Imager structures
- H01L27/14601—Structural or functional details thereof
- H01L27/14625—Optical elements or arrangements associated with the device
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/67—Focus control based on electronic image sensor signals
- H04N23/673—Focus control based on electronic image sensor signals based on contrast or high frequency components of image signals, e.g. hill climbing method
Definitions
- the present invention relates to a method of setting focus of a digital video camera and a digital video camera doing the same.
- Contrast detection auto-focus is widely used in digital video cameras of today.
- Contrast detection auto-focus is achieved by measuring contrast within a sensor field, through the lens.
- the intensity difference between adjacent pixels of the sensor naturally increases with correct image focus.
- the optical system can thereby be adjusted until the maximum contrast is detected.
- a difficulty for contrast detection auto-focus algorithms is that if the image suddenly becomes slightly blurry, for instance when zooming or panning, or if the lighting condition is poor, it is hard to determine if the lens shall be moved closer or further away. If moving in the wrong direction, the image will become even more blurry, and potentially important video information is irrecoverably lost. In the worst case, the digital video camera loses track of the correct focus position altogether, resulting in the necessity of a full focus sweep to find the correct focus position again. Such a sweep might take several seconds to complete, during which the video will be more or less useless.
- Patent publication No. WO 2006/039486 A2 discloses an image sensing apparatus using a method called "Light Field Photography".
- This image sensing apparatus includes an imaging lens, a microlens array, an image sensing element, and an image processing unit.
- Sensed image data obtained from the image sensing element includes not only the light intensity distribution on the light-receiving surface but also the shift revealing information about the light travelling direction, this information is used to calculate the distance to the imaged objects and assist in focusing of the camera.
- Patent publication No. US 2010/0215354 A1 discloses focus micro-lenses which cover an area of several underlying pixels, this arrangement of micro-sensors covers the entire sensing area; a shift amount is calculated for a plurality of focus positions and is used in combination with the defocus amount from contrast to calculate the amount of focus motor drive.
- an objective of the present invention is to provide a digital video camera and a method for supporting and guiding the normal contrast detection auto-focus of the digital video camera to find the correct focus setting.
- a digital video camera for finding a correct focus setting when capturing a video image stream of a scene.
- Said digital video camera comprises: a photo sensor having a plurality of pixels, each pixel being arranged to obtain and output pixel data; a focusing device arranged to set a focus of the camera; a processing unit arranged to process pixel data from a first subset of the pixels such that image frames of said video image stream are rendered based on pixel data in said first subset of pixels; an array of optical elements covering a second subset of the pixels, each of said optical elements covering a number of pixels, wherein the first subset of pixels is confined by a single endless border and the second subset of pixels is arranged outside and adjacent to said single endless border of said first subset, wherein said processing unit further is arranged to process pixel data from said second subset of said plurality of pixels such that information pertaining to distance to one or more objects in said scene is extracted, wherein said focusing device is arranged to
- objects in said scene shall be interpreted in a wide sense, thus objects in said scene may be a person, a car, a house etc., but it may also be an area of the ground, a part of a wall, a part of a territory etc.
- border can be considered as an imaginary line surrounding said first subset of pixels of said photo sensor, i.e. the expression "single endless border” does not mean that there needs to be an actual physical element surrounding the pixels, this is merely a way of describing that the first subset of pixels are all placed within a certain area. Obviously a physical border could be present, but this is not required.
- a benefit compared to a normal digital video camera with contrast detection auto-focus is that with the provided digital video camera it will be possible to know with a much higher probability in which direction a focus lens of the focusing device shall be moved to keep the scene in focus. This will lead to a more robust and potentially also faster auto-focus, at a low extra cost.
- Said focusing device may further be arranged to set the focus of the digital video camera based on a contrast detection autofocus algorithm operating on pixel data from said first subset of pixels and to be guided towards a new focus setting of said digital video camera based on said information pertaining to distance to one or more objects.
- Said processing unit may further be arranged to process said information pertaining to distance to one or more objects in said scene such that information pertaining to if said one or more objects is located further away or closer to said digital video camera as compared to a current focus setting of said digital video camera is extracted.
- Said focusing device may further be arranged to be directed towards a new focus setting by said information pertaining to if said one or more objects is located further away or closer to the camera as compared to a current focus setting of said camera.
- Said optical elements may be integrally formed with a cover glass of said photo sensor.
- Said optical elements may be integrally formed with a cover glass of said photo sensor.
- said optical elements may be integrally formed with an IR filter of said photo sensor. This too will lead to a compact photo sensor assembly.
- a method of setting focus of a digital video camera for capturing a video image stream of a scene comprising the steps of: providing a photo sensor having a plurality of pixels, each pixel being arranged to obtain and output pixel data; defining a first subset of pixels as pixels present within a single endless border; defining a second subset of pixels as pixels outside and adjacent to the single endless border; arranging an array of optical elements covering the second subset of pixels, each of said optical elements covering a number of pixels; processing pixel data from the first subset of pixels such that image frames of said video image stream are rendered based on pixel data in the first subset of pixels; processing pixel data from the second subset of pixels such that information pertaining to distance to one or more objects in said scene is extracted, and setting focus of said digital video camera based on said information pertaining to distance to one or more objects.
- the step of setting the focus of said digital video camera may comprise: applying a contrast autofocus algorithm on the pixel data from the first subset of pixels; processing said information pertaining to distance to one or more objects in said scene such that information pertaining to if said one or more objects is located further away or closer to said digital video camera as compared to a current focus setting of said digital video camera is extracted; and directing said digital video camera towards a new focus setting by applying said information pertaining to if said one or more objects is located further away or closer to said digital video camera as compared to a current focus setting of said digital video camera, wherein said digital video camera is out of focus in the current focus setting.
- the basic idea of this invention is to use the same concept as in a light-field, or plenoptic, camera in order for supporting and helping a digital video camera implementing a contrast detection auto-focus algorithm to find the correct focus point.
- a light-field, or plenoptic, camera records not only the total intensity of light in each "pixel" of the light-field, or plenoptic, camera, but rather the intensity of light coming from each direction, in each "pixel" of the light-field, or plenoptic, camera.
- a light-field "image” is not viewable in a meaningful fashion as-is, but by applying processing it is possible to produce photographic images with various properties. The most well-known such property is the possibility of changing the focus position (the distance to the object(s) being rendered sharply) after the shot has been taken. Following from this fact it is easily understood that information about the distance to objects in the scene is embedded in the light-field image.
- a layer of optical elements e.g. micro lenses
- the layer of optical elements is typically arranged a fraction of a millimeter above the photo sensor.
- Each optical element works as one "pixel" of the light field camera, and the main lens is focused such that its image is in focus (more or less) at the layer of optical elements.
- the size of the optical elements is around 10x10 photo sensor pixels each, hence each one of the optical elements produce a 10-pixels-diameter circular image on the photo sensor.
- a layer having a frame shaped array of optical elements e.g. micro lenses
- optical elements e.g. micro lenses
- the rest of the sensor can be used as an ordinary camera sensor, while it is possible to use the area adjacent to the border to collect information on changes in the depth of the scene. That is, a number of rows/columns of pixels adjacent to the border of the sensor are sacrificed to produce a distance sensor.
- the digital video camera 1 includes a zooming function, i.e. the focus of the digital video camera 1 may vary. Moreover, the digital video camera 1 may be connectable to a digital network. Further, the digital video camera 1 may be a portable camera, a stationary camera or a camera having pan/tilt functionality. The digital video camera 1 may be a camera for monitoring applications, such as surveillance purposes, machine vision, store compliance, business information, etc. In order to facilitate the understanding of the invention, standard features of a digital video camera not being relevant for the present invention are not described.
- the digital video camera 1 comprises a housing 2, a focusing device 3 comprising a focusing lens 4, a focus motor 5 and a controller 6, a photo sensor 7, a processing unit 8, a memory 9 and an I/O-port 10.
- the I/O-port 10 is according to the shown embodiment used to connect the digital video camera 1 to a communication network 11.
- the digital video camera 1 may however also be connected directly to a monitor to show images or a recorder for saving images.
- the communication network 11 may be any kind of communication network for communicating digital image information, such as a wire line or wireless data communication network, e.g. a local area network (LAN) or a wireless local area network (W-LAN) or a Wide Area Network (WAN).
- LAN local area network
- WLAN wireless local area network
- WAN Wide Area Network
- the photo sensor 7 is arranged to capture images of a video image stream of a scene.
- the photo sensor 7 may for example be a Charge Coupled Device (CCD) sensor, a Complementary Metal Oxide Semiconductor (CMOS) sensor, or similar, for registering incident light.
- the photo sensor 7 comprises a plurality of sensor pixels.
- the photo sensor 7 is arranged to output pixel data for each sensor pixel.
- the pixel data of each sensor pixel is a function of the number of photons received by said sensor pixel.
- the photo sensor 7 is partly covered by an array 20 of optical elements 22. More precisely, the array 20 of optical elements 22 is arranged in front of and adjacent to said photo sensor 7. It shall be understood that the sizes of optical elements and sensor pixels are exaggerated for illustrative purposes and, thus, are provided to illustrate the general structures of embodiments of the present invention.
- the pixels of the photo sensor 7 may be divided into two subsets, a first subset 24 of the pixels of the photo sensor 7 not being covered by the array 20 of optical elements 22 and a second subset 26 of the pixels of the photo sensor 7 being covered by the array 20 of optical elements 22.
- Each of said optical elements 22 is covering a number of pixels of the photo sensor 7.
- the optical elements 22 comprise micro lenses of a type used in a light-field camera.
- the array 20 of optical elements 22 is arranged to refract incident light rays over said second subset 26 of pixels.
- the first subset 24 of pixels is confined by a single endless border.
- the second subset 26 of pixels is arranged outside and adjacent to said single endless border of said first subset 24. More precisely, said second subset 26 of pixels is surrounding said single endless border of said first subset such that a frame of pixels of said second subset is formed.
- said frame of pixels of said second subset 26 of pixels is 10-50 pixels wide. It is however realized that the width of said frame of pixels of said second subset 26 may be as small as 1-2 pixels and as wide as a few hundreds of pixels.
- the upper limit of the number of pixels used depends on the size of photo sensor area it is possible to sacrifice for the distance sensor.
- the array 20 of optical elements 22 constitutes a frame of optical elements 22 covering an outer portion, close to the border, of the photo sensor 7 leaving an area in the middle of the photo sensor 7 without optical elements. According to the embodiment shown in Fig. 2 a single row of optical elements 22 is arranged around the border of the photo sensor 7. It shall however be understood that two or more rows of optical elements 22 may be used.
- each optical element 22 covers approximately 10x10 pixels of the photo sensor 7.
- the sizes of the pixels and optical elements as illustrated in Fig. 2 are exaggerated for illustrative purposes and, thus, are provided to illustrate the general structures of embodiments of the present invention and not to represent an actual relationship between the size of the optical elements 22 and the pixels of the photo sensor 7.
- a width of said optical elements 22 correlates with a width of the pixels of said photo detector 7.
- the width of said optical elements 22 are N times the width of the pixels of said photo detector 7, wherein N is an integer value, and the optical elements would typically cover a number of whole pixels, such that one pixel typically would only be covered by one optical element.
- the rest of the photo sensor 7 can be used as an ordinary camera photo sensor, while it is possible to use the border area to collect information on changes in the depth of the scene. That is, a number of rows/columns of pixels around the border of the photo sensor 7 are sacrificed to produce a distance sensor.
- the optical elements 22 may be integrally formed with an IR filter of said photo sensor 7. Alternatively, the optical elements 22 may be integrally formed with a cover glass of said photo sensor 7. However, the person skilled in the art realizes that the optical elements 22 may also be a standalone layer positioned in front of the photo sensor.
- the processing unit 8 is arranged to receive pixel data from the photo sensor 7.
- the processing unit 8 is moreover arranged to process the pixel data from said first subset 24 of the pixels such that image frames of said video image stream are rendered based on pixel data in said first subset 24 of pixels.
- the processing unit 8 is further arranged to process pixel data from said second subset 26 of the pixels such that information pertaining to distance to one or more objects in said scene is extracted. This is done by determining the directions of incoming light rays and therefrom to determine said information pertaining to distance to one or more objects in said scene.
- objects in said scene shall be interpreted in a wide sense, thus objects in said scene may be a person, a car, a house etc., but it may also be an area of the ground, a part of a wall, a part of a territory etc.
- the pixel data from at least some of the pixels of the first subset 24 of pixels constitute pixel data building up image frames of said video image stream captured by the digital video camera 1
- the pixel data from at least some of the pixels of the second subset 26 of pixels is used for determining a distance to one or more objects in said scene captured by the digital video camera 1.
- the processing unit 8 is further arranged to process said information pertaining to distance to one or more objects in said scene such that information pertaining to if said one or more objects is located further away or closer to the camera 1 as compared to a current focus setting of said focusing device 3 is extracted.
- the focusing device 3 is arranged to set a focus of the camera 1. This is done by modifying the focus of the focusing device 3.
- the focus of the focusing device 3 is modified by translating the focus lens 4 along an optical axis of the camera 1.
- the focusing lens 4 is moved by means of the focus motor 5.
- the focus motor 5 is controlled by means of the controller 6.
- the focus motor 5 is connected to the focusing lens 4 and is arranged to move the focusing lens 4.
- the controller 6 is connected to the focusing motor 5 and is arranged to control the focusing motor 5.
- the controller 6 is getting input data from the processing unit 8.
- the focusing device 3 forms part of an auto focus unit of said digital video camera 1.
- the focusing device 3 is arranged to set the focus of the digital video camera 1 based on a contrast detection autofocus algorithm operating on pixel data from said first subset of pixels.
- Contrast detection autofocus is achieved by measuring contrast within a sensor field, through the lens. The intensity difference between adjacent pixels of the sensor naturally increases with correct image focus. The optical system can thereby be adjusted until the maximum contrast is detected. Contrast detection autofocus does not involve actual distance measurement at all. Thus, as the contrast detection autofocus does not calculate whether the subject is in front focus or back focus, focus tracking is not feasible.
- the focusing device 3 is in addition arranged to be guided towards a new focus setting of the digital video camera 1 based on said information pertaining to distance to one or more objects. More precisely, said information pertaining to if said one or more objects is located further away or closer to the camera 1 as compared to a current focus setting of said focusing device 3 is used to direct the focusing device 3 towards a new focus setting.
- said focusing device 3 is arranged to set the focus of the camera 1 based on various factors, including contrast detection autofocus and said information pertaining to distance to one or more objects in the scene.
- the digital video camera 1 is focused at an initial focus distance in a wide angle zoom position.
- a large scene is visible to the camera 1, and the autofocus has been set to give a clear image of a large part of the scene.
- a zooming action takes place.
- the scene parts that are visible to the camera 1 may now be out of focus, as the first focus setting was chosen to give a sharp image of the entire scene, i.e. the focus was set to an average distance for the scene.
- the autofocus is unaware of if the new smaller part of the scene now visible is situated closer to the camera or further away than the average distance in the previous wide-angle scene.
- the output from the distance sensor comprising the optical elements 22 located at the border of the photo sensor 7, may now be used to guide the autofocus to a new correct position.
- This may be made by setting the autofocus to a distance to an object in the scene which is extracted by the distance sensor from said information pertaining to distance to one or more objects in said scene.
- this may be made by instructing or guiding the autofocus algorithm to start searching for focus in a certain direction (further away or closer to the camera as compared with a current focus setting).
- the decision if the new focus setting is further away or closer to the camera as compared with a current focus setting is based on a difference in the distance to an object in the scene.
- the difference in the distance to an object in the scene being extracted by means of the distance sensor by comparing a distance to an object in the scene before the zoom action took place and a distance to the object in the scene after the zoom action took place.
- the autofocus should start its search for focus in the direction further away. This could happen e.g. when going from a wide angle zoom setting to a telezoom setting in order to zoom in on an object further away than the average distance to the wide-angle scene.
- the distance to the object in the scene would be smaller compared to when the image was in focus.
- the direction guidance saves the autofocus from having to search the entire available focus interval, which in turn limits the time when the digital video camera 1 is out of focus, thus, the time when the output video image stream from the digital video camera 1 is blurry and being more or less useless is limited.
- optical elements 22 are placed in the outer region of the photo sensor 7, framing or surrounding the main part (first subset 24) of the pixels of the photo sensor 7, information pertaining to distance to one or more objects in the outermost region of the visible part of the scene is extracted.
- object may as previously mentioned e.g. simply be a part of the ground or other objects of the scene that are often consider as part of the background.
- the distance to the elements in the outer parts of the scene will in virtually all instances change in the same way as the distance to the objects in the more central parts of the image when the digital video camera 1 changes a zoom setting.
- Figure 3 illustrates an embodiment of a method of setting focus of a digital video camera for capturing a video image stream of a scene.
- the method comprises: providing 100 a photo sensor having a plurality of pixels, each pixel being arranged to obtain and output pixel data; defining 102 a first subset of pixels as pixels present within a single endless border; defining 103 a second subset of pixels as pixels outside and adjacent to the single endless border; arranging 104 an array of optical elements covering the second subset of pixels, each of said optical elements covering a number of pixels; processing 106 pixel data from the first subset of pixels such that image frames of said video image stream are rendered based on pixel data in the first subset of pixels; processing 108 pixel data from the second subset of pixels such that information pertaining to distance to one or more objects in said scene is extracted; and setting 110 focus of the camera based on said information pertaining to distance to one or more objects.
- the step of setting the focus of the digital video camera may comprise: applying a contract detection autofocus algorithm on the pixel data from the first subset of pixels; processing said information pertaining to distance to one or more objects in said scene such that information pertaining to if said one or more objects is located further away or closer to the digital video camera as compared to a current focus setting of the digital video camera is extracted; and directing said digital video camera towards a new focus setting by applying said information pertaining to if said one or more objects is located further away or closer to the camera as compared to a current focus setting of the camera, wherein said digital video camera is out of focus in the current focus setting.
- the focus of the focusing device 3 may be modified by using a focus lens made of liquid crystals or oil/water surfaces being electrically affected in order to change the focusing properties of the focus lens instead of using a focus motor moving the focus lens.
- the processing unit may be implemented as a single processor or alternatively as a plurality of processors. If the processing unit comprises a plurality of processors each processor may have a dedicated function, e.g. one processor may be specialized in processing pixel data from the first subset of pixels such that image frames of said video image stream are rendered based on pixel data in the first subset of pixels; and another processor may be specialized in processing pixel data from the second subset of pixels such that information pertaining to distance to one or more objects in said scene is extracted.
- contrast detection auto-focus algorithm is mentioned as an auto-focus algorithm. It should however be realized that also other auto-focus algorithms may be used in connection with the present invention.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Power Engineering (AREA)
- Electromagnetism (AREA)
- Condensed Matter Physics & Semiconductors (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Automatic Focus Adjustment (AREA)
- Studio Devices (AREA)
Claims (12)
- Caméra vidéo numérique (1) pour trouver un réglage de mise au point correct lors de l'acquisition d'un flux d'images vidéo d'une scène, comprenant :un photocapteur (7) comportant une pluralité de pixels, chaque pixel étant agencé pour obtenir et délivrer des données de pixels,un dispositif de mise au point (3) agencé pour régler une mise au point de la caméra vidéo numérique (1),une unité de traitement (8) agencée pour traiter des données de pixels à partir d'un premier sous-ensemble (24) des pixels de sorte que des trames d'images dudit flux d'images vidéo soient rendues sur la base de données de pixels dans ledit premier sous-ensemble (24) de pixels,caractérisée parun réseau (20) de microlentilles (22), du type utilisé dans une caméra de champ de lumière, agencé au-dessus d'un deuxième sous-ensemble (26) des pixels et le recouvrant, chacune desdites microlentilles (22) recouvrant un nombre de pixels,dans laquelle le premier sous-ensemble (24) de pixels est confiné à l'intérieur d'une zone, et délimité par le deuxième sous-ensemble (26) de pixels agencé à l'extérieur et adjacent au dit premier sous-ensemble (24) de pixels de sorte qu'une trame de pixels dudit deuxième sous-ensemble (26) soit formée autour dudit premier sous-ensemble (24) de pixels,dans lequel ladite unité de traitement (8) est en outre agencée pour traiter des données de pixels à partir dudit deuxième sous-ensemble (26) de ladite pluralité de pixels de sorte que des informations concernant une distance à un ou plusieurs objets dans ladite scène soient extraites,dans laquelle ledit dispositif de mise au point (3) est agencé pour régler la mise au point de la caméra vidéo numérique (1) sur la base desdites informations concernant une distance à un ou plusieurs objets.
- Caméra vidéo numérique (1) selon la revendication 1, dans lequel ledit dispositif de mise au point est en outre agencé pour régler la mise au point de la caméra vidéo numérique (1) sur la base d'un algorithme de mise au point automatique utilisant des données de pixels dudit premier sous-ensemble (24) de pixels et pour être guidé vers un nouveau réglage de mise au point de la caméra vidéo numérique (1) sur la base desdites informations concernant une distance à un ou plusieurs objets.
- Caméra vidéo numérique (1) selon la revendication 1 ou 2, dans laquelle ladite unité de traitement (8) est en outre agencée pour traiter lesdites informations concernant une distance à un ou plusieurs objets dans ladite scène de sorte que les informations concernant si lesdits un ou plusieurs objets sont situés plus loin ou plus près de ladite caméra vidéo numérique (1) par comparaison à un réglage de mise au point actuel de ladite caméra vidéo numérique (1) soient extraites.
- Caméra vidéo numérique (1) selon la revendication 3, dans laquelle ledit dispositif de mise au point (3) est agencé pour être dirigé vers un nouveau réglage de mise au point par lesdites informations concernant si lesdits un ou plusieurs objets sont situés plus loin ou plus près de la caméra vidéo numérique (1) par comparaison à un réglage de mise au point actuel de ladite caméra vidéo numérique (1).
- Caméra vidéo numérique (1) selon l'une quelconque des revendications 1 à 4, dans laquelle ledit réseau (20) de microlentilles (22) est agencé pour réfracter des rayons lumineux incidents sur ledit deuxième sous-ensemble (26) de pixels et dans lequel ladite unité de traitement (8) est agencée pour traiter des données de pixels à partir dudit deuxième sous-ensemble (26) de pixels de sorte que les directions de rayons lumineux entrants soient déterminées et pour déterminer à partir de cela lesdites informations concernant une distance à un ou plusieurs objets dans ladite scène.
- Caméra vidéo numérique (1) selon l'une quelconque des revendications 1 à 5, dans laquelle ledit réseau (20) de microlentilles (22) est agencé devant ledit photocapteur (7) et adjacent à celui-ci.
- Caméra vidéo numérique (1) selon l'une quelconque des revendications 1 à 6, dans laquelle lesdites microlentilles (22) font partie intégrante d'un verre de recouvrement dudit photocapteur (7).
- Caméra vidéo numérique (1) selon l'une quelconque des revendications 1 à 7, dans laquelle lesdites microlentilles font partie intégrante d'un filtre infrarouge dudit photocapteur (7).
- Caméra vidéo numérique (1) selon l'une quelconque des revendications 1 à 8, dans laquelle ledit dispositif de mise au point (3) fait partie d'une unité de mise au point automatique de ladite caméra vidéo numérique (1).
- Caméra vidéo numérique (1) selon l'une quelconque des revendications 1 à 9, dans laquelle ledit dispositif de mise au point (3) comprend un moteur de mise au point (5) agencé pour déplacer une lentille de mise au point (4).
- Procédé de réglage de mise au point d'une caméra vidéo numérique (1) pour acquérir un flux d'images vidéo d'une scène, comprenant les étapes de :la fourniture (100) d'un photocapteur (7) comportant une pluralité de pixels, chaque pixel étant agencé pour obtenir et délivrer des données de pixels ;la définition (102) d'un premier sous-ensemble de pixels en tant que pixels confinés à l'intérieur d'une zone ;la définition (103) d'un deuxième sous-ensemble de pixels en tant que pixels agencés à l'extérieur dudit premier sous-ensemble (24) de pixels et adjacents à celui-ci de sorte qu'une trame de pixels dudit deuxième sous-ensemble (26) soit formée autour de la frontière dudit premier sous-ensemble (24) de pixels ;l'agencement (104) d'un réseau de microlentilles (22), d'un type utilisé dans une caméra de champ de lumière, au-dessus du deuxième sous-ensemble de pixels et le recouvrant, chacune desdites microlentilles recouvrant un nombre de pixels ;le traitement (106) de données de pixels à partir du premier sous-ensemble de pixels de sorte que des trames d'images dudit flux d'images vidéo soient rendues sur la base de données de pixels dans le premier sous-ensemble de pixels ;le traitement (108) de données de pixels à partir du deuxième sous-ensemble de pixels de sorte que des informations concernant une distance à un ou plusieurs objets dans ladite scène soient extraites ; et le réglage (110) de la mise au point de la caméra vidéo numérique (1) sur la base desdites informations concernant une distance à un ou plusieurs objets.
- Procédé selon la revendication 11, dans lequel l'étape du réglage de la mise au point de la caméra vidéo numérique comprend :l'application d'un algorithme de mise au point automatique sur les données de pixels à partir du premier sous-ensemble de pixels ;le traitement desdites informations concernant une distance à un ou plusieurs objets dans ladite scène de sorte que des informations concernant si lesdits un ou plusieurs objets se trouvent plus loin ou plus près de la caméra vidéo numérique par comparaison à un réglage de mise au point actuel de la caméra vidéo numérique soient extraites ; etla direction de ladite caméra vidéo numérique vers un nouveau réglage de mise au point en appliquant lesdites informations concernant si lesdits un ou plusieurs objets se trouvent plus loin ou plus près de la caméra vidéo numérique par comparaison à un réglage de mise au point actuel de la caméra vidéo numérique, dans lequel la caméra vidéo numérique est floue au réglage de mise au point actuel.
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP12189977.7A EP2725781B1 (fr) | 2012-10-25 | 2012-10-25 | Procédé de réglage de la focalisation d'une caméra vidéo numérique et caméra vidéo numérique associée |
CN201310463939.7A CN103780831B (zh) | 2012-10-25 | 2013-10-08 | 设置数码摄像机的对焦的方法和执行该方法的数码摄像机 |
US14/054,220 US8860868B2 (en) | 2012-10-25 | 2013-10-15 | Method of setting focus of a digital video camera and a digital video camera doing the same |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP12189977.7A EP2725781B1 (fr) | 2012-10-25 | 2012-10-25 | Procédé de réglage de la focalisation d'une caméra vidéo numérique et caméra vidéo numérique associée |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2725781A1 EP2725781A1 (fr) | 2014-04-30 |
EP2725781B1 true EP2725781B1 (fr) | 2014-06-04 |
Family
ID=47115509
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP12189977.7A Active EP2725781B1 (fr) | 2012-10-25 | 2012-10-25 | Procédé de réglage de la focalisation d'une caméra vidéo numérique et caméra vidéo numérique associée |
Country Status (3)
Country | Link |
---|---|
US (1) | US8860868B2 (fr) |
EP (1) | EP2725781B1 (fr) |
CN (1) | CN103780831B (fr) |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
FR2999729B1 (fr) * | 2012-12-14 | 2015-01-16 | Astrium Sas | Mise au point optique d'un instrument de saisie d'image |
DE102014115292A1 (de) * | 2014-10-21 | 2016-04-21 | Connaught Electronics Ltd. | Verfahren zum Bereitstellen von Bilddateien von einem Kamerasystem, Kamerasystem und Kraftfahrzeug |
CN105423998A (zh) * | 2015-11-07 | 2016-03-23 | 陕西科技大学 | 一种透镜测距装置及其测量方法 |
US20190301935A1 (en) * | 2015-12-23 | 2019-10-03 | Spectricity | Device for recording spectral information |
FR3059156B1 (fr) * | 2016-11-22 | 2019-06-28 | Sodern | Module de detection optique |
EP3879811B1 (fr) * | 2020-03-09 | 2021-12-15 | Axis AB | Détermination si une caméra est défocalisée |
DE102020109929B3 (de) * | 2020-04-09 | 2021-01-14 | Sick Ag | Erfassung von Bilddaten eines bewegten Objekts |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6819360B1 (en) * | 1999-04-01 | 2004-11-16 | Olympus Corporation | Image pickup element and apparatus for focusing |
US7936392B2 (en) * | 2004-10-01 | 2011-05-03 | The Board Of Trustees Of The Leland Stanford Junior University | Imaging arrangements and methods therefor |
EP2028843B1 (fr) | 2007-08-21 | 2013-10-23 | Ricoh Company, Ltd. | Dispositif de mise au point et appareil d'imagerie l'utilisant |
JP4905326B2 (ja) * | 2007-11-12 | 2012-03-28 | ソニー株式会社 | 撮像装置 |
JP2010057067A (ja) * | 2008-08-29 | 2010-03-11 | Sony Corp | 撮像装置および画像処理装置 |
JP5195506B2 (ja) * | 2009-02-24 | 2013-05-08 | 株式会社ニコン | 撮像装置および画像合成方法 |
JP4766133B2 (ja) * | 2009-03-11 | 2011-09-07 | 株式会社ニコン | 撮像装置 |
JP5906062B2 (ja) * | 2010-12-17 | 2016-04-20 | キヤノン株式会社 | 撮像装置およびその制御方法 |
-
2012
- 2012-10-25 EP EP12189977.7A patent/EP2725781B1/fr active Active
-
2013
- 2013-10-08 CN CN201310463939.7A patent/CN103780831B/zh active Active
- 2013-10-15 US US14/054,220 patent/US8860868B2/en active Active
Also Published As
Publication number | Publication date |
---|---|
CN103780831A (zh) | 2014-05-07 |
US20140118571A1 (en) | 2014-05-01 |
CN103780831B (zh) | 2017-05-17 |
EP2725781A1 (fr) | 2014-04-30 |
US8860868B2 (en) | 2014-10-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2725781B1 (fr) | Procédé de réglage de la focalisation d'une caméra vidéo numérique et caméra vidéo numérique associée | |
CN107079087B (zh) | 摄像装置及对象物识别方法 | |
US9036080B2 (en) | Apparatus and method for acquiring information about light-field data | |
US9191578B2 (en) | Enhanced image processing with lens motion | |
US20130050564A1 (en) | Producing focused videos from single captured video | |
US20130083230A1 (en) | Image sensor and image pickup apparatus | |
KR20100038754A (ko) | 촬상 장치 및 촬상 방법 | |
KR20150008831A (ko) | 촬영 화상의 보조 정보를 생성하는 촬상장치, 화상처리장치, 및 화상처리방법 | |
US8902349B2 (en) | Image pickup apparatus | |
JP5753371B2 (ja) | 撮像装置およびその制御方法 | |
US8195042B2 (en) | Imaging apparatus and control method | |
WO2010055835A1 (fr) | Appareil de détection de focalisation, procédé de détection de focalisation et appareil de détection d’image | |
JP7100735B2 (ja) | 撮像素子及び撮像装置 | |
JP2014002283A (ja) | 焦点調節装置及びその制御方法 | |
US11010030B2 (en) | Electronic apparatus capable of performing display control based on display mode, control method thereof, and non-transitory computer readable medium | |
JP5942343B2 (ja) | 撮像装置 | |
JP2021071732A (ja) | 制御装置、制御方法、コンピュータプログラム及び電子機器 | |
JP6801658B2 (ja) | 撮像装置、撮像方法、及びプログラム | |
JP6482247B2 (ja) | 焦点調節装置、撮像装置、焦点調節装置の制御方法、及びプログラム | |
JP2018004916A (ja) | 焦点検出装置 | |
JP5240591B2 (ja) | 撮像装置、合焦装置、撮像方法および合焦方法 | |
KR20170102755A (ko) | 라이트필드 카메라용 오토포커싱 장치 및 그 방법 | |
KR101756274B1 (ko) | 감시용 카메라 및 감시용 카메라의 동작 방법 | |
JP2016071275A (ja) | 撮像装置およびフォーカス制御プログラム | |
JP2017219786A (ja) | 制御装置、撮像装置、制御方法、プログラム、および、記憶媒体 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20130418 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 671665 Country of ref document: AT Kind code of ref document: T Effective date: 20140615 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602012001983 Country of ref document: DE Effective date: 20140717 |
|
REG | Reference to a national code |
Ref country code: SE Ref legal event code: TRGR |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 671665 Country of ref document: AT Kind code of ref document: T Effective date: 20140604 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: VDEP Effective date: 20140604 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140905 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140904 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20141006 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20141004 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602012001983 Country of ref document: DE |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 |
|
26N | No opposition filed |
Effective date: 20150305 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20141025 Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602012001983 Country of ref document: DE Effective date: 20150305 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20141025 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20151031 Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 Ref country code: BE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20121025 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20151031 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 5 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 6 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 7 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140604 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R079 Ref document number: 602012001983 Country of ref document: DE Free format text: PREVIOUS MAIN CLASS: H04N0005225000 Ipc: H04N0023000000 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230505 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20230920 Year of fee payment: 12 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: SE Payment date: 20230922 Year of fee payment: 12 Ref country code: FR Payment date: 20230920 Year of fee payment: 12 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20230920 Year of fee payment: 12 |