WO2012140715A1 - Dispositif de synchronisation et de reproduction de données, procédé de synchronisation et de reproduction de données, et programme de commande de synchronisation de données - Google Patents

Dispositif de synchronisation et de reproduction de données, procédé de synchronisation et de reproduction de données, et programme de commande de synchronisation de données Download PDF

Info

Publication number
WO2012140715A1
WO2012140715A1 PCT/JP2011/059010 JP2011059010W WO2012140715A1 WO 2012140715 A1 WO2012140715 A1 WO 2012140715A1 JP 2011059010 W JP2011059010 W JP 2011059010W WO 2012140715 A1 WO2012140715 A1 WO 2012140715A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
image
frame
reproduction
image data
Prior art date
Application number
PCT/JP2011/059010
Other languages
English (en)
Japanese (ja)
Inventor
章 野島
Original Assignee
東芝三菱電機産業システム株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 東芝三菱電機産業システム株式会社 filed Critical 東芝三菱電機産業システム株式会社
Priority to JP2013509678A priority Critical patent/JP5478776B2/ja
Priority to PCT/JP2011/059010 priority patent/WO2012140715A1/fr
Priority to TW100118960A priority patent/TWI454141B/zh
Publication of WO2012140715A1 publication Critical patent/WO2012140715A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/765—Interface circuits between an apparatus for recording and another apparatus
    • H04N5/77—Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources

Definitions

  • the present invention relates to a data-synchronized playback apparatus and data for synchronizing and displaying image data including a plurality of image frames captured by a camera and process data including a plurality of numerical data acquired in time series from the monitoring object
  • the present invention relates to a synchronized playback method and a data synchronization control program.
  • the numerical data of the plant at the time of the abnormality and the image frame are displayed on the monitor for monitoring the plant, and the numerical data and the image frame are displayed on the storage device. Is stored as a separate file, and when the history information screen is developed for investigating abnormalities, numerical data and image frames at the time of the occurrence of abnormalities are displayed at the same time to enable detailed plant monitoring.
  • a technique has been proposed (see, for example, Patent Document 1).
  • the data synchronous reproduction device of the above-mentioned Patent Document 1 when confirming the numerical data of the plant and the image frame at the time of occurrence of an abnormality later, the record of each numerical data and the date / time information added to each image frame are included. Based on the synchronized playback based on this, when the number of monitoring targets increases, or when storing numerical data and image frames during normal operation, the number of files and the storage capacity of the file increase, so data for a long time is stored, There is a problem that a storage device having a very large size is required when the image frame and the numerical data are reproduced in synchronization later.
  • Patent Document 1 since the data synchronous reproduction apparatus of Patent Document 1 performs synchronous reproduction by adding date and time information to numerical data and an image frame, an image frame compression / decoding technique such as MPEG-4 is used when storing an image. There is a problem that the development cost increases because a general-purpose image data reproduction program such as Quick Time (registered trademark) player cannot be used during image reproduction.
  • the present invention has been made in view of such a problem, and a data synchronous reproduction apparatus and data synchronous reproduction capable of synchronously reproducing an image frame included in image data and data included in process data at high speed. It is an object to provide a method and a data synchronization control program.
  • a first feature of the data synchronous playback device is that a plurality of image frames captured by a camera are imaged, and a frame number is assigned to each image frame, and the frame period of the image frames.
  • an image data storage unit that stores the data as image data
  • a process data storage unit that stores a plurality of data acquired in time series from the monitoring target as process data together with a sampling number of the data by assigning a record number to each data.
  • the image data stored in the image data storage unit is output to a monitor for playback, and the image data playback unit outputs the frame number of the image frame to be played back, and the image data playback unit outputs the frame number.
  • the frame number is set to the frame period of the image frame and the sampling interval of the data. It converts the record number of the data Zui is the data having the converted record number to having a data synchronization reproduction control unit for reproducing on the monitor.
  • a second feature of the data synchronous reproduction device is that the data synchronous reproduction control unit is designated when the data at a specific timing is designated in the process data displayed on the monitor.
  • the converted record number of the data is converted into the frame number based on the sampling interval of the data and the frame period of the image frame, and the converted frame number is output to the image data reproduction unit, The image data having the specified frame number is reproduced.
  • the third feature of the data synchronous reproduction device is that, when the range of data stored as a separate file is designated by external input in the process data displayed on the monitor, the data is designated.
  • An image in a range synchronized with the data in the specified range by converting the record number of the data in the specified range to the frame number of the image frame based on the sampling interval of the data and the frame period of the image frame A synchronous image frame selection unit for selecting a frame; a synchronous storage control unit for storing the data in a range specified by an external input and the image frame in a range selected by the synchronous image frame selection unit as the same file; , To have.
  • the fourth feature of the data synchronous reproduction device is that, based on an external input, an image data reproduction area where the image data is displayed by the image data reproduction unit, and a data synchronous reproduction control unit.
  • a screen layout setting unit for setting a process data reproduction area on which the process data is displayed on the monitor and setting the number and size of image data to be reproduced within a predetermined condition in the image data reproduction area It is in having.
  • the first feature of the data synchronous reproduction method is that a plurality of image frames obtained by photographing a monitoring target by a camera are stored as image data together with a frame number of each image frame by assigning a frame number to each image frame.
  • the output frame number is converted into the record number of the data based on the frame period of the image frame and the sampling interval of the data, The data having the converted record number is reproduced on the monitor. It lies in having a step.
  • the second feature of the data synchronous reproduction method is that, in the process data displayed on the monitor, when the data at a specific timing is designated, the record number of the designated data is designated. Is converted into the frame number based on the sampling interval of the data and the frame period of the image frame, and the image data having the converted frame number is reproduced.
  • the third feature of the data synchronous reproduction method is that, in the process data displayed on the monitor, when a range of data to be stored as a separate file is specified, the specified range of the data A data record number is converted into a frame number of the image frame based on the sampling interval of the data and a frame period of the image frame, and an image frame in a range synchronized with the data in a specified range is selected. And storing the data in the specified range and the image frame in the selected range as the same file.
  • the fourth feature of the data synchronous reproduction method is that, based on an external input, an image data reproduction area where the image data is displayed by the image data reproduction unit, and a data synchronous reproduction control unit. Setting a process data reproduction area on which the process data is displayed on the monitor, and setting the number and size of image data to be reproduced within a predetermined condition in the image data reproduction area. There is.
  • the first feature of the data synchronization control program is that a plurality of image frames in which a camera captures a monitoring target is imaged on a computer together with a frame number of each image frame and a frame period of the image frame.
  • the output frame number is converted into the record number of the data based on the frame period of the image frame and the sampling interval of the data.
  • the data having the converted record number Certain that to execute the steps of reproducing on the monitor.
  • the second feature of the data synchronization control program according to the present invention is that, in the process data displayed on the monitor, when the data at a specific timing is specified, the data of the specified data The record number is converted into the frame number based on the sampling interval of the data and the frame period of the image frame, and the step of reproducing the image data having the converted frame number is executed.
  • the third feature of the data synchronization control program is that, in the process data displayed on the monitor, when a range of data to be stored as a separate file is specified, the specified range of the data A data record number is converted into a frame number of the image frame based on the sampling interval of the data and a frame period of the image frame, and an image frame in a range synchronized with the data in a specified range is selected. And executing the step of storing the data in the specified range and the image frame in the selected range as the same file.
  • the fourth feature of the data synchronization control program according to the present invention is that, based on an external input, an image data reproduction area where the image data is displayed by the image data reproduction unit, and the data synchronization reproduction control unit. Setting a process data reproduction area on which the process data is displayed on the monitor, and setting the number and size of image data to be reproduced within a predetermined condition in the image data reproduction area; There is to make it.
  • the frame number of the image frame included in the image data and the record number of the data included in the process data are not based on the time information. Therefore, it is not necessary to add time information to the data and the image frame.
  • the number of files and the storage capacity of the files can be reduced, and storage and synchronized playback over a longer period than before can be performed with a storage device having the same storage capacity.
  • image frame compression / decoding technology such as MPEG-4 can be used when saving images
  • a general-purpose image data playback program such as Quick Time (registered trademark) player can be used when playing back images. This can reduce development costs.
  • data and image frames can be synchronized and played back at high speed.
  • FIG. 1 is a diagram showing a configuration example of an online data management system including a data synchronous reproduction apparatus according to the present invention.
  • FIG. 2 is a block diagram showing a configuration example of the data synchronous reproduction apparatus according to the present invention.
  • FIG. 3A is a diagram illustrating an example of a file format of image data and process data stored in the data storage unit of the data synchronous playback device.
  • FIG. 3B is a diagram illustrating an example of a data structure of image data.
  • FIG. 3C is a diagram illustrating an example of a data structure of process data.
  • FIG. 4 is a flowchart illustrating a procedure of data synchronization display processing by the data synchronization reproduction apparatus of the first embodiment.
  • FIG. 5 is a diagram showing a specific example when the synchronous reproduction processing from image data to numerical data is performed.
  • FIG. 6 is a flowchart illustrating a procedure of data synchronization display processing of the data synchronization reproduction apparatus according to the first embodiment.
  • FIG. 7 is a diagram showing a specific example when the synchronous reproduction process from numerical data to image data is performed.
  • FIG. 8 is a diagram illustrating an example of a display screen displayed on the monitor of the data synchronous reproduction device of the first embodiment.
  • FIG. 9 is a diagram illustrating another example of the display screen displayed on the monitor of the data synchronous reproduction device of the first embodiment.
  • FIG. 10 is a diagram illustrating another example of the display screen displayed on the monitor of the data synchronous reproduction device of the first embodiment.
  • FIG. 11 is a diagram illustrating another example of the display screen displayed on the monitor of the data synchronous playback device of the first embodiment.
  • FIG. 12 is a diagram illustrating another example of the display screen displayed on the monitor of the data synchronous reproduction device of the first embodiment.
  • FIG. 1 is a diagram showing a configuration example of an online data management system including a data synchronous reproduction apparatus according to the present invention.
  • This online data management system is, for example, a hot rolling plant system that rolls heated iron, stainless steel, and aluminum slabs (steel pieces) with a rolling mill and finishes them into hot rolled coils with a thickness of several mm to several tens of mm.
  • Image data of each monitoring target in Processa Implant System which performs pickling, annealing, plating, painting, cutting, etc., to finish coils manufactured in hot rolling plants and cold rolling plants into final products
  • the process data acquired from each monitoring target are displayed in synchronization on the monitor and managed.
  • this online data management system includes a data synchronized playback apparatus 1 according to the present invention, a plurality of ITV (Industrial Television) cameras 21 to 2m (m is a natural number), a camera switch 3 and a digital image.
  • the conversion unit 4 and control devices 51 to 5n (n is a natural number) such as a programmable logic controller (PLC) are connected to each other.
  • PLC programmable logic controller
  • the operation state and control state of each of the monitoring objects 61 to 6p are determined by image data captured by a plurality of ITV cameras 21 to 2m and process data acquired by the control devices 51 to 5n. This is a monitoring system.
  • the data synchronous reproduction apparatus 1 and the digital image conversion unit 4 are connected by a LAN 7 such as Ethernet (registered trademark), while the data synchronous reproduction apparatus 1 and the control devices 51 to 5n are: It is connected by a control network 8 different from the LAN 7.
  • the control network 8 can be replaced with the LAN 7.
  • the data synchronous reproduction apparatus 1 is composed of a computer apparatus in which a monitor (display) 12, a keyboard 13, a mouse 14 and the like are connected to the apparatus main body 11.
  • the data synchronous reproduction apparatus 1 includes image data from a plurality of ITV cameras 21 to 2m converted from an analog video signal to a digital video signal by the digital image conversion unit 4, and a plurality of data acquired from the control devices 51 to 5n.
  • the process data including the numerical data is input and displayed on the monitor 12 for normal display or synchronous display processing.
  • the configuration and operation of the data synchronous reproduction apparatus 1 according to the present invention, the configuration of image data, process data, and the like will be described later.
  • the ITV cameras 21 to 2m shoot multiple monitoring objects 61 to 6p and output analog video signals such as PAL and NTSC formats. Even fixed cameras can send pan and tilt by sending control signals. A camera capable of remote operation such as zooming may also be used.
  • the ITV cameras 21 to 2m may or may not be provided with an audio microphone.
  • One ITV camera 21 to 2m may be provided for each monitoring object 61 to 6p, or a plurality of ITV cameras 21 to 2m may be provided.
  • the camera switch 3 switches and outputs a predetermined number (for example, four in this embodiment) of video signals to be displayed on the monitor 12 from the video signals of the plurality of ITV cameras 21 to 2m. To do.
  • a predetermined number for example, four in this embodiment
  • this switching function is further applied to the digital image conversion unit 4 or the data synchronous reproduction apparatus 1.
  • the camera switch 3 may be omitted.
  • the digital image conversion unit 4 compresses and encodes NTSC or PAL format analog video signals output from a plurality of ITV cameras 21 to 2m into digital image data such as MPEG-4 and sends the digital image data to the data synchronous reproduction apparatus 1. Output. Note that the digital image conversion unit 4 can be omitted when a plurality of ITV cameras 21 to 2m can perform A / D conversion of video signals photographed inside the camera and output digital image data.
  • the control devices 51 to 5n control the operations of the monitoring objects 61 to 6p such as motors, heaters, and hydraulic machines by a program, and control the respective I / O (input / output) units 511 to 5n1.
  • the process data including numerical data detected in time series at a predetermined sampling interval to be described later is acquired from each of the monitoring objects 61 to 6p.
  • FIG. 2 is a block diagram showing a configuration example of the data synchronous reproduction apparatus 1 according to the present invention.
  • the data synchronous reproduction apparatus 1 is composed of a computer device in which a monitor (display) 12, a keyboard 13, a mouse 14 and the like are connected to the apparatus main body 11.
  • the apparatus main body 11 includes a CPU 111, a memory 112, a storage unit 113 such as an HDD or a large-capacity memory, an external device interface (hereinafter abbreviated as I / F) unit 114, and a control network.
  • the I / F unit 115 is connected to the LAN I / F unit 116 and the like via an internal bus 117.
  • the CPU 111 functions as an image data reproduction unit of the present invention by executing an image data reproduction program among various application programs stored in the program storage unit 113a of the storage unit 113, and executes a data synchronization control program. Accordingly, the data synchronization playback control unit, the synchronized image frame selection unit, the synchronization storage control unit, and the screen layout setting unit of the present invention function.
  • the memory 112 is used as a calculation area or the like for temporarily storing or developing data when the CPU 111 executes various application programs.
  • the storage unit 113 includes a program storage unit 113a and a data storage unit 113b.
  • the program storage unit 113a includes an OS (operating system), an image data reproduction program for causing the CPU 111 to function as an image data reproduction unit of the present invention, a data synchronization reproduction control unit, a synchronous image frame selection unit of the present invention, An application program such as a data synchronization control program for functioning as a synchronization storage control unit and a screen layout setting unit is stored.
  • an image data reproduction program an image data reproduction program programmed exclusively for this apparatus or a general-purpose image data reproduction program such as a Quick Time (registered trademark) player may be used. .
  • the data storage unit 113b captures image data including a plurality of image frames obtained by photographing a plurality of monitoring objects 61 to 6p with a plurality of ITV cameras 21 to 2m and A / D-converting them.
  • the process data storage unit of the present invention stores process data including a plurality of numerical data acquired in time series from the plurality of monitoring objects 61 to 6p by the control devices 51 to 5n.
  • the program storage unit 113a and the data storage unit 113b are provided in one storage unit 113.
  • the program storage unit 113a and the data storage unit 113b are different from each other.
  • it may be configured as a storage unit.
  • the external device I / F unit 114 is an interface for connecting an external device such as the monitor 12, the keyboard 13, and the mouse 14 to the apparatus main body 11.
  • the control network I / F unit 115 is an interface for connecting the control network 19 and the apparatus main body 11.
  • the LAN I / F unit 116 is an interface for connecting the LAN 7 and the apparatus main body 11.
  • 3A to 3C are diagrams showing examples of file formats of image data and process data stored in the data storage unit 113b of the data synchronous reproduction apparatus 1, and data structures of the image data and process data, respectively. .
  • Data D1 and process data D2 including a plurality of numerical data acquired in time series from the plurality of monitoring objects 61 to 6p by the control devices 51 to 5n are included in two sections S1 and S2 in one data file Df. I remember it.
  • the image data D1 and the process data D2 may be stored in different sections of different files.
  • FIG. 3B is a diagram illustrating a data structure of the image data D1 stored in the data storage unit 113b.
  • the image data D1 is obtained by converting analog video signals output from each of the plurality of ITV cameras 21 to 2m into digital image data such as an MPEG-4 encoding system by the digital image conversion unit 4. It is composed of image header information D11 and a plurality of image frames D12 having frame numbers # 1 to q (q is a natural number) obtained by photographing the monitoring objects 61 to 6p targeted by the ITV cameras 21 to 2m. Has been.
  • image data D1 is shown, but actually there is image data D1 for each of a plurality of ITV cameras 21 to 2m.
  • the image header information D11 constitutes at least a frame period Tf which is a time per frame of the image frame D12 constituting the image data D1 and the image data D1, as shown in FIG. 3B. And the total number of frames q of the plurality of image frames D12.
  • the image header information D11 includes the camera identification information for identifying the ITV cameras 21 to 2m that output the image data D1, and the date and time of the image data D1 from the same ITV cameras 21 to 2m.
  • date and time identification information for identifying different image data D1 may be included.
  • link information for linking with the corresponding process data D2 may be included in the image header information D11.
  • the image frame D12 is data obtained by performing high-efficiency compression coding by the digital image conversion unit 4 using a moving image compression method such as MPEG-4 on an analog video signal photographed by the ITV cameras 21 to 2m.
  • a moving image compression method such as MPEG-4
  • MPEG-4 a moving image compression method
  • the H.264 encoding method, Motion JPEG that does not perform inter-frame prediction, or the like may be used.
  • any format of digital image data transmitted by the LAN 7 may be used.
  • time information such as decoding time information (DTS) and reproduction time information (PTS) may be given to each of the plurality of image frames D12 for normal decoding and reproduction. Time information such as decoding time information (DTS) and reproduction time information (PTS) is not used for the synchronous reproduction processing with process data.
  • DTS decoding time information
  • PTS reproduction time information
  • the image data D1 does not have time information or is not used for synchronized reproduction with the process data D2 even if it has time information. For this reason, in the first embodiment, during the synchronous reproduction process of the image data D1 and the process data D2, the sampling start time Ts1 stored in the process data header information D21 of the process data D2 corresponding to the image data D1 is This is used as the start time of the image data D1, that is, the reproduction start information of the image frame D12 of frame number # 1. Then, the frame playback start time of the image frame D12 of frame number # 2 to p after the image frame D12 of frame number # 1 is the sampling start time Ts1 used as the playback start time of the image frame D12 of frame number # 1.
  • the time information of each image frame D12 is not necessary for the synchronized reproduction process with the process data D2, and even if there is time information or the like, it is not used (referenced). Thereby, since reproduction is performed based on the frame number without using (referring to) the time information of each image frame D12, the reproduction and cueing can be speeded up.
  • FIG. 3C is a diagram illustrating a data structure of the process data D2 stored in the data storage unit 113b.
  • the process data D2 is binary data composed of numerical data D22 acquired in time series by the control devices 51 to 5n from the plurality of monitoring objects 61 to 6p at predetermined sampling intervals Ts2, as shown in FIG. 3C. And the process data header information D21 and a plurality of numerical data D22 of record numbers # 1 to r (r is a natural number) acquired from the monitoring objects 61 to 6p by the plurality of control devices 51 to 5n. .
  • the process data header information D21 includes at least a sampling start time Ts1 that is a time at which acquisition of the process data D2 is started, a sampling interval Ts2 that is an interval at which the numerical data D22 is acquired, and a monitoring target Stores the total number of records r of a plurality of numerical data D22 acquired from the objects 61 to 6p at a predetermined sampling interval Ts2.
  • the sampling interval Ts2 of the process data header information D21 is, for example, 1 msec. However, this is an example and may be other than 1 msec.
  • each numerical data D22 includes one or a plurality of items (items) of item numbers # 1 to s (s is a natural number) as shown in FIG. 3C.
  • item # 1 is a temperature sensor value
  • item # 2 is a pressure sensor value.
  • the item is not related to the main part of the invention and may be omitted.
  • the plurality of image frames D12 constituting the image data D1 and the plurality of numerical data D22 constituting the process data D2 are not synchronized at the time of recording. Further, 1/30 second of the frame period Tf of the image frame does not coincide with 1 ms of the numerical data sampling interval Ts2. For this reason, the data synchronous playback apparatus 1 of the first embodiment displays time information of the image data D1 such as decoding time information (DTS) and playback time information (PTS), and time information indicating the data acquisition time of the process data D2.
  • DTS decoding time information
  • PTS playback time information
  • the data synchronous reproduction apparatus 1 of the first embodiment performs synchronous reproduction processing using image frames and numerical data using frame numbers and record numbers without using or referring to time information.
  • FIG. 4 is a flowchart showing a procedure of data synchronization display processing by the data synchronization reproduction apparatus 1 of the first embodiment.
  • the user operates the keyboard 13 and mouse 14 of the data synchronized playback apparatus 1 and clicks (selects) a synchronized playback button (not shown) or the like, and the image data D1 and the process are transmitted to the apparatus main body 11.
  • a synchronous reproduction instruction with data D2 is input (S400 “YES”)
  • the CPU 111 activates the image data reproduction program stored in the program storage unit 113a and the data synchronization control program, and starts synchronous reproduction processing.
  • the CPU 111 of the apparatus main body 11 first functions as an image data reproduction unit by executing an image data reproduction program, and the image data header information D11 of the image data D1 stored in the data storage unit 113b and the process data.
  • the process data header information D21 of D2 is read out and temporarily stored in the memory 112, and the frame period Tf included in the image data header information D11 and the sampling interval Ts2 included in the process data header information D21 are acquired (S410).
  • the CPU 111 acquires, for example, 1/30 seconds as the frame period Tf and 1 ms as the sampling interval Ts2.
  • the CPU 111 may acquire the frame period Tf and the sampling interval Ts2 in advance by the process of S410 before the process of S400. Further, when the frame period Tf included in the image data header information D11 is known, the CPU 111 does not acquire the frame period Tf included in the image data header information D11, and the sampling interval Ts2 included in the process data header information D21. Only get.
  • the CPU 111 functions as an image data reproduction unit of the present invention by executing an image data reproduction program, and uses a timer (not shown) of the OS to 1 / which is the frame period Tf of the image data D1.
  • a timer event signal is generated at intervals of 30 seconds (S420).
  • the CPU 111 When the CPU 111 generates a timer event signal at 1/30 second intervals that are the frame period Tf (S430 “YES”), the CPU 111 functions as the image data reproduction unit of the present invention and stores the image stored in the data storage unit 113b.
  • the image frame D12 constituting the data D1 is read and the image frame D12 is encoded in the MPEG-4 format or the like, the image frame D12 is decoded in the MPEG-4 format or the like and reproduced on the monitor 12, The frame number of the reproduced image frame D12 is output (S440). At this time, when the image frame to be reproduced is designated by the user by the frame number or the like, reproduction is performed from the designated image frame.
  • the designation of the image frame D12 is not limited to the frame number or the like, but may be designated by the cursor on the display screen displayed on the monitor 12 with the keyboard 13 or the mouse 14, or when the image frame D12 is not designated. Then, reproduction is started from the top image frame D12 in the image data D1, that is, the image frame D12 having the frame number # 1. At this stage, the image data D1 is reproduced asynchronously with the process data D2.
  • the CPU 111 functions as the data synchronous reproduction control unit of the present invention by executing the data synchronous control program according to the present invention, and first, the frame number of the currently reproduced image frame D12 and the frame period Tf. Are acquired (S450).
  • the CPU 111 uses the known frame period Tf.
  • the CPU 111 temporarily stores the “relative time from the first image frame” obtained from the previous equation (1) in the memory 112 or the like as shown in the following equation (2). By dividing the stored process data D2 by the sampling interval Ts2, the record number of the numerical data D22 synchronized with the currently displayed image frame D12 is calculated (S470).
  • Record number of numerical data to be synchronized relative time from first image frame / sampling interval Ts2 (2)
  • the CPU 111 selects and displays the numerical data D22 of the corresponding record number from the process data D2 based on the “record number of the numerical data to be synchronized” obtained by the above equation (2) (S480).
  • the CPU 111 does not refer to the time information, and the numerical data D22 synchronized in time with the currently displayed image frame D12 through the frame number of the image frame D12 and the record number of the numerical data D22. Can be played back synchronously.
  • the CPU 111 converts the numerical data D22 of the record number into the graph format or the like and updates the value included in the range to be displayed in the graph. To update the graph display. If the numerical data D22 is composed of a plurality of items # 1 to #s, the plurality of items # 1 to #s are reflected in the graph.
  • the CPU 111 has a frame period Tf of 1 until a synchronous reproduction stop instruction is input by the user or the reproduction ends up to the final image frame D12 of the currently reproduced image data D1 (S490 “NO”). / Every time a timer event signal is generated at 30-second intervals (S430 “YES”), the processing from S440 to S480 described above synchronizes the image frame D12 in which the frame number and the record number are associated with the numerical data D22. Perform playback processing.
  • the image frame D12 that forms the image data D1 is made by associating the frame number of the image data D1 with the record number of the numerical data D22 without referring to the time information. And the numerical data D22 constituting the process data D2 can be reproduced in synchronization.
  • the CPU 111 terminates the synchronous playback process when the user inputs a synchronous playback stop instruction or when the playback is completed up to the final image frame D12 of the currently reproduced image data D1 (S490 “YES”). .
  • FIG. 5 shows a specific example when the synchronous reproduction process from the image data D1 to the process data D2 is performed.
  • the image data header information D11 of the image data D1 and the process data header information D21 of the process data D2 are omitted.
  • the sampling interval Ts2 is 1 mm second
  • the image frame D12 constituting the image data D1 and the numerical data D22 constituting the process data D2 are not synchronized at the time of recording. Even if the frame period Tf of the frame D12 and the sampling interval Ts2 of the numerical data D22 do not coincide with each other, the frame number of the image frame D12 and the numerical data D22 are not used without using the time information of the image frame D12 or the numerical data D22. Using the record number or the like, the image frame D12 and the numerical data D22 can be reproduced synchronously.
  • the data synchronous reproduction apparatus 1 of the first embodiment it is not necessary to refer to the time information of both data in the synchronous reproduction processing of the image data D1 and the process data D2, so that the image data D1 and the process data There is no need to analyze and find out the syntax (structure) of D2 and header information, and it is possible to execute synchronized playback processing at high speed.
  • the relative time from the first image frame is calculated by the above-described formula (1) and formula (2), Although the description has been made so that the record number of the numerical data D22 to be synchronously reproduced is calculated one by one based on the relative time from the first image frame, the present invention is not limited to this, and the above-described equations (1) and (2) ) Once, the record number of the numerical data D22 to be synchronously reproduced is calculated, and thereafter the record number of the numerical data D22 to be synchronously reproduced is generated every time a timer event occurs every frame period Tf from the OS timer. Of course, it may be incremented by +1.
  • the record number of the numerical data D22 to be synchronized and reproduced first is calculated by the processing of S450 to S470 and the synchronized reproduction is performed, the calculation is performed first without executing the processing of S450 to S470.
  • the synchronous reproduction processing of the image frame D12 constituting the image data D1 and the numerical data D22 constituting the process data D2 becomes possible. Synchronous playback processing with a light processing load is possible.
  • FIG. 6 is a flowchart showing a procedure of data synchronization display processing by the data synchronization reproduction apparatus 1 of the first embodiment.
  • the user designates the numerical data D22 in the process data D2 by moving, for example, a scroll bar or cursor with the mouse 14 or the keyboard 13, etc.
  • a synchronous reproduction instruction with the image frame D12 from the numerical data D22 is input to the apparatus main body 11 (S600 “YES”)
  • the CPU 111 functions as a data synchronous reproduction control unit by executing the data synchronous control program.
  • the sampling interval Ts2 is acquired from the record number of the numerical data D21 and the process data header information D21 (S610).
  • the CPU 111 functions as a data synchronous reproduction control unit, and acquires the frame period Tf from the image data header information D11 of the image data D1 (S620). Note that the CPU 111 has already stored the sampling interval Ts2 and the frame period Tf, and if it is known, the acquisition processing may be omitted.
  • the CPU 111 functions as a data synchronous reproduction control unit, and based on the sampling interval Ts2 and the record number of the numerical data D22 designated by a scroll bar, cursor, etc., as shown in the following equation (3). Then, “relative time from the first numerical data” in the process data D2 is calculated (S630).
  • Relative time from the first numerical data current record number ⁇ sampling interval Ts2 ... (3)
  • the CPU 111 divides the “relative time from the first record” calculated by the equation (3) by the frame period Tf that is the time per frame of the image frame D12.
  • the “frame number of the image frame to be synchronized” synchronized with the current numerical data is calculated (S640).
  • Frame number of image frame to be synchronized relative time from head / frame period Tf (4)
  • the CPU 111 functions as an image data reproducing unit, and based on the frame number calculated by the equation (4), the image frame having the corresponding frame number is selected from the image data D1 stored in the data storage unit 113b. D12 is read and reproduced (S640).
  • FIG. 7 shows a specific example when the synchronous reproduction process from the process data D2 to the image data D1 is performed.
  • the image data header information D11 of the image data D1 and the process data header information D21 of the process data D2 are omitted.
  • the process data D2 display area of the process data D2 displayed on the monitor 12 among the process data D2 the mouse 14 and the keyboard 13 are displayed.
  • the scroll bar, the cursor, or the like is moved by specifying the numerical data D22 of the 134th record number # 134 from the first numerical data D22.
  • the frame number of the image frame D12 and the record number of the numerical data D22 are used without using the time information of the image frame D12 and the numerical data D22.
  • the image frame D12 of the corresponding image data D1 can be synchronously reproduced.
  • the data storage unit 113b of the storage unit 113 includes a plurality of numerical data acquired in time series from the plurality of monitoring objects 61 to 6p by the control devices 51 to 5n.
  • the process data D2 is stored, the data included in the process data D2 is not limited to numerical data, and may be bit data or text data.
  • FIGS. 8 to 12 are diagrams showing specific examples of display screens 121 to 125 displayed on the monitor 12 of the data synchronous reproduction apparatus 1 of the first embodiment, respectively.
  • the display screens 121 to 125 shown in FIGS. 8 to 12 are displayed on the screen layout setting by the CPU 111 of the data synchronization playback apparatus 1 of the first embodiment executing the data synchronization control program stored in the program storage unit 113a.
  • the display screen which functions as a part and is set or changed. That is, in the display screens 121 to 125 shown in FIGS. 8 to 12, the CPU 111 functions as a screen layout setting unit to display an image data reproduction area where image data is displayed based on a user instruction and process data.
  • This is an example of a display screen in which a process data reproduction area to be reproduced is set on the monitor 12 and the number and size of image data to be reproduced are set and displayed within predetermined conditions in the image data reproduction area.
  • the image data display area 121 a is provided above the center of the display screen 121 and functions as a screen layout setting unit, while the process data display area 121 b is provided below the center of the display screen 121. ing.
  • the display screen 121 four image data display windows 121a1 to 121a4 are provided in the image data display area 121a, and the four image data display windows 121a1 to 121a4 are arranged in a plurality of ITV cameras 21 to 2m.
  • the process data display area 121b displays the monitoring object captured by the four ITV cameras with camera numbers # 1 to 4 (camera1 to 4) switched to the camera switch 3.
  • Process data D2 is being reproduced.
  • the horizontal axis in the horizontal direction in the figure indicates the time axis
  • the vertical axis in the vertical direction in the figure indicates the value (level) of the numerical data D22 constituting the process data D2. Show.
  • the image data D1 reproduced in each of the four image data display windows 121a1 to 121a4 and the process data D2 reproduced in the process data display area 121b are described above by the CPU 111 functioning as a data synchronous reproduction control unit.
  • the image frame D12 is reproduced in synchronization with the frame number of the image frame D12 and the record number of the numerical data D22.
  • a process data time change bar 121b2 that moves in the horizontal direction in the figure is displayed to change the time of the process data D2 to be reproduced.
  • the user may operate the cursor (not shown) of the process data time change bar 121b2 on the display screen 121 with the keyboard 13 or the mouse 14 to execute enlargement / reduction or enlargement display of the time axis of the process data D2. it can.
  • 121b3 is an extraction recording start position designation bar for designating the extraction recording (snapshot) start position of the process data displayed in the process data display area 121b
  • 121b4 is displayed in the process data display area 121b.
  • This is an extraction recording end position designation bar for designating the extraction recording end position of the processed process data, and is a bar used in extraction recording (snapshot) of Embodiment 12 to be described later, and is not used in Embodiment 1.
  • the user can move these bars in the process data display area 121b by using the mouse 14 or the keyboard 13 as described above.
  • FIG. 9 shows a state in which the process data display area 122b is enlarged on the entire display screen 122 in order to display the process data D2 in an enlarged manner. Therefore, in the display screen 122 shown in FIG. 9, the display screen 121 shown in FIG. 8 cannot display the image data display area 121a existing in the upper half of the center of the display screen 121. Therefore, the image data is superimposed on the process data display area 122b.
  • a display area 122a is displayed, and four image data display windows 122a1 to 122a4 are provided in the data display area 122a.
  • an image data display area 123 a is provided above the center of the display screen 123, while a process data display area is provided below the center of the display screen 123.
  • 123b is provided.
  • two image data display areas 123a1 and 123a2 are provided in the image data display area 123a, and the image data D1 of the moving image from the ITV camera 21 with the camera number # 1 is displayed in the image data display area 123a1.
  • still image data D1 from the ITV camera 21 (camera1) of the same camera number # 1 is displayed in the image data display area 123a2.
  • the process data display area 123b is synchronized with the image data D1 of the moving image from the ITV camera 21 of the camera number # 1 displayed in the image data display area 123a1, and the camera number #
  • the process data D2 acquired by the control device from the monitored object photographed by one ITV camera 21 is displayed.
  • the process data display area 123b is not different from the process data display areas 121b and 122b on the display screens 121 and 122 shown in FIGS.
  • the image data display area 124a is provided above the center of the display screen 124, while the process data is displayed below the center of the display screen 124, as in the display screen 121 shown in FIG.
  • An area 124b is provided.
  • two image data display areas 124a1 are provided in the image data display area 124a as in the display screen 123 shown in FIG. 10, and the ITV camera with the camera number # 1 is provided in the image data display area 124a1. While moving image data D1 from 21 is displayed, image data D1 of four still images for different shooting times from the ITV camera 21 (camera1) of the same camera number # 1 are displayed in the image data display area 124a2. is doing.
  • the process data D2 includes text data such as the lot number of the rolled material being rolled
  • the text data 124c is displayed in the process data display area 124b.
  • the process data display area 124b is not different from the process data display areas 121b to 123b of the display screens 121 to 123 shown in FIGS.
  • the image data display area 125a is provided above the center of the display screen 125, while the process data is displayed below the center of the display screen 125, like the display screen 121 shown in FIG. An area 125b is provided.
  • the image data display area 125a is selected by, for example, a camera image designation button 125a1 for selecting two camera images from four camera images and the camera image designation button 125a1. Moving image data from the two ITV cameras 21, 23 (camera 1, 3) with camera numbers # 1, 3 are displayed in the image data display areas 125a2, 125a3.
  • the window size of the image data display areas 125a2 and 125a3 on the display screen 125 is smaller than the window of the image data display areas 123a1 and 123a2 on the display screen 123 shown in FIG. Since it is larger than the window of the image data display areas 123a1 to 121a4 on the display screen 121, the image data D1 is easy to see.
  • the process data display area 125b is no different from the process data display areas 121b to 124b of the display screens 121 to 124 shown in FIGS.
  • the frame number and frame of the image frame D12 constituting the image data D1 without referring to the time information of the image data D1 and the process data D2.
  • the period Tf the record number of the numerical data D22 constituting the process data D2, the sampling interval Ts2, and the like
  • the image frame D12 and the numerical data D22 can be synchronized and reproduced, and the display shown in FIGS. Display screens as shown on the screens 121 to 125 can be displayed on the monitor 12.
  • the data synchronous reproduction apparatus 1 of the first embodiment it is not necessary to refer to the time information of both data in the synchronous reproduction processing of the image data D1 and the numerical data D22.
  • the capacity can be reduced, and a storage device having the same storage capacity as that of the prior art can perform storage and synchronous reproduction for a longer period than before.
  • image frame compression / decoding technology such as MPEG-4 can be used when saving images
  • a general-purpose image data playback program such as Quick Time (registered trademark) player can be used when playing back images. This can reduce development costs.
  • data and image frames can be synchronized and played back at high speed.
  • Embodiment 2 In the first embodiment, the synchronous reproduction process of the image data D1 and the process data D2 using the frame number of the image frame D12 and the record number of the numerical data D22 without referring to the time information has been described.
  • the second embodiment in addition to the function of the first embodiment, by further extracting and recording the numerical data D22 in the specified range from the process data D2, that is, by specifying the range of the numerical data D22 to be recorded by snapshot, The image frame D12 synchronized with the numerical data D22 in the specified range can be automatically stored. Therefore, in the data synchronous reproduction device of the second embodiment, the snapshot recording function is further added to the function of the data synchronous reproduction device 1 of the first embodiment, and the data stored in the program storage unit 113a.
  • the synchronization control program includes a program for functioning as a synchronization storage control unit that is a function of this snapshot recording. Since the configuration of the data synchronous reproduction apparatus of the second embodiment is the same as that of the first embodiment, only the characteristic operation will be described with reference to the configuration of the first embodiment shown in FIGS.
  • the user moves the process data display area 121b displayed on the monitor 12 by using the mouse 14 or the keyboard 13 and the like, for example, as shown in FIG.
  • the extraction record start position designation bar 121b3 and the extraction record end position designation bar 121b4 are extracted from the process data D2 and designated separately for the range of process data D2.
  • the CPU 111 executes the data synchronization control program stored in the program storage unit 113a, and first functions as a synchronized image frame selection unit.
  • the record number of the numerical data D22 in the specified range is set in the range synchronized with the numerical data D22 in the specified range according to the equations (3), (4), etc., as in the case of the synchronized playback in the first embodiment.
  • the frame number of the image frame D12 is converted.
  • the CPU 111 functions as a synchronous storage control unit by executing the data synchronization control program, and associates the numerical data D22 in the range specified by the user with the image frame D12 synchronized with the numerical data D22 in the specified range.
  • the data file Df of the image data D1 and the process data D2 is different from the data file Df, that is, the numerical data D22 in the range specified by the user from the data file Df of the image data D1 and the process data D2,
  • the image frame D12 synchronized with the numerical data D22 in the specified range is stored in the data storage unit 113b in a form that is extracted.
  • the file can be attached to an e-mail or the like by a mail program or the like and transmitted to another computer apparatus via a network such as the Internet or an intranet. .
  • the present invention can be applied to an online data management system for monitoring a plant.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Television Signal Processing For Recording (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Studio Devices (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Recording Measured Values (AREA)

Abstract

La présente invention a pour objet de pourvoir à un dispositif de synchronisation et de reproduction de données qui puisse rapidement synchroniser et reproduire des images individuelles et des données numériques. En conséquence, ce dispositif de synchronisation et de reproduction de données (1) comprend les éléments suivants : une unité de stockage de données (113b) qui stocke des données d'image qui comprennent une pluralité d'images individuelles d'objets de surveillance (61-6p) captées par des caméras (31-3n), et des données de processus qui comprennent une pluralité d'éléments de données numériques acquis en séquence chronologique à partir des objets de surveillance ; une unité de stockage de programme (113a) qui stocke un programme de commande de synchronisation de données qui amène des données d'image à être reproduites et, sur la base du numéro d'image et de la période d'image d'une image individuelle et de l'intervalle d'échantillonnage et du numéro d'enregistrement de données numériques, synchronise et reproduit des données d'image et des données numériques ; et une unité centrale (CPU) (111) qui exécute le programme de commande de synchronisation de données.
PCT/JP2011/059010 2011-04-11 2011-04-11 Dispositif de synchronisation et de reproduction de données, procédé de synchronisation et de reproduction de données, et programme de commande de synchronisation de données WO2012140715A1 (fr)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2013509678A JP5478776B2 (ja) 2011-04-11 2011-04-11 データ同期再生装置、データ同期再生方法およびデータ同期制御プログラム
PCT/JP2011/059010 WO2012140715A1 (fr) 2011-04-11 2011-04-11 Dispositif de synchronisation et de reproduction de données, procédé de synchronisation et de reproduction de données, et programme de commande de synchronisation de données
TW100118960A TWI454141B (zh) 2011-04-11 2011-05-31 資料同步再生裝置、資料同步再生方法及資料同步控制程式

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2011/059010 WO2012140715A1 (fr) 2011-04-11 2011-04-11 Dispositif de synchronisation et de reproduction de données, procédé de synchronisation et de reproduction de données, et programme de commande de synchronisation de données

Publications (1)

Publication Number Publication Date
WO2012140715A1 true WO2012140715A1 (fr) 2012-10-18

Family

ID=47008926

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2011/059010 WO2012140715A1 (fr) 2011-04-11 2011-04-11 Dispositif de synchronisation et de reproduction de données, procédé de synchronisation et de reproduction de données, et programme de commande de synchronisation de données

Country Status (3)

Country Link
JP (1) JP5478776B2 (fr)
TW (1) TWI454141B (fr)
WO (1) WO2012140715A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015060612A (ja) * 2013-09-19 2015-03-30 株式会社デンソー 情報処理装置
CN104707868A (zh) * 2013-12-11 2015-06-17 东芝三菱电机产业***株式会社 数据分析装置
JP6113370B1 (ja) * 2016-01-22 2017-04-12 三菱電機株式会社 記憶装置、制御装置、表示装置及び記憶方法
JP2020102130A (ja) * 2018-12-25 2020-07-02 東芝三菱電機産業システム株式会社 データ収集再生システム

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0579951A (ja) * 1991-09-18 1993-03-30 Hitachi Ltd 監視システム
JPH0620180A (ja) * 1992-06-30 1994-01-28 Hitachi Ltd 事象再現装置
JP2005188998A (ja) * 2003-12-24 2005-07-14 Toyota Motor Corp 運転行動解析装置
JP2005332069A (ja) * 2004-05-18 2005-12-02 Nippon Steel Corp オンライン監視・解析画面作成装置、オンライン監視・解析画面作成方法、コンピュータプログラム及び記録媒体
WO2008143315A1 (fr) * 2007-05-22 2008-11-27 Nec San-Ei Instruments, Ltd. Dispositif d'enregistrement/de reproduction de données
JP2010072739A (ja) * 2008-09-16 2010-04-02 Shinano Kenshi Co Ltd データ収集装置

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5093807A (en) * 1987-12-23 1992-03-03 Texas Instruments Incorporated Video frame storage system
JP2895269B2 (ja) * 1991-07-31 1999-05-24 株式会社東芝 プラント監視システム
KR101411627B1 (ko) * 2006-10-24 2014-06-25 소니 주식회사 촬상장치와 재생제어장치
JP2009076084A (ja) * 2008-10-16 2009-04-09 Geo Technical Laboratory Co Ltd 画像生成支援装置、画像生成支援方法、コンピュータプログラム、および、記録媒体

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0579951A (ja) * 1991-09-18 1993-03-30 Hitachi Ltd 監視システム
JPH0620180A (ja) * 1992-06-30 1994-01-28 Hitachi Ltd 事象再現装置
JP2005188998A (ja) * 2003-12-24 2005-07-14 Toyota Motor Corp 運転行動解析装置
JP2005332069A (ja) * 2004-05-18 2005-12-02 Nippon Steel Corp オンライン監視・解析画面作成装置、オンライン監視・解析画面作成方法、コンピュータプログラム及び記録媒体
WO2008143315A1 (fr) * 2007-05-22 2008-11-27 Nec San-Ei Instruments, Ltd. Dispositif d'enregistrement/de reproduction de données
JP2010072739A (ja) * 2008-09-16 2010-04-02 Shinano Kenshi Co Ltd データ収集装置

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015060612A (ja) * 2013-09-19 2015-03-30 株式会社デンソー 情報処理装置
CN104707868A (zh) * 2013-12-11 2015-06-17 东芝三菱电机产业***株式会社 数据分析装置
JP6113370B1 (ja) * 2016-01-22 2017-04-12 三菱電機株式会社 記憶装置、制御装置、表示装置及び記憶方法
JP2020102130A (ja) * 2018-12-25 2020-07-02 東芝三菱電機産業システム株式会社 データ収集再生システム
JP7020392B2 (ja) 2018-12-25 2022-02-16 東芝三菱電機産業システム株式会社 データ収集再生システム

Also Published As

Publication number Publication date
TWI454141B (zh) 2014-09-21
JP5478776B2 (ja) 2014-04-23
JPWO2012140715A1 (ja) 2014-07-28
TW201242349A (en) 2012-10-16

Similar Documents

Publication Publication Date Title
JP5820055B2 (ja) データ同期再生装置、データ同期再生方法およびデータ同期制御プログラム
JP4685465B2 (ja) 監視記録装置
CN101267523B (zh) 处理运动图像的设备、成像设备、图像显示控制方法
JP5478776B2 (ja) データ同期再生装置、データ同期再生方法およびデータ同期制御プログラム
EP2227029A2 (fr) Appareil de traitement d'images numériques, procédé de traitement d'images numériques et programme
KR20190005188A (ko) 복수의 비디오 세그먼트로부터 합성 비디오 스트림을 생성하는 방법 및 장치
TWI635389B (zh) 工廠運轉狀態解析系統
JP2004236211A (ja) 画像処理システム
JP2007324663A (ja) ビデオカメラ
JPH11327634A (ja) 設備ロギング装置
JP3288715B2 (ja) 画像取り込み装置、及び画像取り込み装置を装着する画像記憶再生装置並びに画像入力装置
JP4594161B2 (ja) 映像処理装置及び映像処理方法
JP3261993B2 (ja) 動画像記録方法及び動画像記憶再生方法
JP2024065386A (ja) 動画像記録装置、動画像記録装置の制御方法、及びプログラム
JP3812508B2 (ja) 映像記録装置、映像変換装置及び映像編集装置
JP5859100B2 (ja) 画像記録装置
JP2011188193A (ja) 記録再生装置及び記録再生方法
JP2001186469A (ja) 画像間欠記録装置及びその記録方法
JP2010224995A (ja) 操作画面記録装置
JP2014086815A (ja) 電子機器
JP2005276383A (ja) 画像ファイル記録再生装置及び画像ファイル記録再生プログラム
JP2012227615A (ja) 記録再生装置
JP2007251310A (ja) 画像記録装置
JP2005223485A (ja) 画像処理装置、画像処理システム及び画像処理方法
JP2005045606A (ja) 画像処理方法、画像処理装置、プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11863591

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2013509678

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11863591

Country of ref document: EP

Kind code of ref document: A1