WO2017203789A1 - Difference image generation method, image restoration method, difference detection device, image restoration device, and monitoring method - Google Patents

Difference image generation method, image restoration method, difference detection device, image restoration device, and monitoring method Download PDF

Info

Publication number
WO2017203789A1
WO2017203789A1 PCT/JP2017/008397 JP2017008397W WO2017203789A1 WO 2017203789 A1 WO2017203789 A1 WO 2017203789A1 JP 2017008397 W JP2017008397 W JP 2017008397W WO 2017203789 A1 WO2017203789 A1 WO 2017203789A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
difference
images
camera
difference image
Prior art date
Application number
PCT/JP2017/008397
Other languages
French (fr)
Japanese (ja)
Inventor
昭弘 佐久間
Original Assignee
株式会社Ｎｅｘｐｏｉｎｔ
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社Ｎｅｘｐｏｉｎｔ filed Critical 株式会社Ｎｅｘｐｏｉｎｔ
Publication of WO2017203789A1 publication Critical patent/WO2017203789A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B25/00—Alarm systems in which the location of the alarm condition is signalled to a central station, e.g. fire or police telegraphic systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/91—Television signal processing therefor
    • H04N5/911—Television signal processing therefor for the suppression of noise
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • Embodiments include a monitoring method in a surveillance camera system for transferring and storing captured images and data to a server for crime prevention and management, a difference image creation method, an image transfer method, and difference detection suitable for image transfer
  • the present invention also relates to an image restoration method and an image restoration device for restoring transferred data to an image that can be displayed on a terminal.
  • the monitoring target points such as the entrances and exits of buildings such as apartment buildings, stores or streets, factories and distribution centers, etc.
  • the monitoring results are sent to the Internet and other electricity
  • a camera system for sending and monitoring to a monitor of a user terminal via a communication line has been introduced.
  • a camera system that captures a still image and compresses the image acquired by the server and then displays the image on a terminal has been developed (see, for example, Japanese Patent Laid-Open No. 2013-51534). Is known (for example, see Japanese Patent Application Laid-Open No. 2002-77882).
  • the difference image creation method is: Based on a first all image and a second all image acquired after the first all image, a first area in which the first all image and the second all image are not different is defined as a mask area. Creating a difference image of Creating a second difference image in which transparent color information is set for the mask area of the first difference image, except for the mask area of the first difference image, corresponding to the second whole image; including.
  • the server can compress the image, but the transfer from the monitoring camera to the server has a large transfer capacity, particularly in the case of moving images.
  • the first object of the embodiment described below is to solve the above-described problem and to reduce the transfer capacity to the recording server without reducing the number of transfer frames to the recording server and without reducing the resolution. It is an object of the present invention to provide a difference image creation method, an image transfer method using the same, a difference detection device, and a monitoring method in a monitoring camera system.
  • a second object of the embodiment is to provide an image restoration method and an image restoration apparatus that restores data transferred by such a method to an image that can be displayed on a terminal.
  • the difference image creation method is: Based on a first all image and a second all image acquired after the first all image, a first area in which the first all image and the second all image are not different is defined as a mask area. Creating a difference image of Creating a second difference image in which transparent color information is set for the mask area of the first difference image, except for the mask area of the first difference image, corresponding to the second whole image; including.
  • the number of transfer frames to the recording server is not reduced when transferring image data to the server.
  • the transfer capacity to the recording server can be reduced without reducing the resolution. From the viewpoint of transfer capacity, even if there is a limit to the acquisition of one frame per second for all two images, according to this aspect, the acquisition of three frames per second can be realized.
  • image is a word including both the meanings of a still image and a moving image, but in the present disclosure, a still image is displayed as “image”, and a moving image is displayed as “video” when particularly distinguished.
  • An image restoration method is a difference image restoration method created by the difference image creation method of the first aspect of the embodiment described above, Synthesizing the second difference image with the first full image.
  • an image with a high resolution can be restored, so that a high-resolution video can be reproduced.
  • the difference detection apparatus is connected to each camera of a camera group that captures a moving image and / or a camera group that captures a still image, and is connected to a recording server via a router and the Internet,
  • a difference detection apparatus having a folder for storing images, Acquire all images from the camera at regular time intervals or every time a request for all images from the recording server is received, store all images for all image transfer, and store all images for all image transfer in the folder
  • all image transfer means for transferring all images for all image transfer to the recording server, every time a predetermined image is received during the predetermined time period or every time a difference image request is received from the recording server, all images are transmitted from the camera.
  • a difference image all-image acquisition means for obtaining all images for difference image transfer, Extraction means for extracting all images stored in the folder;
  • First difference image creating means for creating a first difference image using a mask area as an area that is not different from all images for transfer of difference images acquired by the all image acquiring means;
  • the difference image is a difference image between the entire difference image transfer image acquired by the difference image all image acquisition unit and the first difference image, except for the mask area of the first difference image.
  • the transfer capacity of the recording server can be reduced without reducing the number of transfer frames of the recording server and without reducing the resolution. Enables reduction.
  • a monitoring method includes a camera group that captures a moving image and / or a camera group that captures a still image, and a difference detection device that includes a folder that is connected to the camera of the camera group and stores an image.
  • a monitoring server system having a recording server connected to the difference detection device via a router and the Internet, The difference detection device obtains all images from the camera every predetermined time or every time a request for all images from the recording server is received, and sets all images for all image transfer to the folder. And a full image transfer step of transferring all the images for full image transfer to the recording server.
  • the difference detection device acquires all images from the camera and sets all images for difference image transfer as a difference image all image acquisition step; An extraction step in which the difference detection device extracts all images stored in the folder; The difference detection device is a difference image between all images extracted in the extraction step and all images for transfer of difference images acquired in the difference image all-image acquisition step, and extracted in the extraction step.
  • the difference detection device is a difference image between the first difference image and the entire difference image transfer image acquired in the difference image all image acquisition and extraction step, and a mask area of the first difference image Except for the second differential image in which transparent color information is set for the mask area of the first differential image, which is a corresponding portion of the total image for differential image transfer acquired in the differential image all-image acquisition step.
  • the difference detection apparatus replaces all images stored in the folder with all images for differential image transfer acquired in the difference image all image acquisition step, Including repeating each step of And,
  • the recording server includes a difference image conversion step in which the difference image acquired from the difference detection device is combined with all the images at the previous time point and stored as all images.
  • the transfer capacity to the recording server without reducing the number of frames transferred to the recording server and without reducing the resolution.
  • a video camera can be installed to acquire data via the Internet network.
  • the original image is combined with an image having a small data amount and an image before the difference image. Since the differential data that can be restored can be created and transferred, the transfer capacity flowing through the Internet network can be reduced.
  • a monitoring method is the monitoring method according to the fifth aspect, In the all image transfer step and the difference image all image acquisition step, after acquiring all images from the camera, the entire image stored in the folder is compared with the color configuration of the image, and noise due to a change in light amount is removed. Then save.
  • the fifth aspect of the embodiment it is possible to reduce the amount of difference due to the change in the color tone of the image caused by the influence of sunlight in the outdoors or the influence of lighting on / off of the indoor lighting. Even with a surveillance camera installed in a place where the image changes little by little, the amount of transfer can be reduced because the difference can be reduced.
  • the “network” includes a network constituted by a communication network such as a LAN, the Internet, a Wi-Fi line, a 3G / LTE line, a dedicated line, or a combination thereof.
  • the camera group is either or both of a camera group that shoots a still image and a camera group that shoots a moving image, and the image to be captured by the camera group may be either or both of a still image and a moving image. Good.
  • Each camera has an output function, a still image shooting function, and / or a moving image shooting function.
  • the camera is preferably an IP camera, but can also be an analog camera.
  • the compression format for moving images is H.P.
  • the camera that shoots a moving image in the camera group does not need to be connected to the recording server via a LAN network, and for monitoring a moving image connected to the Internet via a 3G / LTE line. Even with a camera, data that can be restored and reproduced with a high-resolution video can be transferred to the recording server, and the range in which a video surveillance camera can be placed is widened.
  • the recording server may be a cloud server.
  • the transfer capacity of the recording server can be reduced without reducing the number of transfer frames of the recording server and without reducing the resolution.
  • the difference image creation method of the first embodiment is such that the monitoring camera is operated on the recording server via the network, and the recording server captures, collects, edits, and stores the camera images of moving images and still images.
  • This is a difference image creation method used for a monitoring method in a monitoring camera system that transmits an image to be monitored to a terminal of the terminal via a network in response to a request from the terminal.
  • the difference detection apparatus of the first embodiment used for the difference image creating method of the first embodiment, the image transfer method of the monitoring camera of the first embodiment in the monitoring camera system, and the image restoration method of the first embodiment are also combined. explain.
  • the monitoring camera system transmits an image restored by the image restoration method of the first embodiment to a terminal or the like, and performs reproduction / rewinding / fast-forwarding of live images and library reproduction.
  • the surveillance camera system compresses and stores the restored image, transmits the image to a terminal or the like, and performs playback / rewinding / fast-forwarding of past images.
  • the transfer capacity of still images can be reduced by transferring the still images from the camera to the recording server, and still images can be collected on a cloud server or the like without installing a recording server on site.
  • the number of frames can be increased or the resolution can be increased as compared with simple still image collection.
  • library playback refers to playback by rewinding from a live image to the past or fast-forward playback of a past image.
  • FIG. 1 is a diagram illustrating an example of a system configuration of a monitoring camera system that uses a first example of the differential image creation method of the embodiment.
  • a network 500 configured by a combination of communication networks such as a LAN, the Internet, Wi-Fi lines, 3G / LTE lines, and dedicated lines, (1) a plurality of surveillance cameras (moving images) 700A-C and multiple monitoring cameras (for still images) 701A-C, (2) Connect to each of monitoring cameras (for moving images) 700B and C / monitoring cameras (for still images) 701B, C
  • the difference detection devices 400A to 400D that acquire images taken by the monitoring camera and transfer them to the recording server 100, and (3) are connected to the difference detection device 400 via the routers 600A to 600D and the Internet to detect the difference.
  • Recording server 100 that acquires image data from the apparatus 400, stores it, and transmits it to the terminal.
  • the mobile terminal 200A such as smartphones, and B, viewing PC 300A ⁇ C, such as a desktop PC (personal computer) or a notebook PC is connected.
  • the recording server 100 is also connected to a monitoring camera (for moving images) 700A and a monitoring camera (for still images) 701A via a LAN.
  • the difference image creation method of this embodiment is different from the recording server 100 and the LAN.
  • the cameras described below include cameras connected to the recording server 100 on the LAN (the monitoring camera (for moving image) 700A and the monitoring camera (still image in FIG. 1)). For example) 701A).
  • the flow of a moving image (stream moving image) taken by a monitoring camera (for moving image) is indicated by a dotted arrow.
  • terminal or “Viewer”.
  • the recording server 100 is connected to the monitoring cameras (for moving images) 700B and C and the monitoring cameras (for still images) 701B and C via a network 500 including the Internet.
  • a network 500 including the Internet In the present embodiment, not only the monitoring camera for shooting a still image but also a monitoring camera for shooting a moving image can be provided on a network other than the same LAN as the recording server 100.
  • the difference detection devices 400D, C, B, and A are connected to the monitoring cameras (for moving images) 700B and C and the monitoring cameras (for still images) 701B and C, respectively.
  • a 3G / LTE line is connected to the LAN connected to the recording server 100 via the router 600A and another LAN is connected via the router 600C.
  • the difference detection device 400D is connected to the Internet network via a router 600D and a 3G / LTE line
  • the difference detection device 400B is connected to the Internet network via a router 600B and a 3G / LTE line.
  • the difference detection apparatuses 400 ⁇ / b> A and 400 ⁇ / b> C are connected to a different LAN from the recording server 100.
  • FIG. 1 shows a connection example in which a monitoring camera (for moving images), a monitoring camera (for still images), a viewing PC, and a mobile terminal are connected to a recording server via a plurality of networks. Not.
  • the communication between the monitoring camera (for still image) 401 and the recording server 100 and the communication between the monitoring camera (for moving image) 400 and the recording server 100 are reliable after the session is established.
  • TCP having high characteristics
  • UDP that transmits data without establishing a session and that is fast and excellent in real time may be used, and an image can be acquired by using both TCP and UDP protocols.
  • the monitoring camera (for moving images) and the monitoring camera (for still images) are IP cameras in this embodiment, but may be analog cameras. However, in the case of analog, a converter is required.
  • the viewing PC 300 that is an image display terminal is not limited to being in the same LAN as the recording server 100, and may be connected via a plurality of networks such as the Internet, a LAN, or a Wi-Fi line.
  • the mobile terminal 200 which is an image display terminal, may also be connected via a plurality of networks such as a LAN, the Internet, and a cellular phone network (Wi-Fi line, 3G / LTE line, etc.).
  • the router 600 is connected between the LAN and the Internet, and between the monitoring camera (for moving images) and the monitoring camera (for still images) and the Internet.
  • One recording server 100 can acquire a camera moving image from a plurality of monitoring cameras (for moving images) 700 and a camera still image from a plurality of monitoring cameras (for still images) 701.
  • the recording server 100 automatically distributes network traffic, network design is not required, and it is only necessary to connect the camera to the port, so that the operation becomes very simple. In addition, resuming is easy even when communication is interrupted.
  • a method may be used in which an image is transferred from the difference detection device to the recording server without accessing the difference detection device from the recording server.
  • a timer in the difference detection device acquires an image at a timing such as 1 second and transfers it to the recording server. After the transfer is normally completed, the sleep is set at a setting time such as 1 second. To do.
  • the burden on the network becomes excessive.
  • the number of frames transferred to the recording server is reduced. Since the transfer capacity to the recording server can be reduced without dropping the resolution and without reducing the resolution, a video surveillance camera may be provided on the Internet or on a 3G / LTE line such as a portable terminal. Because it can, the degree of freedom becomes very high.
  • FIG. 2 is a configuration diagram of the recording server of the first example of the differential image creating method of the embodiment.
  • the recording server 100 includes a CPU 101 with a memory 102 that is a cache memory, a control / arithmetic unit having a device driver, a storage device 110 having a main storage device such as a DRAM and an auxiliary storage device such as a hard disk, a network interface 104, and the like. And an input / output device including a display 103 as a display device, a keyboard 105, a mouse 106, and the like.
  • the storage device 110 stores an operating system 114 in addition to a primary image folder 111, a secondary video folder, a program 113, an authentication database, an environment setting folder, and the like.
  • the program 113 is normally stored in the auxiliary storage device of the storage device 110, and is loaded into the main storage device at the time of execution.
  • the primary image folder 111 a moving image including only all images acquired from the difference detection device 400 and all images created from the difference images is accumulated as primary data.
  • the primary data stored in the primary image folder 111 is converted into moving image compression data and stored as secondary data.
  • the program 113 includes various programs such as a photographing / collecting / editing program and a transmission program.
  • IDs, passwords, port numbers and IP addresses of the monitoring cameras 700 and 701, the mobile terminal 200, and the viewing PC 300, and individual identification information (UID) are stored in terminals having no IP address.
  • the recording server 100 is integrated with the recording server and the terminal, and also serves as a terminal for displaying images by itself, and for maintenance and management, the display 103 and a keyboard as an input means. 105 and a mouse 106.
  • the terminal function as the image display device may not be provided.
  • the environment setting folder stores image acquisition timing from each difference detection device, primary data and secondary data creation intervals, compression conditions, and the like.
  • the recording server 100 is a stationary server, but may be a cloud server.
  • the recording server 100 causes the monitoring camera (for still image) 701 connected via the Internet to obtain an image as a still image by requesting an image from the difference detection device 400, so that the difference for each monitoring camera is obtained.
  • the entire image is left as it is or converted into a difference image and transmitted to the recording server 100.
  • the recording server 100 causes the monitoring camera (moving image) 700 connected via the Internet to request an image from the difference detecting device 400 to acquire an image with a moving image, and the difference detecting device for each monitoring camera.
  • the whole image is converted into a differential image or transmitted to the recording server 100.
  • the CPU 101 loads the image capturing / collecting / editing program to the memory 102 and executes the program to acquire and edit the image from the difference detection apparatus in the monitoring method using the difference image creating method of the present embodiment.
  • the recording server 100 realizes the function of a computer capable of sending an image to a terminal by causing the CPU 101 to load and execute a transmission program in the memory 102.
  • the CPU 101 is an arithmetic processing device mounted on a normal computer, executes various programs, and performs various controls.
  • the recording server 100 may be a single server or a server group including a plurality of recording servers.
  • the storage destination may be a secondary video folder provided in a recording server separate from the recording server that acquires the camera image. .
  • the shooting / collection / editing program records (1) a difference detection device connection function for connecting to the difference detection device, and (2) records an image on the connected difference detection device every predetermined time (for example, 1 second).
  • An image acquisition function to be inputted to the server (3) a difference image determination function for determining whether an image acquired from the difference detection device is a difference image, an all image storage function for storing all images acquired from the difference detection device, and (4) a difference A difference image converter that combines the difference image acquired from the detection device with all the images at the previous time and stores it as an entire image.
  • a program for realizing a primary data creation function for accumulating all acquired or synthesized images as primary data in a primary image folder.
  • the photographing / collecting / editing program further compresses the data in a moving image format by combining the primary data with a computer every predetermined time (for example, 10 minutes) and attaching a time stamp.
  • This is a program for realizing a secondary data creation function for converting into secondary data.
  • the primary data is all images (I frame (intra frame) when the camera image is a moving image), and the secondary data is compressed into a moving image format even when the camera image is a still image, and the camera image is a moving image.
  • the ratio of one I frame is 200 frames, and not only P frames (predicted interframes) but also B frames (bidirectional predictive interframes) are inserted in combination to reduce capacity without deterioration.
  • a future I frame can be referred to, so that a B frame can be inserted. Since the B frame is a front-back comparison difference, it is smaller than the P frame.
  • the transmission program has a terminal connection function for connecting to a terminal on a computer, an image selection receiving function for displaying a camera list on the terminal and receiving a camera selection from the terminal, and an image of primary data for the selected camera as a live image Live image transmission function to be transmitted to the terminal, live image rewind reception function to receive a rewind request from the live image from the terminal, and for each rewind request, a certain amount of time (for example, 1 second) from the image previously transmitted to the terminal Rewind live image transmission function that extracts primary data of the past time point and transmits it to the terminal as a rewind live image, rewind live image fast forward reception function that accepts a fast forward request from the rewind live image from the terminal, for each fast forward request Until the current image is reached, a certain time (for example, 1 second) from the previous image sent to the terminal.
  • a certain amount of time for example, 1 second
  • Data is a program for also realizing the live image transmission function feeding early to be transmitted to the terminal as the extraction to fast forward a live image.
  • “Current image” means the latest image acquired from the difference detection device. Therefore, the current image changes to a new image each time an image from the difference detection apparatus is input to the recording server.
  • the transmission program further includes a library image reception function for accepting a library image request from a rewind live image from the terminal to the computer, and for each library image request, a certain amount of time from the image previously transmitted to the terminal, in detail, a difference detection
  • This is a program for realizing a library image transmission function that extracts primary data at a future time for an image acquisition interval from the apparatus 400 to the recording server 100 and transmits it to the terminal as a fast-forward live image.
  • the “library image” here refers to a fast-forward playback image of a past image.
  • connection between the terminals 200 and 300 and the recording server 100 is also performed by the TCP / IP method, authentication is performed with the user ID and password, and it is confirmed that the terminal is a terminal registered in the recording server. Then send the image.
  • the authentication is preferably performed by an authentication database on the recording server.
  • the terminal Even if the terminal is not on the same LAN, if the recording server is connected to the network via a router, an image (all images or compressed images) is sent from the recording server using the IP address of the connected router and the port number assigned to the router. Acquire. For terminal connection, the terminal is authenticated using the UID registered at the time of terminal authentication.
  • the recording server 100 Since the recording server 100 identifies the terminal based on the terminal unique information using the UID at the start of connection of the mobile terminal 200, the recording server 100 can display an image by authenticating the user ID and the password and matching the terminal unique information.
  • FIG. 3 is a configuration diagram of the difference detection apparatus according to the first example of the difference image creation method of the embodiment.
  • the difference detection device 400 includes a control / arithmetic unit having a CPU 401 and a device driver with a memory 402, a storage device 410 having a main storage device such as a DRAM and an auxiliary storage device such as a hard disk, and a communication control for a network interface 404 and the like. And an input / output device including a display 403, a keyboard 405, a mouse 406, and the like, which are connected and used only during maintenance.
  • the storage device 410 stores an accumulation folder 411, an environment setting folder, a difference detection program 413, and an operating system 414.
  • the accumulation folder 411 is a folder for storing images.
  • the accumulation folder 411 includes all images of the camera images acquired from the monitoring camera (for moving image) 700 and the monitoring camera (for still image) 701, and the first.
  • the first difference image having a mask area as an area where the stored all images and the next stored difference image transfer image are not different from each other, and the difference image transfer all except for the mask area of the first difference image
  • the second difference image in which transparent color information is set is stored.
  • the environment setting folder stores noise removal conditions, conditions for determining whether a camera image for storing all images or a camera image for storing difference images, and the like.
  • the difference detection device 400 is connected to a monitoring camera (for moving images) 700 and a monitoring camera (for still images) 701 that are connected to the recording server 100 via the Internet one by one.
  • the difference detection apparatus 400 is connected to the monitoring camera (for moving image) 700 or the monitoring camera (for still image) 701 to which the difference detection apparatus 400 is connected and the camera image by the recording server 100.
  • the request is controlled, and the camera image acquired from the monitoring camera (for moving image) 700 and the monitoring camera (for still image) 701 is converted into a predetermined image and transmitted to the recording server 100.
  • the CPU 401 loads and executes the difference detection program 413 into the memory 402, thereby executing camera image acquisition, editing and storage in the monitoring method using the difference image creation method of the present embodiment, and the recording server Realize computer functions that can be transmitted.
  • the CPU 401 is an arithmetic processing device mounted on a normal computer, executes various programs, and performs various controls.
  • the difference detection program 413 has (1) a camera connection function for connecting to the monitoring camera every time a connection request to the monitoring camera is received from the recording server, and (2) all images from the recording server. Every time a request is received, all images are acquired from the connected monitoring camera (for moving images) or monitoring camera (for still images), and all images are transferred for transfer to the storage folder. All image transfer function for saving all images and transferring all images for all image transfer to the recording server.
  • the connected monitoring camera for moving images
  • a difference image all image acquisition function for acquiring all images from a monitoring camera (for still images) and making all images for difference image transfer
  • an extraction function for extracting all images stored in a storage folder
  • a difference image between all images extracted by the extraction function and all images for transfer of difference images acquired by the all-image acquisition function for difference images, and all images extracted by the extraction function and all images for difference images are acquired.
  • a first difference image creation function for creating a first difference image using a mask area as an area that is not different from the difference image transfer all images acquired by the function; (6) acquired by a difference image all image acquisition function; The difference image is a difference image between the entire difference image transfer image and the first difference image, except for the mask area of the first difference image.
  • a second difference image creation function for creating a second difference image in which transparent color information is set for the mask area of the first difference image, and (7) a second difference to the recording server.
  • image Difference image transfer function for transferring, (8) All image replacement function for replacing all images stored in the storage folder with all images for difference image transfer acquired by the all image acquisition function for difference images It is a program.
  • the connected monitoring camera for moving images
  • monitoring is performed every certain time (for example, 1 second). All images may be acquired from a camera (for still images).
  • the difference image all-image acquisition function is further stored in the storage folder after all images are acquired from the monitoring camera (for moving images) or the monitoring camera (for still images). It includes a function that compares the color configuration of all images and the image, and removes noise caused by changes in the amount of light before saving.
  • the connection between the monitoring camera and the recording server via the difference detection device is performed by the TCP / IP method, and in order to identify the monitoring camera, authentication is performed with the user ID and password set on the recording server side. After that, a request for taking a camera image is made.
  • the authentication is preferably performed by an authentication database on the recording server.
  • an image is acquired from the difference detection device using the IP address of the router to be connected and the port number assigned to the router. Do.
  • an image is transferred from the difference detection device to the recording server. At this time, the port number of the router on the recording server side is released and the port is transferred to the recording server.
  • the recording server 100 can specify a connection partner by connecting using an IP address and a port number when starting a monitoring camera connection in a local environment (for example, on the same LAN), and authenticates with a user ID and a password.
  • the recording server 100 uses a global IP address and port number by using router port forwarding (also expressed as port forwarding, etc.) when connecting to a surveillance camera in a remote environment such as via the Internet network. Identify the connection partner.
  • FIG. 4 is a configuration diagram of the terminal (mobile) of the first example of the differential image creating method of the embodiment.
  • the mobile terminal 200 includes a control / arithmetic apparatus having a CPU 201 with a memory 202, a device driver, and the like, a storage device 210, a communication control device for transmitting and receiving data, a display 203 as a display device, operation buttons, a touch panel and the like. An output device is provided.
  • the storage device 210 stores an image display program 213 and an operating system 214.
  • the mobile terminal 200 is, for example, a mobile phone such as a smartphone, and the CPU 201 loads the image display program 213 into the memory 202 and executes it, thereby realizing a computer function capable of displaying an image according to the embodiment.
  • the CPU 201 is an arithmetic processing device mounted on a normal mobile terminal, executes various programs, and performs various controls.
  • FIG. 5 is a configuration diagram of the terminal (PC) of the first example of the differential image creating method of the embodiment.
  • the viewing PC 300 includes a control / arithmetic unit having a CPU 301 and a device driver with a memory 302, a storage device 310 having a main storage device such as a DRAM and an auxiliary storage device such as a hard disk, a communication control device such as a network interface 304, And an input / output device including a display 303 as a display device, a keyboard 305, a mouse 306, and the like.
  • the storage device 310 stores an image display program 313 and an operating system 314.
  • the browsing PC 300 is, for example, a desktop PC, a notebook PC, a tablet terminal, or the like.
  • the CPU 301 loads the image display program 313 into the memory 302 and executes it, thereby realizing a computer function capable of displaying an image according to the embodiment.
  • the CPU 301 is an arithmetic processing device mounted on a normal PC, executes various programs, and performs various controls.
  • the image display program is a program for realizing a terminal connection function for connecting to a recording server and an image display function for displaying an image transmitted from the recording server on a computer.
  • the image display program further includes a live image request function for requesting a live image to the recording server at every camera image acquisition interval of the recording server from the monitoring camera during live image display on the computer.
  • a rewind start request function for accepting an input of a rewind request during image display and requesting a rewind live image from the recording server, and a fixed time (for example, 0. 0) shorter than the live image request interval during rewind live image display. Every 2 seconds), a rewind continuation request function for requesting a rewind live image to the recording server, while receiving a rewind live image, accepts a fast forward request input from the user, and requests a rewind live image to the recording server.
  • a fixed time shorter than the live image request interval (for example, 0.2 seconds) Every is a program for also realizing the rewind live image fast-forward continuation request function for requesting fast forward a live image to the recording server.
  • the image display program further accepts an input of a library image request while displaying a rewind live image on the computer, and requests a library image from the recording server.
  • This is a program for realizing a library image continuation request function for requesting a library image to the recording server at every camera image acquisition interval of the recording server from the camera during display of the library image.
  • the “library image” here is a replay image obtained by rewinding the live image to the past or a fast-forward replay image of the past image.
  • the recording server 100, the viewing PC 300, and the difference detection device 400 are all configured as personal computers, and have a clock function or the like that a normal personal computer has.
  • the mobile terminal 200 also has a clock function and the like.
  • the difference detection apparatus 400 includes a camera group that shoots a moving image and a camera group that shoots a still image (monitoring camera (for moving image) 700, monitoring camera). (For still image) 701), connected to the recording server 100 via the router 600 and the network 500 including the Internet, and having a storage folder 411 for storing images, and (1) recording Each time a connection request with the monitoring camera (for moving image) 700 or the monitoring camera (for still image) 701 is received from the server 100, the monitoring camera (for moving image) 700 or the monitoring camera (for still image) 701 is received. (2) for monitoring for which the difference detection device 400 is connected every time when a request for all images from the recording server 100 is received.
  • All images are acquired from the camera (for moving images) 700 or the monitoring camera (for still images) 701, are transferred to all images, and are stored in the storage folder 411.
  • a difference image all-image acquisition unit that acquires all images from the monitoring camera (moving image) 700 or the monitoring camera (still image) 701 and sets all images for difference image transfer;
  • storage folder 411 (5) a difference image between the entire image extracted by the extracting unit and the entire image for differential image transfer acquired by the differential image all-image acquiring unit.
  • Extraction First difference image creation for creating a first difference image using a mask area as an area where all the images extracted by the means and all the images for difference image transfer obtained by the difference image all-image obtaining means are not different
  • (6) a difference image between the entire difference image transfer image acquired by the difference image all image acquisition unit and the first difference image, except for the mask area of the first difference image.
  • a second difference that creates a second difference image in which transparent color information is set for the mask area of the first difference image, as a corresponding portion of all images for difference image transfer acquired by the image all-image acquisition means Image creation means, (7) difference image transfer means for transferring the second difference image to the recording server 100, and (8) difference obtained by the difference image all image acquisition means for all images stored in the storage folder 411.
  • the difference image all-image acquisition unit further acquires all images from the monitoring camera (for moving image) 700 or the monitoring camera (for still image) 701 and then stores them in the accumulation folder 411. Compare all the images and the color composition of the images, remove the noise due to the change in the amount of light, and save.
  • the difference detection device 400 functions as means (1) to (8) by the hardware configuration and the difference detection program 413 described above.
  • the recording server 100 includes (1) a difference detection device connection means for connecting to the difference detection device, and (2) a connected difference detection device every fixed time (for example, 1 second).
  • Image acquisition means for inputting an image to a recording server, (3) difference image determination means for determining whether the image acquired from the difference detection apparatus is a difference image, all image storage means for storing all images acquired from the difference detection apparatus, 4) Difference image conversion means for combining the difference image acquired from the difference detection device with all the images at the previous time and storing it as all images, (5) The acquired or combined all images in the memory until the next image generation Image holding means for holding; (6) primary data creating means for storing all acquired or synthesized images as primary data in a primary image folder; and (7) binding primary data every predetermined time (for example, 10 minutes). Is allowed is provided with a secondary data generating means for converting the secondary data compressed video format time stamped.
  • the recording server 100 functions as means (1) to (7) according to the hardware configuration and the shooting / collection / editing program described above.
  • the recording server 100 functions as an image transmission unit such as a live image, a library image, and rewinding or fast-forwarding of these images to the terminal by the above-described hardware configuration and transmission program.
  • the “library image” here refers to a playback image that is rewound from the live image to the past or a fast-forward playback image of the past image.
  • the terminal 200, 300 is (1) live playback requesting means for requesting the recording server 100 to perform live playback of camera moving images for one or more monitoring cameras 700 with the authority to monitor, and (2) transmitted from the recording server 100.
  • Live reproduction display means for displaying primary data (3) past video reproduction request means for requesting the recording server 100 to reproduce past camera moving images for one or a plurality of monitoring cameras 700 having the authority to monitor, and (4) recording.
  • Past video reproduction display means for displaying secondary data transmitted from the server 100 is provided.
  • the terminal functions as means (1) to (4) by the hardware configuration and the image display programs 213 and 313 described above.
  • the terminal Since the primary data created by compressing the camera image data is converted into secondary data by the recording server at regular time intervals, the terminal is not limited to rewinding live images or fast-forwarding after rewinding. It is also possible to reproduce, rewind, and fast forward past camera images, which is possible with a conventional recording / playback apparatus. Although the image quality is high, the data amount is small, and the rewind live composite image and fast-forward live composite image use primary data compressed for primary storage, so the data amount is small while high image quality. Since the secondary data, which is a moving image file that has been compressed and converted, is used for playback and fast forward rewinding more than the secondary data creation unit that is not frequently used, the amount of data required for storage can be reduced while moving images. Further, it becomes possible to reproduce the live composite image after rewinding.
  • the above-described monitoring camera system including the monitoring camera, the difference detection device, the recording server, and the terminal includes the difference image creation method of the first embodiment and the image of the first embodiment using the difference image creation method.
  • a surveillance camera system that realizes a transfer method and an image restoration method according to the first embodiment, which is a restoration method of an image transferred by the image transfer method. Even if a video camera is installed in addition to the environment, problems such as data delays are unlikely to occur, and the amount of data transmitted from the recording server to the terminal is small. it can. In other words, the transfer capacity to the recording server can be reduced without reducing the number of transfer frames of the recording server and without reducing the resolution.
  • a camera image can be sent to the recording server through a general Internet line, and network camera information can also be sent to a general Internet line.
  • data can be stored in a cloud server over the Internet line (WAN).
  • the difference image creation procedure of the first example of the difference image creation method of the embodiment will be described.
  • the monitoring camera system including the above-described monitoring camera, difference detection device, recording server, and terminal is used.
  • FIG. 6 is an image diagram of data transmission from the monitoring camera to the recording server in the first example of the differential image creating method of the embodiment.
  • Both a stream IP camera that is a monitoring camera (for moving images) and a still image IP camera that is a monitoring camera (for still images) are connected to a difference detection device via a network HUB.
  • the server machine that is connected to the Internet network via the HUB and is a recording server is connected to the Internet network.
  • An all-image request or a difference image request is transmitted from the recording server to the difference detection device, and the difference detection device sends a monitoring camera (for video) or a monitoring camera (for still image) based on the request received from the recording server.
  • Request an image.
  • the monitoring camera (for moving image) transmits a stream moving image to the difference detection device
  • the monitoring camera (for still image) transmits a still image to the difference detection device.
  • the stream moving image and still image transmitted to the difference detection device are edited into all images or difference images that are still images and transmitted to the recording server in the difference detection device. Therefore, only a still image flows through the Internet network, and the data amount of the difference image is smaller than that of all images, so that the load on the network can be reduced.
  • FIG. 7 is an explanatory diagram of an image restoration method from the difference image created by the difference extraction method in the example of the difference image creation method of the embodiment and the example of the difference image creation method of the embodiment.
  • a first all image (“A image” in FIG. 7) and a second all image (FIG. 7) acquired successively after the first all image.
  • the first image having a mask area that is a difference image between the first all image and the second all image and that is not different between the first all image and the second all image.
  • a difference image (“difference between A image and B image” in FIG. 7)
  • (2) a difference image between the second whole image and the first difference image, Except for the mask area, the second differential image in which transparent color information is set for the mask area of the first difference image is used as the corresponding portion of the second entire image (“extract difference generation area from B image” in FIG. 7). “Difference image”).
  • the mask area is a blackened portion.
  • the blacked out area is an area in which transparent color information is set by converting into a format that can handle transparent colors as an ⁇ channel.
  • the image capacity can be reduced by making the difference image into a format capable of holding the ⁇ channel and efficiently compressing the capacity.
  • the difference image creation method of the present embodiment is extremely effective in reducing the transfer capacity because the difference is small particularly in a surveillance camera system installed at a fixed point.
  • the image transfer method of the surveillance camera including the difference image creation method of the present embodiment includes a camera group for capturing a moving image, a camera group for capturing a still image, and each camera of the camera group (monitoring camera (for moving image) / monitoring)
  • An image transfer method in a surveillance camera system having a difference detection device connected to a camera (for still images) and having a folder for storing images, and a recording server connected to the difference detection device via a router and the Internet. (1) Every time when a request for all images from the recording server is received, the difference detection device acquires all images from the camera, sets all images for all image transfer, and transfers all images to the storage folder.
  • the difference image from all the images of the first image is a mask area that is an area in which the entire image extracted in the extraction step and the entire image for differential image transfer acquired in the differential image all-image acquisition and extraction step are not different.
  • Difference image In addition, the areas other than the mask area of the first difference image are the corresponding parts of all the images for transferring the difference image acquired in the difference image all-image acquisition step, and the mask area of the first difference image is transparent in color.
  • Steps (2) to (7) of the all-image replacement step in which the detection device replaces all images stored in the storage folder with all images for differential image transfer acquired in the difference image all-image acquisition step.
  • the recording server stores all images acquired from the difference detection device, and (9) the difference image acquired from the difference detection device by the recording server. It includes a difference image conversion step of combining the entire previous image and storing it as an entire image.
  • the difference detection device and / or recording server acquires all images from the camera and then saves them in the accumulation folder. All the images and the color configuration of the image are compared, and the difference amount due to the change in the amount of light is removed and stored.
  • the removal of the difference amount is performed in order to reduce the difference in white balance between the first all images and the second all images, and the timing when the color configuration of the image is largely changed (outdoor daytime, Adjustment is performed to reduce the difference in white balance using several still images (outdoor night, indoor fluorescent lamp, no indoor lighting, etc.).
  • the surveillance camera system program having the photographing / collecting / editing program and the difference detection program causes the computer to execute these steps. This will be described more specifically with reference to the drawings.
  • FIG. 8 is a flowchart showing an outline of the procedure from connection of the monitoring camera (for still image) and recording server to writing of primary image data in the first example of the differential image creating method of the embodiment.
  • the recording server requests connection to the difference detection device by the TCP / IP method.
  • the difference detection apparatus issues a connection request to the monitoring camera (for still image).
  • One or a plurality of monitoring cameras (for still images) are configured, and each camera and recording server authenticate with a user ID and a password via a difference detection device.
  • the difference detection apparatus receives a connection response from each camera and successfully authenticates with reference to the authentication database of the recording server, the connection state is established.
  • the recording server makes an image request to the difference detection device.
  • the difference detection device requests (image request) the monitoring camera (for still image) to take a still image and transmit the captured camera image to the difference detection device in JPEG format.
  • the surveillance camera (for still image) receives an image request, shoots a snapshot (still image), and uses the TCP / IP method to detect a still image obtained by shooting as a camera image in JPEG format. Send to.
  • the video recording server side since the video recording server side performs video compression conversion to secondary data at regular intervals, the camera that captures a still image does not need to have a video format function.
  • the difference detection device When receiving the camera image of the still image, the difference detection device performs noise removal as described above, and the image (one image) transmitted from the monitoring camera (for still image) after the all-image request is transmitted from the recording server. Eyes).
  • the difference detection device stores all images for all image transfer, stores all images for all image transfer in the storage folder, and transfers all images for all image transfer to the recording server. Transfer all images.
  • the difference detection device stores and stores all images for the difference image transfer.
  • the second difference image in which the transparent color information is set is created as described above, and the difference image transfer is performed to transfer the second difference image to the recording server.
  • the difference detection device replaces all images stored in the accumulation folder with the above-described all images for differential image transfer (second all images).
  • the difference detection apparatus repeats these steps every time an image request is transmitted from the recording server.
  • the recording server When the recording server obtains an image from the difference detection device, it performs image restoration. First, the recording server determines whether or not the acquired image is a difference image. If the acquired image is a difference image, the recording server generates a whole image by combining all the images at the previous time point. In the case of all images, the image is left as it is. In either case, the image is developed in the memory and held until the next image generation. Then, the recording server accumulates the held image as primary data in the primary image folder. Thereafter, the recording server compresses and converts the primary data into a moving image and stores it in the secondary video folder.
  • FIG. 9 is a flowchart showing an outline of the procedure from the connection of the camera (moving image shooting) and the recording server to the writing of the primary image data in the first example of the differential image creating method of the embodiment.
  • the recording server requests connection to the difference detection device by the TCP / IP method.
  • the difference detection apparatus issues a connection request to the monitoring camera (for moving image).
  • the monitoring camera for moving image
  • One or a plurality of monitoring cameras (for moving images) are configured, and each camera and the recording server authenticate with a user ID and a password via a difference detection device.
  • the difference detection device receives a connection response from each camera, the difference detection device requests the monitoring camera (for moving image) to send a camera image of a streamed moving image taken to the difference detection device.
  • Image request the difference detection device requests the monitoring camera (for moving image) to send a camera image of a streamed moving image taken to the difference detection device.
  • Image request
  • the surveillance camera receives image requests, continuously shoots moving images, and shoots the moving images in a stream format.
  • H.264 baseline profile moving image as a camera image is transmitted to the difference detection device.
  • the difference detection device receives the camera image of the moving image, the difference detection device transmits a tangential response to the recording server, and enters the connection state when the authentication is successful with reference to the authentication database of the recording server.
  • the recording server makes an image request to the difference detection device.
  • the difference detection device extracts still images from the received camera image of the moving image at predetermined intervals, and removes noise.
  • the difference detection apparatus saves the first still image extracted from the camera image of the received moving image as all images for all image transfer, stores them as all images for all image transfer in the storage folder, and stores them in the recording server. All image transfer for transferring all images for all image transfer is performed.
  • the difference detection device sets the transparent color information as described above based on the difference from all images stored in the storage folder as all images for difference image transfer.
  • a second difference image is created, and difference image transfer is performed to transfer the second difference image to the recording server.
  • the difference detection device replaces all images stored in the accumulation folder with the above-described all images for differential image transfer (second all images).
  • the difference detection apparatus repeats these steps every time an image request is transmitted from the recording server.
  • the recording server When the recording server obtains an image from the difference detection device, it performs image restoration. First, the recording server determines whether or not the acquired image is a difference image. If the acquired image is a difference image, the recording server generates a whole image by combining all the images at the previous time point. In the case of all images, the image is left as it is. In either case, the image is developed in the memory and held until the next image generation. Then, the recording server accumulates the held image as primary data in the primary image folder. Thereafter, the recording server compresses and converts the primary data into a moving image and stores it in the secondary video folder.
  • FIG. 10 is an image diagram showing an image conversion method in the first example of the difference image creation method of the embodiment.
  • FIG. 10 shows a state in which all images (I frames) and difference images (W frames) created by the difference detection device based on images acquired from the monitoring camera (for moving images) are transmitted to the recording server.
  • the frame is a difference image including a transparent color.
  • the primary data stored in the primary image folder of the recording server is all I frames, and the secondary data stored in the secondary video folder is from the primary data to the ratio of one I frame to 200 frames.
  • This is a compressed moving image file including B frames and P frames.
  • This embodiment is a differential image creation method that can be used in a surveillance camera system for crime prevention, management, etc., and captured images and data are transferred to a server and stored.
  • the data transferred to the recording server by the image transfer method using this is restored to an image that can be displayed on the terminal by the image restoration method described above.
  • the transfer capacity of the recording server can be reduced without reducing the number of transfer frames of the recording server and without reducing the resolution.
  • the number of frames can be increased or the resolution can be increased. For example, even if the acquisition of an image of 1 frame per second with the I frame alone has been the limit, it is possible to acquire an image of 3 frames per second by using the I frame and the W frame. Can be raised.
  • the resolution is increased under the same conditions, a delay occurs in the first image, but after that, the difference becomes a difference and the capacity decreases, so that the on-time can be kept.

Abstract

A difference image generation method includes: a step for, on the basis of a first whole image and a second whole image obtained after the first image, generating a first difference image which sets the area where the first and second whole images do not differ as a mask area; and a step for generating a second difference image by setting the area other than the mask area of the first difference image as a section corresponding to the second whole image, and setting the mask area of the first difference image as transparent color information.

Description

差分画像作成方法、画像復元方法、差分検出装置、画像復元装置及び監視方法Difference image creation method, image restoration method, difference detection device, image restoration device, and monitoring method
　実施形態は、防犯や管理等のために、撮影した画像やデータをサーバに転送し、蓄積させる監視カメラシステムにおける監視方法と、差分画像作成方法、画像転送方法、及び画像転送に適した差分検出装置、また、転送されたデータについて端末で表示できる画像に復元する画像復元方法、及び画像復元装置に関する。 Embodiments include a monitoring method in a surveillance camera system for transferring and storing captured images and data to a server for crime prevention and management, a difference image creation method, an image transfer method, and difference detection suitable for image transfer The present invention also relates to an image restoration method and an image restoration device for restoring transferred data to an image that can be displayed on a terminal.
　従来、防犯や調査あるいは管理といった目的で、集合住宅などの建物の出入口や店頭あるいは街頭、工場や配送センター等、監視対象となるポイントを監視用カメラにより適時監視し、監視結果を、インターネットその他電気通信回線を介して、ユーザ端末のモニターに送出し、モニターするカメラシステムが導入されている。静止画を撮影して、サーバで取得した画像を圧縮してから端末に表示するカメラシステムも開発されているが（例えば、特開２０１３－５１５３４号公報参照）、一般には、カメラで撮影した動画を記録するシステムの方が知られている（例えば、特開２００２－７７８８２号公報参照）。 Conventionally, for the purpose of crime prevention, investigation or management, the monitoring target points such as the entrances and exits of buildings such as apartment buildings, stores or streets, factories and distribution centers, etc., are monitored in a timely manner, and the monitoring results are sent to the Internet and other electricity A camera system for sending and monitoring to a monitor of a user terminal via a communication line has been introduced. A camera system that captures a still image and compresses the image acquired by the server and then displays the image on a terminal has been developed (see, for example, Japanese Patent Laid-Open No. 2013-51534). Is known (for example, see Japanese Patent Application Laid-Open No. 2002-77882).
　本開示の一側面に係る差分画像作成方法は、
　第１の全画像と、前記第１の全画像の後に取得した第２の全画像に基づき、前記第１の全画像と前記第２の全画像とが相違しないエリアをマスクエリアとした第１の差分画像を作成するステップ、
　前記第１の差分画像のマスクエリア以外については第２の全画像の対応部分とし、第１の差分画像のマスクエリアについては透明色情報を設定した第２の差分画像を作成するステップ、
　を含む。 The difference image creation method according to one aspect of the present disclosure is:
Based on a first all image and a second all image acquired after the first all image, a first area in which the first all image and the second all image are not different is defined as a mask area. Creating a difference image of
Creating a second difference image in which transparent color information is set for the mask area of the first difference image, except for the mask area of the first difference image, corresponding to the second whole image;
including.
実施形態の差分画像作成方法の第１実施例を利用する監視カメラシステムのシステム構成の一例を示す図である。It is a figure which shows an example of the system configuration | structure of the surveillance camera system using the 1st Example of the difference image production method of embodiment. 実施形態の差分画像作成方法の第１実施例の録画サーバの構成図である。It is a block diagram of the video recording server of 1st Example of the difference image production method of embodiment. 実施形態の差分画像作成方法の第１実施例の差分検出装置の構成図である。It is a block diagram of the difference detection apparatus of 1st Example of the difference image production method of embodiment. 実施形態の差分画像作成方法の第１実施例の端末（モバイル）の構成図である。It is a block diagram of the terminal (mobile) of the 1st Example of the difference image creation method of embodiment. 実施形態の差分画像作成方法の第１実施例の端末（ＰＣ）の構成図である。It is a block diagram of the terminal (PC) of the 1st Example of the difference image production method of embodiment. 実施形態の差分画像作成方法の第１実施例における監視用カメラから録画サーバへのデータ送信のイメージ図である。It is an image figure of the data transmission to the recording server from the monitoring camera in 1st Example of the difference image production method of embodiment. 実施形態の差分画像作成方法の実施例における差分抽出方法及び実施形態の差分画像作成方法の実施例により作成された差分画像からの画像復元方法の説明図である。It is explanatory drawing of the image restoration method from the difference image produced by the example of the difference extraction method in the Example of the difference image creation method of embodiment, and the difference image creation method of embodiment. 実施形態の差分画像作成方法の第１実施例における監視用カメラ（静止画用）と録画サーバとの接続から一次画像データ書き込みまでの手順概要を示すフロー図である。It is a flowchart which shows the procedure outline | summary from the connection of the monitoring camera (for still images) and the recording server in 1st Example of the difference image production method of embodiment to primary image data writing. 実施形態の差分画像作成方法の第１実施例におけるカメラ（動画撮影）と録画サーバとの接続から一次画像データ書き込みまでの手順概要を示すフロー図である。It is a flowchart which shows the procedure outline | summary from the connection of the camera (moving image | video photography) and recording server in 1st Example of the difference image production method of embodiment to primary image data writing. 実施形態の差分画像作成方法の第１実施例における画像の変換方法を示すイメージ図である。It is an image figure which shows the conversion method of the image in 1st Example of the difference image production method of embodiment.
　しかし、従来のシステムでは、サーバにおいては画像圧縮ができるが、監視カメラからサーバまでの転送では、転送容量が大きく、特に動画の場合に顕著であるという問題があった。 However, in the conventional system, the server can compress the image, but the transfer from the monitoring camera to the server has a large transfer capacity, particularly in the case of moving images.
　また、監視カメラからサーバまでの転送容量を小さくするには、撮影間隔を長くして転送するフレーム数を減らすか、解像度を下げて撮影することが必要となるという問題があった。 Also, in order to reduce the transfer capacity from the monitoring camera to the server, there is a problem that it is necessary to increase the shooting interval to reduce the number of frames to be transferred, or to reduce the resolution and take a picture.
［実施形態］
　以下で説明する実施形態の第１の目的は、上記した課題を解決し、録画サーバへの転送フレーム数を落とさず、また解像度も落とさずに、録画サーバへの転送容量の削減を可能とする差分画像作成方法、及びこれを用いた画像転送方法と、差分検出装置、並びに監視カメラシステムにおける監視方法を提供することにある。また、実施形態の第２の目的は、かかる方法で転送されたデータについて端末で表示できる画像に復元する画像復元方法並びに画像復元装置を提供することにある。 [Embodiment]
The first object of the embodiment described below is to solve the above-described problem and to reduce the transfer capacity to the recording server without reducing the number of transfer frames to the recording server and without reducing the resolution. It is an object of the present invention to provide a difference image creation method, an image transfer method using the same, a difference detection device, and a monitoring method in a monitoring camera system. A second object of the embodiment is to provide an image restoration method and an image restoration apparatus that restores data transferred by such a method to an image that can be displayed on a terminal.
　実施形態の第１の態様に係る差分画像作成方法は、
　第１の全画像と、前記第１の全画像の後に取得した第２の全画像に基づき、前記第１の全画像と前記第２の全画像とが相違しないエリアをマスクエリアとした第１の差分画像を作成するステップ、
　前記第１の差分画像のマスクエリア以外については第２の全画像の対応部分とし、第１の差分画像のマスクエリアについては透明色情報を設定した第２の差分画像を作成するステップ、
　を含む。 The difference image creation method according to the first aspect of the embodiment is:
Based on a first all image and a second all image acquired after the first all image, a first area in which the first all image and the second all image are not different is defined as a mask area. Creating a difference image of
Creating a second difference image in which transparent color information is set for the mask area of the first difference image, except for the mask area of the first difference image, corresponding to the second whole image;
including.
　実施形態の第１の態様によれば、後で元の全画像を復元でき容量が小さい差分画像を作成できるので、画像データをサーバへ転送する際に、録画サーバへの転送フレーム数を落とさず、また解像度も落とさずに、録画サーバへの転送容量の削減を可能とする。転送容量の点から、仮に全画像２枚で１秒１コマの画像取得が限界であった場合でも、本態様によれば、１秒３コマの画像取得が実現できる。 According to the first aspect of the embodiment, since the original image can be restored later and a difference image with a small capacity can be created, the number of transfer frames to the recording server is not reduced when transferring image data to the server. In addition, the transfer capacity to the recording server can be reduced without reducing the resolution. From the viewpoint of transfer capacity, even if there is a limit to the acquisition of one frame per second for all two images, according to this aspect, the acquisition of three frames per second can be realized.
　なお、「画像」は、静止画像と動画像の両方の意味を含む言葉であるが、本開示においては、特に区別するときには、静止画像を「画像」、動画像を「映像」と表示する。 Note that “image” is a word including both the meanings of a still image and a moving image, but in the present disclosure, a still image is displayed as “image”, and a moving image is displayed as “video” when particularly distinguished.
　実施形態の第２の態様に係る画像復元方法は、上述した実施形態の第１の態様の差分画像作成方法によって作成された差分画像の復元方法であって、
　前記第１の全画像に前記第２の差分画像を合成するステップを含む。 An image restoration method according to a second aspect of the embodiment is a difference image restoration method created by the difference image creation method of the first aspect of the embodiment described above,
Synthesizing the second difference image with the first full image.
　実施形態の第２の態様によれば、解像度の高い画像を復元できるので、高解像度の動画を再生できる。 According to the second aspect of the embodiment, an image with a high resolution can be restored, so that a high-resolution video can be reproduced.
　実施形態の第３の態様に係る差分検出装置は、動画を撮影するカメラ群及び／又は静止画を撮影するカメラ群の各カメラに接続され、ルータとインターネットとを介して録画サーバに接続され、画像を保存するフォルダを有する差分検出装置であって、
　一定時間毎又は前記録画サーバからの全画像要求を受信する毎に、前記カメラから全画像を取得し、全画像転送用の全画像とし、前記フォルダに前記全画像転送用の全画像を保存するとともに、前記録画サーバに前記全画像転送用の全画像を転送する全画像転送手段、前記一定時間中の所定時間毎又は前記録画サーバからの差分画像要求を受信する毎に、前記カメラから全画像を取得し、差分画像転送用の全画像とする差分画像用全画像取得手段、
　前記フォルダに保存されている全画像を抽出する抽出手段、
　前記抽出手段で抽出された全画像と前記差分画像用全画像取得手段で取得された差分画像転送用の全画像との差分画像であって、前記抽出手段で抽出された全画像と前記差分画像用全画像取得手段で取得された差分画像転送用の全画像とが相違しないエリアをマスクエリアとした第１の差分画像を作成する第１の差分画像作成手段、
　前記差分画像用全画像取得手段で取得された差分画像転送用の全画像から前記第１の差分画像との差分画像であって、前記第１の差分画像のマスクエリア以外については、前記差分画像用全画像取得手段で取得された差分画像転送用の全画像の対応部分とし、前記第１の差分画像のマスクエリアについては透明色情報を設定した第２の差分画像を作成する第２の差分画像作成手段、
　前記録画サーバに前記第２の差分画像を転送する差分画像転送手段、
　前記フォルダに保存されている全画像を前記差分画像用全画像取得手段で取得された差分画像転送用の全画像に置き換える全画像置換手段、
　を有する。 The difference detection apparatus according to the third aspect of the embodiment is connected to each camera of a camera group that captures a moving image and / or a camera group that captures a still image, and is connected to a recording server via a router and the Internet, A difference detection apparatus having a folder for storing images,
Acquire all images from the camera at regular time intervals or every time a request for all images from the recording server is received, store all images for all image transfer, and store all images for all image transfer in the folder In addition, all image transfer means for transferring all images for all image transfer to the recording server, every time a predetermined image is received during the predetermined time period or every time a difference image request is received from the recording server, all images are transmitted from the camera. A difference image all-image acquisition means for obtaining all images for difference image transfer,
Extraction means for extracting all images stored in the folder;
A difference image between the entire image extracted by the extraction unit and the entire image for differential image transfer acquired by the differential image all-image acquisition unit, the entire image extracted by the extraction unit and the difference image First difference image creating means for creating a first difference image using a mask area as an area that is not different from all images for transfer of difference images acquired by the all image acquiring means;
The difference image is a difference image between the entire difference image transfer image acquired by the difference image all image acquisition unit and the first difference image, except for the mask area of the first difference image. A second difference that creates a second difference image in which transparent color information is set for the mask area of the first difference image, as a corresponding portion of all images for difference image transfer acquired by the all-image acquisition unit Image creation means,
Difference image transfer means for transferring the second difference image to the recording server;
All-image replacement means for replacing all images stored in the folder with all images for differential image transfer acquired by the differential image all-image acquisition means;
Have
　実施形態の第３の態様によれば、カメラ群から画像データを録画サーバへ転送する際に、録画サーバ－の転送フレーム数を落とさず、また解像度も落とさずに、録画サーバ－の転送容量の削減を可能とする。 According to the third aspect of the embodiment, when transferring the image data from the camera group to the recording server, the transfer capacity of the recording server can be reduced without reducing the number of transfer frames of the recording server and without reducing the resolution. Enables reduction.
　実施形態の第４の態様に係る監視方法は、動画を撮影するカメラ群及び／又は静止画を撮影するカメラ群と、前記カメラ群のカメラに接続され画像を保存するフォルダを有する差分検出装置と、前記差分検出装置にルータ及びインターネットを介して接続された録画サーバと、を有する監視カメラシステムにおける監視方法であって、
　一定時間毎又は前記録画サーバからの全画像要求を受信する毎に、前記差分検出装置が、前記カメラから全画像を取得し、全画像転送用の全画像とし、前記フォルダに前記全画像転送用の全画像を保存するとともに、前記録画サーバに前記全画像転送用の全画像を転送する全画像転送ステップを含み、
かつ、
　前記一定時間中の所定時間毎又は前記録画サーバからの差分画像要求を受信する毎に、
前記差分検出装置が、前記カメラから全画像を取得し、差分画像転送用の全画像とする差分画像用全画像取得ステップと、
　前記差分検出装置が、前記フォルダに保存されている全画像を抽出する抽出ステップと、
前記差分検出装置が、前記抽出ステップで抽出された全画像と前記差分画像用全画像取得ステップで取得された差分画像転送用の全画像との差分画像であって、前記抽出ステップで抽出された全画像と前記差分画像用全画像取得手段で取得された差分画像転送用の全画像とが相違しないエリアをマスクエリアとした第１の差分画像を作成する第１の差分画像作成ステップと、
　前記差分検出装置が、前記差分画像用全画像取得抽出ステップで取得された差分画像転送用の全画像から前記第１の差分画像との差分画像であって、前記第１の差分画像のマスクエリア以外については、前記差分画像用全画像取得ステップで取得された差分画像転送用の全画像の対応部分とし、前記第１の差分画像のマスクエリアについては透明色情報を設定した第２の差分画像を作成する第２の差分画像作成ステップと、
　前記差分検出装置が、前記録画サーバに前記第２の差分画像を転送する差分画像転送ステップと、
　前記差分検出装置が、前記フォルダに保存されている全画像を前記差分画像用全画像取得ステップで取得された差分画像転送用の全画像に置き換える全画像置換ステップ、
の各ステップを順に繰り返すことを含み、
かつ、
　前記録画サーバが、前記差分検出装置から取得した全画像を保存する全画像保存ステップ、
　前記録画サーバが、前記差分検出装置から取得した差分画像をその一つ前の時点の全画像に合成して全画像として保存する差分画像変換ステップを含む。 A monitoring method according to a fourth aspect of the embodiment includes a camera group that captures a moving image and / or a camera group that captures a still image, and a difference detection device that includes a folder that is connected to the camera of the camera group and stores an image. A monitoring server system having a recording server connected to the difference detection device via a router and the Internet,
The difference detection device obtains all images from the camera every predetermined time or every time a request for all images from the recording server is received, and sets all images for all image transfer to the folder. And a full image transfer step of transferring all the images for full image transfer to the recording server.
And,
Every time the difference image request from the recording server is received every predetermined time in the fixed time,
The difference detection device acquires all images from the camera and sets all images for difference image transfer as a difference image all image acquisition step;
An extraction step in which the difference detection device extracts all images stored in the folder;
The difference detection device is a difference image between all images extracted in the extraction step and all images for transfer of difference images acquired in the difference image all-image acquisition step, and extracted in the extraction step. A first difference image creation step of creating a first difference image using a mask area as an area where all images and the difference image transfer all images acquired by the difference image all image acquisition means are not different from each other;
The difference detection device is a difference image between the first difference image and the entire difference image transfer image acquired in the difference image all image acquisition and extraction step, and a mask area of the first difference image Except for the second differential image in which transparent color information is set for the mask area of the first differential image, which is a corresponding portion of the total image for differential image transfer acquired in the differential image all-image acquisition step. A second difference image creation step for creating
A difference image transfer step in which the difference detection device transfers the second difference image to the recording server;
The difference detection apparatus replaces all images stored in the folder with all images for differential image transfer acquired in the difference image all image acquisition step,
Including repeating each step of
And,
An all-image storage step in which the recording server stores all images acquired from the difference detection device;
The recording server includes a difference image conversion step in which the difference image acquired from the difference detection device is combined with all the images at the previous time point and stored as all images.
　実施形態の第４の態様によれば、録画サーバへの転送フレーム数を落とさず、また解像度も落とさずに、録画サーバへの転送容量の削減を可能とするうえ、録画サーバにおいて解像度の高い画像を復元できるので、高解像度の動画を再生できる。また、ローカルネットワーク以外にも動画カメラを設置してインターネット網を介したデータ取得が可能となる。また、互いにインターネット網を介して接続されたカメラからサーバに画像を送信する際に、所定のタイミングでの全画像の他、データ量が小さく差分画像より前の画像と組みあわせて元の画像に復元可能な差分データを作成してから転送できるので、インターネット網を流れる転送容量を削減することができる。 According to the fourth aspect of the embodiment, it is possible to reduce the transfer capacity to the recording server without reducing the number of frames transferred to the recording server and without reducing the resolution. Can be restored, so high-resolution video can be played. In addition to the local network, a video camera can be installed to acquire data via the Internet network. In addition, when images are transmitted to the server from cameras connected to each other via the Internet network, in addition to all images at a predetermined timing, the original image is combined with an image having a small data amount and an image before the difference image. Since the differential data that can be restored can be created and transferred, the transfer capacity flowing through the Internet network can be reduced.
　実施形態の第５の態様に係る監視方法は、上記第５の態様に係る監視方法であって、
　前記全画像転送ステップ及び前記差分画像用全画像取得ステップにおいて、前記カメラから全画像を取得した後、前記フォルダに保存されている全画像と画像の色構成を比較し、光量変化によるノイズを除去してから保存する。 A monitoring method according to a fifth aspect of the embodiment is the monitoring method according to the fifth aspect,
In the all image transfer step and the difference image all image acquisition step, after acquiring all images from the camera, the entire image stored in the folder is compared with the color configuration of the image, and noise due to a change in light amount is removed. Then save.
　実施形態の第５の態様によれば、屋外等での太陽の光の影響や、屋内での照明の点灯消灯の影響等で生じる画像の色調の変化による差分量を軽減できるので、外部環境の画像がすこしずつ変化するような箇所に設置された監視用カメラでも差分を小さくできるので転送量を削減できる。 According to the fifth aspect of the embodiment, it is possible to reduce the amount of difference due to the change in the color tone of the image caused by the influence of sunlight in the outdoors or the influence of lighting on / off of the indoor lighting. Even with a surveillance camera installed in a place where the image changes little by little, the amount of transfer can be reduced because the difference can be reduced.
　本開示において、「ネットワーク」には、ＬＡＮやインターネット、Ｗｉ－Ｆｉ回線、３Ｇ／ＬＴＥ回線、専用線等の通信網及びそれらの組合せから構成されるネットワークを含む。前記カメラ群は、静止画を撮影するカメラ群及び動画を撮影するカメラ群のいずれか又は両方であって、前記カメラ群に撮影させる画像が、静止画及び動画のいずれか又は両方であってもよい。各カメラは、出力機能と、静止画撮影機能及び／又は動画撮影機能を有する。カメラはＩＰカメラであることが好ましいが、アナログカメラでも可能である。静止画を撮影するカメラ群から取得した画像について、動画への圧縮形式は、現時点で高圧縮であるＨ．２６４であることがさらに好ましい。軽量で、ハードウェア、ソフトウェアいずれであっても実装が容易である。また、本態様においては、前記カメラ群のうち、動画を撮影するカメラは、前記録画サーバとＬＡＮネットワークで接続される必要がなく、３Ｇ／ＬＴＥ回線を介してインターネットに接続された動画の監視用カメラであっても、復元して解像度の高い動画を再生できるデータを録画サーバに転送でき、動画の監視用カメラを置ける範囲が広くなる。なお、録画サーバはクラウドサーバであってもよい。 In the present disclosure, the “network” includes a network constituted by a communication network such as a LAN, the Internet, a Wi-Fi line, a 3G / LTE line, a dedicated line, or a combination thereof. The camera group is either or both of a camera group that shoots a still image and a camera group that shoots a moving image, and the image to be captured by the camera group may be either or both of a still image and a moving image. Good. Each camera has an output function, a still image shooting function, and / or a moving image shooting function. The camera is preferably an IP camera, but can also be an analog camera. For images acquired from a group of cameras that shoot still images, the compression format for moving images is H.P. More preferably, it is H.264. It is lightweight and easy to implement regardless of hardware or software. Further, in this aspect, the camera that shoots a moving image in the camera group does not need to be connected to the recording server via a LAN network, and for monitoring a moving image connected to the Internet via a 3G / LTE line. Even with a camera, data that can be restored and reproduced with a high-resolution video can be transferred to the recording server, and the range in which a video surveillance camera can be placed is widened. Note that the recording server may be a cloud server.
　実施形態によれば、録画サーバ－の転送フレーム数を落とさず、また解像度も落とさずに、録画サーバ－の転送容量の削減を可能とする。 According to the embodiment, the transfer capacity of the recording server can be reduced without reducing the number of transfer frames of the recording server and without reducing the resolution.
　以下、実施形態について、実施例を用いて実施形態を具体的に説明するが、実施形態はこれらに限定されるものではない。 Hereinafter, the embodiment will be described in detail with reference to examples, but the embodiment is not limited thereto.
　［第１実施例］
　第１実施例の差分画像作成方法は、ネットワークを介して録画サーバで監視用カメラを作動させて、録画サーバが動画及び静止画のカメラ画像を撮影・収集・編集・保存し、録画
サーバがユーザの端末に端末からの要求に応じてネットワークを介してモニターする画像を送信する監視カメラシステムにおける監視方法に用いる差分画像作成方法である。第１実施例の差分画像作成方法に用いる第１実施例の差分検出装置と、上記監視カメラシステムにおける第１実施例の監視カメラの画像転送方法と第１実施例の画像復元方法についても合わせて説明する。 [First embodiment]
The difference image creation method of the first embodiment is such that the monitoring camera is operated on the recording server via the network, and the recording server captures, collects, edits, and stores the camera images of moving images and still images. This is a difference image creation method used for a monitoring method in a monitoring camera system that transmits an image to be monitored to a terminal of the terminal via a network in response to a request from the terminal. The difference detection apparatus of the first embodiment used for the difference image creating method of the first embodiment, the image transfer method of the monitoring camera of the first embodiment in the monitoring camera system, and the image restoration method of the first embodiment are also combined. explain.
　上記監視カメラシステムは、第１実施例の画像復元方法によって復元された画像を端末等に送信して、ライブ画像の再生・巻戻し・早送りもライブラリ再生も行うものである。また、上記監視カメラシステムは、復元された画像を圧縮して保存し、画像を端末等に送信して、過去の画像の再生・巻戻し・早送りも行うものである。本実施例では、静止画像の差分転送により、カメラから録画サーバへの転送容量を少なくでき、現地に録画サーバを設置せずにクラウドサーバ等での静止画像収集が可能となる。また、単純な静止画像収集に比べて、フレーム数の増加、もしくは解像度の拡大が可能となる。なお、ここで「ライブラリ再生」は、ライブ画像から過去へ巻き戻しての再生または過去の画像の早送り再生を指す。 The monitoring camera system transmits an image restored by the image restoration method of the first embodiment to a terminal or the like, and performs reproduction / rewinding / fast-forwarding of live images and library reproduction. The surveillance camera system compresses and stores the restored image, transmits the image to a terminal or the like, and performs playback / rewinding / fast-forwarding of past images. In this embodiment, the transfer capacity of still images can be reduced by transferring the still images from the camera to the recording server, and still images can be collected on a cloud server or the like without installing a recording server on site. In addition, the number of frames can be increased or the resolution can be increased as compared with simple still image collection. Here, “library playback” refers to playback by rewinding from a live image to the past or fast-forward playback of a past image.
（構成）
　図１は、実施形態の差分画像作成方法の第１実施例を利用する監視カメラシステムのシステム構成の一例を示す図である。ＬＡＮやインターネット、Ｗｉ－Ｆｉ回線、３Ｇ／ＬＴＥ回線、専用線等の通信網の組み合わせから構成されるネットワーク５００上には、（１）画像を撮影するカメラとして、複数台の監視用カメラ（動画用）７００Ａ～Ｃと複数台の監視用カメラ（静止画用）７０１Ａ～Ｃ、（２）監視用カメラ（動画用）７００Ｂ、Ｃ・監視用カメラ（静止画用）７０１Ｂ、Ｃのそれぞれに接続されており、監視用カメラで撮影した画像を取得して録画サーバ１００に転送する差分検出装置４００Ａ～Ｄ、（３）差分検出装置４００にルータ６００Ａ～Ｄ及びインターネットを介して接続され、差分検出装置４００から画像データを取得し蓄積し端末に送信する録画サーバ１００、（４）ネットワーク５００を介して録画サーバ１００と接続され、録画サーバ１００からの画像データを受信して表示する端末として、スマートフォンなどのモバイル端末２００Ａ、Ｂや、デスクトップＰＣ（パーソナルコンピュータ）やノートＰＣなどの閲覧ＰＣ３００Ａ～Ｃが接続されている。録画サーバ１００にはＬＡＮを介して監視用カメラ（動画用）７００Ａと監視用カメラ（静止画用）７０１Ａにも接続されているが、本実施例の差分画像作成方法は、録画サーバ１００とＬＡＮ上で接続されたカメラについては対象としていないため、以下で述べるカメラには、録画サーバ１００とＬＡＮ上で接続されたカメラ（図１では監視用カメラ（動画用）７００Ａと監視用カメラ（静止画用）７０１Ａで例示される）を含まない。図１中では、監視用カメラ（動画用）で撮影された動画（ストリーム動画）の流れを点線矢印で図示している。なお、以下、モバイル端末と閲覧ＰＣとをまとめるときは、「端末」又は「Ｖｉｅｗｅｒ」という。 (Constitution)
FIG. 1 is a diagram illustrating an example of a system configuration of a monitoring camera system that uses a first example of the differential image creation method of the embodiment. On a network 500 configured by a combination of communication networks such as a LAN, the Internet, Wi-Fi lines, 3G / LTE lines, and dedicated lines, (1) a plurality of surveillance cameras (moving images) 700A-C and multiple monitoring cameras (for still images) 701A-C, (2) Connect to each of monitoring cameras (for moving images) 700B and C / monitoring cameras (for still images) 701B, C The difference detection devices 400A to 400D that acquire images taken by the monitoring camera and transfer them to the recording server 100, and (3) are connected to the difference detection device 400 via the routers 600A to 600D and the Internet to detect the difference. Recording server 100 that acquires image data from the apparatus 400, stores it, and transmits it to the terminal. (4) Connected to the recording server 100 via the network 500, As a terminal for receiving and displaying image data from over server 100, the mobile terminal 200A such as smartphones, and B, viewing PC 300A ~ C, such as a desktop PC (personal computer) or a notebook PC is connected. The recording server 100 is also connected to a monitoring camera (for moving images) 700A and a monitoring camera (for still images) 701A via a LAN. However, the difference image creation method of this embodiment is different from the recording server 100 and the LAN. Since the cameras connected above are not targeted, the cameras described below include cameras connected to the recording server 100 on the LAN (the monitoring camera (for moving image) 700A and the monitoring camera (still image in FIG. 1)). For example) 701A). In FIG. 1, the flow of a moving image (stream moving image) taken by a monitoring camera (for moving image) is indicated by a dotted arrow. Hereinafter, when the mobile terminal and the viewing PC are combined, they are referred to as “terminal” or “Viewer”.
　録画サーバ１００は、監視用カメラ（動画用）７００Ｂ、Ｃ及び監視用カメラ（静止画用）７０１Ｂ、Ｃとは、インターネットを含むネットワーク５００を介して接続されている。本実施例では、録画サーバ１００と同じＬＡＮ以外のネットワーク上にも、静止画を撮影する監視用カメラのみならず、動画を撮影する監視用カメラを設けることができる。監視用カメラ（動画用）７００Ｂ、Ｃ、監視用カメラ（静止画用）７０１Ｂ、Ｃには、それぞれ差分検出装置４００Ｄ、Ｃ、Ｂ、Ａが接続されている。録画サーバ１００が接続されたＬＡＮとルータ６００Ａを介して接続されたインターネット網には、３Ｇ／ＬＴＥ回線が接続され、またルータ６００Ｃを介して別のＬＡＮが接続されている。差分検出装置４００Ｄは、ルータ６００Ｄと３Ｇ／ＬＴＥ回線を介してインターネット網に接続されており、差分検出装置４００Ｂは、ルータ６００Ｂと３Ｇ／ＬＴＥ回線を介してインターネット網に接続されている。差分検出装置４００Ａ、Ｃは、録画サーバ１００とは異なるＬＡＮに接続されている。図１は、監視用カメラ（動画用）及び監視用カメラ（静止画用）や閲覧ＰＣやモバイル端末を、録画サーバと複数のネットワークを介して接続した接続例を示すものであり、これに限定されない。本実施例では、監視用カメラ（静止画用）４０１と録画サーバ１００との通信及び監視用カメラ（動画用）４００と録画サーバ１００との通信は、セッションを確立してから通信を開始する信頼性の高いＴＣＰで行うが、セッションを確立しないでデータを送信する早くてリアルタイム性に優れたＵＤＰでもよく、ＴＣＰ、ＵＤＰどちらのプロトコルでも画像を取得できる。 The recording server 100 is connected to the monitoring cameras (for moving images) 700B and C and the monitoring cameras (for still images) 701B and C via a network 500 including the Internet. In the present embodiment, not only the monitoring camera for shooting a still image but also a monitoring camera for shooting a moving image can be provided on a network other than the same LAN as the recording server 100. The difference detection devices 400D, C, B, and A are connected to the monitoring cameras (for moving images) 700B and C and the monitoring cameras (for still images) 701B and C, respectively. A 3G / LTE line is connected to the LAN connected to the recording server 100 via the router 600A and another LAN is connected via the router 600C. The difference detection device 400D is connected to the Internet network via a router 600D and a 3G / LTE line, and the difference detection device 400B is connected to the Internet network via a router 600B and a 3G / LTE line. The difference detection apparatuses 400 </ b> A and 400 </ b> C are connected to a different LAN from the recording server 100. FIG. 1 shows a connection example in which a monitoring camera (for moving images), a monitoring camera (for still images), a viewing PC, and a mobile terminal are connected to a recording server via a plurality of networks. Not. In the present embodiment, the communication between the monitoring camera (for still image) 401 and the recording server 100 and the communication between the monitoring camera (for moving image) 400 and the recording server 100 are reliable after the session is established. Although it is performed by TCP having high characteristics, UDP that transmits data without establishing a session and that is fast and excellent in real time may be used, and an image can be acquired by using both TCP and UDP protocols.
　監視用カメラ（動画用）や監視用カメラ（静止画用）は、本実施例ではＩＰカメラであるが、アナログカメラでもよい。ただし、アナログの場合、変換器を要する。画像表示用の端末である閲覧ＰＣ３００も、録画サーバ１００と同じＬＡＮ内に存在する場合に限らず、インターネットやＬＡＮやＷｉ－Ｆｉ回線等、複数のネットワークを介して接続されていてもよい。画像表示用の端末であるモバイル端末２００についても、ＬＡＮ、インターネット、携帯電話用ネットワーク（Ｗｉ－Ｆｉ回線や３Ｇ／ＬＴＥ回線等）等、複数のネットワークを介して接続されていてもよい。なお、ＬＡＮとインターネット間や、監視用カメラ（動画用）及び監視用カメラ（静止画用）とインターネット間は、ルータ６００を介する。 The monitoring camera (for moving images) and the monitoring camera (for still images) are IP cameras in this embodiment, but may be analog cameras. However, in the case of analog, a converter is required. The viewing PC 300 that is an image display terminal is not limited to being in the same LAN as the recording server 100, and may be connected via a plurality of networks such as the Internet, a LAN, or a Wi-Fi line. The mobile terminal 200, which is an image display terminal, may also be connected via a plurality of networks such as a LAN, the Internet, and a cellular phone network (Wi-Fi line, 3G / LTE line, etc.). The router 600 is connected between the LAN and the Internet, and between the monitoring camera (for moving images) and the monitoring camera (for still images) and the Internet.
　１台の録画サーバ１００は、複数台の監視用カメラ（動画用）７００からカメラ動画を、また複数台の監視用カメラ（静止画用）７０１からカメラ静止画を取得することができる。本実施例では、録画サーバ１００側がネットワークトラフィックを自動的に分散させるので、ネットワーク設計が不要で、カメラをポートに繋ぐだけで済むため、作業が非常に簡便となる。また、通信が不通になった場合にも再開が容易である。 One recording server 100 can acquire a camera moving image from a plurality of monitoring cameras (for moving images) 700 and a camera still image from a plurality of monitoring cameras (for still images) 701. In this embodiment, since the recording server 100 automatically distributes network traffic, network design is not required, and it is only necessary to connect the camera to the port, so that the operation becomes very simple. In addition, resuming is easy even when communication is interrupted.
　なお、本実施例と異なるが、録画サーバから差分検出装置にアクセスせずに、差分検出装置から録画サーバに画像を転送する方式（変形例）でも良い。かかる変形例の場合は、通信が不通になったときに録画サーバ側で状況把握できない一方、ルータのポート変更の必要がないというメリットがある．変形例での処理としては、差分検出装置内のタイマーで、１秒などのタイミングで画像取得を行い録画サーバに転送し、転送が正常に終わってから、１秒等の設定時問でスリープをする。 Although different from the present embodiment, a method (variation) may be used in which an image is transferred from the difference detection device to the recording server without accessing the difference detection device from the recording server. In the case of such a modified example, there is an advantage that the situation cannot be grasped on the recording server side when communication is interrupted, while the router port need not be changed. As a process in the modified example, a timer in the difference detection device acquires an image at a timing such as 1 second and transfers it to the recording server. After the transfer is normally completed, the sleep is set at a setting time such as 1 second. To do.
　従来のＩＰカメラシステムでは、動画を取得する監視用カメラは、録画サーバと同じＬＡＮ上に置かないとネットワークへの負担が過大となったが、本実施例では、録画サーバへの転送フレーム数を落とさず、また解像度も落とさずに、録画サーバへの転送容量の削減を可能とするので、インターネット上、また、携帯端末等、３Ｇ／ＬＴＥ回線上にも動画用の監視用カメラを設けることができるため、自由度が非常に高くなる。 In the conventional IP camera system, if the surveillance camera for acquiring the moving image is not placed on the same LAN as the recording server, the burden on the network becomes excessive. In this embodiment, the number of frames transferred to the recording server is reduced. Since the transfer capacity to the recording server can be reduced without dropping the resolution and without reducing the resolution, a video surveillance camera may be provided on the Internet or on a 3G / LTE line such as a portable terminal. Because it can, the degree of freedom becomes very high.
　図２は、実施形態の差分画像作成方法の第１実施例の録画サーバの構成図である。録画サーバ１００は、キャッシュメモリであるメモリ１０２を伴うＣＰＵ１０１やデバイスドライバ等を有する制御・演算装置と、ＤＲＡＭ等の主記憶装置やハードディスク等の補助記憶装置を有する記憶装置１１０と、ネットワークインターフェース１０４等の通信制御装置や表示装置としてのディスプレイ１０３、キーボード１０５、マウス１０６等で構成される入出力装置とを備えている。記憶装置１１０には、一次画像フォルダ１１１と二次映像フォルダとプログラム１１３と認証用データベースと環境設定フォルダ等の他、オペレーティングシステム１１４が格納されている。プログラム１１３は、通常、記憶装置１１０の補助記憶装置に格納されており、実行時には主記憶装置にロードされる。一次画像フォルダ１１１には、差分検出装置４００から取得した、全画像と、差分画像から作成された全画像と、で構成される全画像のみからなる動画を一次データとして蓄積し、二次映像フォルダには、一次画像フォルダ１１１内に蓄積された一定時間分の一次データを動画圧縮変換して二次データとして蓄積する。プログラム１１３には、撮影・収集・編集プログラムや送信プログラム等、各種プログラムを含む。認証用データベースには、ＩＤ、パスワード、各監視用カメラ７００、７０１やモバイル端末２００や閲覧ＰＣ３００のポート番号とＩＰアドレス、ＩＰアドレスのない端末では個体識別情報（ＵＩＤ）が蓄積されている。本実施例では、録画サーバ１００は、録画サーバと端末が一体となっており、自ら画像を表示する端末の役割も果たすため、また、メンテナンスや管理のため、ディスプレイ１０３、また入力手段としてのキーボード１０５やマウス１０６を有している。録画サーバでカメラ画像の再生を要しない場合は、画像表示装置としての端末機能はなくてもよい。環境設定フォルダには、各差分検出装置からの画像取得タイミング、一次データや二次データの作成間隔、圧縮条件等が蓄積されている。録画サーバ１００は、設置型のサーバであるが、クラウドサーバであってもよい。 FIG. 2 is a configuration diagram of the recording server of the first example of the differential image creating method of the embodiment. The recording server 100 includes a CPU 101 with a memory 102 that is a cache memory, a control / arithmetic unit having a device driver, a storage device 110 having a main storage device such as a DRAM and an auxiliary storage device such as a hard disk, a network interface 104, and the like. And an input / output device including a display 103 as a display device, a keyboard 105, a mouse 106, and the like. The storage device 110 stores an operating system 114 in addition to a primary image folder 111, a secondary video folder, a program 113, an authentication database, an environment setting folder, and the like. The program 113 is normally stored in the auxiliary storage device of the storage device 110, and is loaded into the main storage device at the time of execution. In the primary image folder 111, a moving image including only all images acquired from the difference detection device 400 and all images created from the difference images is accumulated as primary data. The primary data stored in the primary image folder 111 is converted into moving image compression data and stored as secondary data. The program 113 includes various programs such as a photographing / collecting / editing program and a transmission program. In the authentication database, IDs, passwords, port numbers and IP addresses of the monitoring cameras 700 and 701, the mobile terminal 200, and the viewing PC 300, and individual identification information (UID) are stored in terminals having no IP address. In this embodiment, the recording server 100 is integrated with the recording server and the terminal, and also serves as a terminal for displaying images by itself, and for maintenance and management, the display 103 and a keyboard as an input means. 105 and a mouse 106. When the recording server does not need to reproduce the camera image, the terminal function as the image display device may not be provided. The environment setting folder stores image acquisition timing from each difference detection device, primary data and secondary data creation intervals, compression conditions, and the like. The recording server 100 is a stationary server, but may be a cloud server.
　録画サーバ１００は、インターネットを介して接続された監視用カメラ（静止画用）７０１には、差分検出装置４００に画像要求することにより、静止画で画像を取得させて、監視用カメラ毎の差分検出装置４００において、全画像のまま或いは差分画像に変換して録画サーバ１００に送信させる。録画サーバ１００は、インターネットを介して接続された監視用カメラ（動画用）７００には、差分検出装置４００に画像要求することにより、動画で画像を取得させて、監視用カメラ毎の差分検出装置４００において、全画像のまま或いは差分画像に変換して録画サーバ１００に送信させる。 The recording server 100 causes the monitoring camera (for still image) 701 connected via the Internet to obtain an image as a still image by requesting an image from the difference detection device 400, so that the difference for each monitoring camera is obtained. In the detection device 400, the entire image is left as it is or converted into a difference image and transmitted to the recording server 100. The recording server 100 causes the monitoring camera (moving image) 700 connected via the Internet to request an image from the difference detecting device 400 to acquire an image with a moving image, and the difference detecting device for each monitoring camera. In 400, the whole image is converted into a differential image or transmitted to the recording server 100.
　録画サーバ１００は、ＣＰＵ１０１が、撮影・収集・編集プログラムをメモリ１０２にロードして実行することにより本実施例の差分画像作成方法を利用する監視方法における差分検出装置からの画像取得から編集保存までが可能なコンピュータの機能を実現する。録画サーバ１００は、ＣＰＵ１０１が、送信プログラムをメモリ１０２にロードして実行することにより端末への画像送出処理が可能なコンピュータの機能を実現する。ＣＰＵ１０１は、通常のコンピュータに搭載する演算処理装置であり、各種プログラムを実行し、各種制御等を行う。 In the recording server 100, the CPU 101 loads the image capturing / collecting / editing program to the memory 102 and executes the program to acquire and edit the image from the difference detection apparatus in the monitoring method using the difference image creating method of the present embodiment. Realize computer functions that are possible. The recording server 100 realizes the function of a computer capable of sending an image to a terminal by causing the CPU 101 to load and execute a transmission program in the memory 102. The CPU 101 is an arithmetic processing device mounted on a normal computer, executes various programs, and performs various controls.
　録画サーバ１００は、１台のサーバとする他、複数の録画サーバからなるサーバ群であってもよい。例えば、二次映像フォルダについて、一定期間（例えば２４時間）経過後の二次データについては、保存先を、カメラ画像を取得する録画サーバと別の録画サーバに設けた二次映像フォルダとしてもよい。頻繁には再生しない過去の保存データを別にすることで、さらに多くの台数のカメラを同一ネットワーク上で監視可能となる。 The recording server 100 may be a single server or a server group including a plurality of recording servers. For example, for the secondary video folder, for secondary data after a certain period (for example, 24 hours) has elapsed, the storage destination may be a secondary video folder provided in a recording server separate from the recording server that acquires the camera image. . By separating past stored data that is not frequently played back, a larger number of cameras can be monitored on the same network.
　撮影・収集・編集プログラムは、コンピュータに、（１）差分検出装置との接続を行う差分検出装置接続機能、（２）一定時間（例えば１秒）毎に、接続した差分検出装置に画像を録画サーバに入力させる画像取得機能、（３）差分検出装置から取得した画像が差分画像かを判断する差分画像判断機能、差分検出装置から取得した全画像を保存する全画像保存機能、（４）差分検出装置から取得した差分画像をその一つ前の時点の全画像に合成して全画像として保存する差分画像変換機、（５）取得又は合成した全画像を次回画像生成までメモリに保持する画像保持機能、（６）取得又は合成した全画像を一次データとして一次画像フォルダに蓄積する一次データ作成機能を実現させるためのプログラムである。 The shooting / collection / editing program records (1) a difference detection device connection function for connecting to the difference detection device, and (2) records an image on the connected difference detection device every predetermined time (for example, 1 second). An image acquisition function to be inputted to the server, (3) a difference image determination function for determining whether an image acquired from the difference detection device is a difference image, an all image storage function for storing all images acquired from the difference detection device, and (4) a difference A difference image converter that combines the difference image acquired from the detection device with all the images at the previous time and stores it as an entire image. (5) An image that holds the acquired or combined all images in the memory until the next image generation. (6) A program for realizing a primary data creation function for accumulating all acquired or synthesized images as primary data in a primary image folder.
　本実施例では、撮影・収集・編集プログラムは、より好ましい態様として、さらに、コンピュータに、一次データを、一定時間分（例えば１０分分）毎に結合させタイムスタンプを付して動画形式で圧縮した二次データに変換する二次データ作成機能をも実現させるためのプログラムである。一次データは、全て全画像（カメラ画像が動画の場合はＩフレーム（イントラフレーム））であり、二次データは、カメラ画像が静止画の場合でも動画形式に圧縮し、カメラ画像が動画の場合、例えば、Ｉフレームを２００フレームにつき１枚の割合とし、Ｐフレーム（予測インターフレーム）だけでなくＢフレーム（双方向予測インターフレーム）を組み合わせて挿入し、劣化せずに容量を削減する。本実施例では、未来のＩフレームを参照することができるので、Ｂフレームを挿入できる。Ｂフレームは前後比較差分であるので、Ｐフレームよりも小さくなる。 In the present embodiment, as a more preferable mode, the photographing / collecting / editing program further compresses the data in a moving image format by combining the primary data with a computer every predetermined time (for example, 10 minutes) and attaching a time stamp. This is a program for realizing a secondary data creation function for converting into secondary data. The primary data is all images (I frame (intra frame) when the camera image is a moving image), and the secondary data is compressed into a moving image format even when the camera image is a still image, and the camera image is a moving image. For example, the ratio of one I frame is 200 frames, and not only P frames (predicted interframes) but also B frames (bidirectional predictive interframes) are inserted in combination to reduce capacity without deterioration. In this embodiment, a future I frame can be referred to, so that a B frame can be inserted. Since the B frame is a front-back comparison difference, it is smaller than the P frame.
　送信プログラムは、コンピュータに、端末との接続を行う端末接続機能、端末にカメラ一覧を表示して端末からのカメラの選択を受け付ける画像選択受付機能、選択されたカメラについて一次データの画像をライブ画像として端末に送信するライブ画像送信機能、端末よりライブ画像からの巻戻し要求を受け付けるライブ画像巻戻し受付機能、巻戻し要求毎に、端末に前回送信した画像より一定時間分（例えば１秒分）過去の時点の一次データを抽出して巻戻しライブ画像として端末に送信する巻戻しライブ画像送信機能、端末より巻戻しライブ画像からの早送り要求を受け付ける巻戻しライブ画像早送り受付機能、早送り要求毎に、現時点の画像に達するまでは、端末に前回送信した画像より一定時間分（例えば１秒分）未来の時点の一次データを抽出して早送りライブ画像として端末に送信する早送ライブ画像送信機能をも実現させるためのプログラムである。「現時点の画像」とは、差分検出装置から取得した最新の画像を意味する。したがって、現時点の画像は、差分検出装置からの画像が録画サーバに入力される度に、新しい画像に変わる。 The transmission program has a terminal connection function for connecting to a terminal on a computer, an image selection receiving function for displaying a camera list on the terminal and receiving a camera selection from the terminal, and an image of primary data for the selected camera as a live image Live image transmission function to be transmitted to the terminal, live image rewind reception function to receive a rewind request from the live image from the terminal, and for each rewind request, a certain amount of time (for example, 1 second) from the image previously transmitted to the terminal Rewind live image transmission function that extracts primary data of the past time point and transmits it to the terminal as a rewind live image, rewind live image fast forward reception function that accepts a fast forward request from the rewind live image from the terminal, for each fast forward request Until the current image is reached, a certain time (for example, 1 second) from the previous image sent to the terminal. Data is a program for also realizing the live image transmission function feeding early to be transmitted to the terminal as the extraction to fast forward a live image. “Current image” means the latest image acquired from the difference detection device. Therefore, the current image changes to a new image each time an image from the difference detection apparatus is input to the recording server.
　送信プログラムは、さらに、コンピュータに、端末より巻戻しライブ画像からのライブラリ画像要求を受け付けるライブラリ画像受付機能、ライブラリ画像要求毎に、端末に前回送信した画像より一定時間分、詳細には、差分検出装置４００から録画サーバ１００への画像取得間隔分、未来の時点の一次データを抽出して早送りライブ画像として端末に送信するライブラリ画像送信機能をも実現させるためのプログラムである。なお、ここでいう「ライブラリ画像」は、過去の画像の早送り再生画像を指す。 The transmission program further includes a library image reception function for accepting a library image request from a rewind live image from the terminal to the computer, and for each library image request, a certain amount of time from the image previously transmitted to the terminal, in detail, a difference detection This is a program for realizing a library image transmission function that extracts primary data at a future time for an image acquisition interval from the apparatus 400 to the recording server 100 and transmits it to the terminal as a fast-forward live image. The “library image” here refers to a fast-forward playback image of a past image.
　本実施例では、端末２００、３００と録画サーバ１００との接続もＴＣＰ／ＩＰ方式で行い、ユーザＩＤとパスワードで認証を行って、端末が録画サーバに登録してある端末であることを確認してから画像送信を行う。認証は、録画サーバ上の認証用データベースによる認証が好ましい。 In the present embodiment, the connection between the terminals 200 and 300 and the recording server 100 is also performed by the TCP / IP method, authentication is performed with the user ID and password, and it is confirmed that the terminal is a terminal registered in the recording server. Then send the image. The authentication is preferably performed by an authentication database on the recording server.
　端末が同じＬＡＮ上になくても録画サーバがルータを介してネットワークで繋がっていれば、接続するルータのＩＰアドレスとルータに割り振られたポート番号で録画サーバから画像（全画像若しくは圧縮画像）の取得を行う。端末の接続には、端末認証時に登録したＵＩＤを利用して端末の認証を行う。 Even if the terminal is not on the same LAN, if the recording server is connected to the network via a router, an image (all images or compressed images) is sent from the recording server using the IP address of the connected router and the port number assigned to the router. Acquire. For terminal connection, the terminal is authenticated using the UID registered at the time of terminal authentication.
　録画サーバ１００は、モバイル端末２００の接続開始時にＵＩＤを用いた端末固有情報を元に端末を特定するため、ユーザＩＤとパスワードの認証及び端末固有情報の一致により画像表示を可能とする。 Since the recording server 100 identifies the terminal based on the terminal unique information using the UID at the start of connection of the mobile terminal 200, the recording server 100 can display an image by authenticating the user ID and the password and matching the terminal unique information.
　図３は、実施形態の差分画像作成方法の第１実施例の差分検出装置の構成図である。差分検出装置４００は、メモリ４０２を伴うＣＰＵ４０１やデバイスドライバ等を有する制御・演算装置と、ＤＲＡＭ等の主記憶装置やハードディスク等の補助記憶装置を有する記憶装置４１０と、ネットワークインターフェース４０４等の通信制御装置や、メンテナンス時のみ接続して用いる表示装置としてのディスプレイ４０３、キーボード４０５、マウス４０６等で構成される入出力装置とを備えている。記憶装置４１０には、蓄積フォルダ４１１や環境設定フォルダや差分検出プログラム４１３やオペレーティングシステム４１４が格納されている。蓄積フォルダ４１１は、画像を保存するフォルダであって、蓄積フォルダ４１１には、監視用カメラ（動画用）７００及び監視用カメラ（静止画用）７０１から取得したカメラ画像の全画像と、先に記憶した全画像と次に記憶した差分画像転送用の全画像とが相違しないエリアをマスクエリアとした第１の差分画像と、第１の差分画像のマスクエリア以外については差分画像転送用の全画像の対応部分とし第１の差分画像のマスクエリアについては透明色情報を設定した第２の差分画像と、を蓄積する。環境設定フォルダには、ノイズ除去条件や、全画像を保存するカメラ画像か差分画像を保存するカメラ画像かを判断する条件等が蓄積されている。 FIG. 3 is a configuration diagram of the difference detection apparatus according to the first example of the difference image creation method of the embodiment. The difference detection device 400 includes a control / arithmetic unit having a CPU 401 and a device driver with a memory 402, a storage device 410 having a main storage device such as a DRAM and an auxiliary storage device such as a hard disk, and a communication control for a network interface 404 and the like. And an input / output device including a display 403, a keyboard 405, a mouse 406, and the like, which are connected and used only during maintenance. The storage device 410 stores an accumulation folder 411, an environment setting folder, a difference detection program 413, and an operating system 414. The accumulation folder 411 is a folder for storing images. The accumulation folder 411 includes all images of the camera images acquired from the monitoring camera (for moving image) 700 and the monitoring camera (for still image) 701, and the first. The first difference image having a mask area as an area where the stored all images and the next stored difference image transfer image are not different from each other, and the difference image transfer all except for the mask area of the first difference image For the mask area of the first difference image as the corresponding part of the image, the second difference image in which transparent color information is set is stored. The environment setting folder stores noise removal conditions, conditions for determining whether a camera image for storing all images or a camera image for storing difference images, and the like.
　差分検出装置４００は、録画サーバ１００とインターネットを介して接続された監視用カメラ（動画用）７００及び監視用カメラ（静止画用）７０１に、一台につき一台ずつ接続されている。 The difference detection device 400 is connected to a monitoring camera (for moving images) 700 and a monitoring camera (for still images) 701 that are connected to the recording server 100 via the Internet one by one.
　差分検出装置４００は、本実施例では、録画サーバ１００によって、差分検出装置４００が接続している監視用カメラ（動画用）７００或いは監視用カメラ（静止画用）７０１への接続要求及びカメラ画像要求を制御され、監視用カメラ（動画用）７００や監視用カメラ（静止画用）７０１から取得したカメラ画像を所定の画像にして録画サーバ１００に送信する。 In this embodiment, the difference detection apparatus 400 is connected to the monitoring camera (for moving image) 700 or the monitoring camera (for still image) 701 to which the difference detection apparatus 400 is connected and the camera image by the recording server 100. The request is controlled, and the camera image acquired from the monitoring camera (for moving image) 700 and the monitoring camera (for still image) 701 is converted into a predetermined image and transmitted to the recording server 100.
　差分検出装置４００は、ＣＰＵ４０１が、差分検出プログラム４１３をメモリ４０２にロードして実行することにより本実施例の差分画像作成方法を利用する監視方法におけるカメラ画像取得から編集保存、そして録画サーバ－の送信までが可能なコンピュータの機能を実現する。ＣＰＵ４０１は、通常のコンピュータに搭載する演算処理装置であり、各種プログラムを実行し、各種制御等を行う。 In the difference detection device 400, the CPU 401 loads and executes the difference detection program 413 into the memory 402, thereby executing camera image acquisition, editing and storage in the monitoring method using the difference image creation method of the present embodiment, and the recording server Realize computer functions that can be transmitted. The CPU 401 is an arithmetic processing device mounted on a normal computer, executes various programs, and performs various controls.
　差分検出プログラム４１３は、コンピュータに、（１）録画サーバからの監視用カメラとの接続要求を受信する毎に、監視用カメラとの接続を行うカメラ接続機能、（２）録画サーバからの全画像要求を受信する毎に、接続した監視用カメラ（動画用）或いは監視用カメラ（静止画用）から全画像を取得し、全画像転送用の全画像とし、蓄積フォルダに全画像転送用の全画像を保存するとともに、録画サーバに全画像転送用の全画像を転送する全画像転送機能、（３）録画サーバからの差分画像要求を受信する毎に、接続した監視用カメラ（動画用）或いは監視用カメラ（静止画用）から全画像を取得し、差分画像転送用の全画像とする差分画像用全画像取得機能、（４）蓄積フォルダに保存されている全画像を抽出する抽出機能、（５）抽出機能により抽出された全画像と差分画像用全画像取得機能により取得された差分画像転送用の全画像との差分画像であって、抽出機能により抽出された全画像と差分画像用全画像取得機能により取得された差分画像転送用の全画像とが相違しないエリアをマスクエリアとした第１の差分画像を作成する第１の差分画像作成機能、（６）差分画像用全画像取得機能により取得された差分画像転送用の全画像から第１の差分画像との差分画像であって、第１の差分画像のマスクエリア以外については、差分画像用全画像取得機能により取得された差分画像転送用の全画像の対応部分とし、第１の差分画像のマスクエリアについては透明色情報を設定した第２の差分画像を作成する第２の差分画像作成機能、（７）録画サーバに第２の差分画像を転送する差分画像転送機能、（８）蓄積フォルダに保存されている全画像を差分画像用全画像取得機能により取得された差分画像転送用の全画像に置き換える全画像置換機能、を実現させるためのプログラムである。 The difference detection program 413 has (1) a camera connection function for connecting to the monitoring camera every time a connection request to the monitoring camera is received from the recording server, and (2) all images from the recording server. Every time a request is received, all images are acquired from the connected monitoring camera (for moving images) or monitoring camera (for still images), and all images are transferred for transfer to the storage folder. All image transfer function for saving all images and transferring all images for all image transfer to the recording server. (3) Every time a difference image request is received from the recording server, the connected monitoring camera (for moving images) or A difference image all image acquisition function for acquiring all images from a monitoring camera (for still images) and making all images for difference image transfer; (4) an extraction function for extracting all images stored in a storage folder; (5) A difference image between all images extracted by the extraction function and all images for transfer of difference images acquired by the all-image acquisition function for difference images, and all images extracted by the extraction function and all images for difference images are acquired. A first difference image creation function for creating a first difference image using a mask area as an area that is not different from the difference image transfer all images acquired by the function; (6) acquired by a difference image all image acquisition function; The difference image is a difference image between the entire difference image transfer image and the first difference image, except for the mask area of the first difference image. A second difference image creation function for creating a second difference image in which transparent color information is set for the mask area of the first difference image, and (7) a second difference to the recording server. image Difference image transfer function for transferring, (8) All image replacement function for replacing all images stored in the storage folder with all images for difference image transfer acquired by the all image acquisition function for difference images It is a program.
　なお、録画サーバからの全画像要求や差分画像要求を受信する毎にカメラから全画像を取得する代わりに、一定時間（例えば１秒）毎に、接続した監視用カメラ（動画用）或いは監視用カメラ（静止画用）から全画像を取得するものであってもよい。 In addition, instead of acquiring all images from the camera every time an image request or a difference image request is received from the recording server, the connected monitoring camera (for moving images) or monitoring is performed every certain time (for example, 1 second). All images may be acquired from a camera (for still images).
　なお、差分画像用全画像取得機能は、本実施例においては、さらに、監視用カメラ（動画用）又は監視用カメラ（静止画用）から全画像を取得した後、蓄積フォルダに保存されている全画像と画像の色構成を比較し、光量変化によるノイズを除去してから保存する機能を含む。 In this embodiment, the difference image all-image acquisition function is further stored in the storage folder after all images are acquired from the monitoring camera (for moving images) or the monitoring camera (for still images). It includes a function that compares the color configuration of all images and the image, and removes noise caused by changes in the amount of light before saving.
　本実施例では、差分検出装置を介しての監視用カメラと録画サーバとの接続はＴＣＰ／ＩＰ方式で行い、監視用カメラを特定するため、録画サーバ側で設定したユーザＩＤとパスワードで認証を行ってからカメラ画像の撮影等の要求を行う。認証は、録画サーバ上の認証用データベースによる認証が好ましい。 In this embodiment, the connection between the monitoring camera and the recording server via the difference detection device is performed by the TCP / IP method, and in order to identify the monitoring camera, authentication is performed with the user ID and password set on the recording server side. After that, a request for taking a camera image is made. The authentication is preferably performed by an authentication database on the recording server.
　カメラが同じＬＡＮ上になくてもルータを介してネットワークで繋がっていれば、接続するルータのＩＰアドレスとルータに割り振られたポート番号で差分検出装置から画像（全画像若しくは差分画像）の取得を行う。グローバルＩＰを持たない環境で接続されているときは、差分検出装置から録画サーバに向けて画像（全画像若しくは差分画像）転送を行う。この際は、録画サーバ側ルータのポート番号を開放し、録画サーバにポート転送する。 If the camera is connected to the network via a router even if it is not on the same LAN, an image (all images or difference images) is acquired from the difference detection device using the IP address of the router to be connected and the port number assigned to the router. Do. When connected in an environment that does not have a global IP, an image (all images or a difference image) is transferred from the difference detection device to the recording server. At this time, the port number of the router on the recording server side is released and the port is transferred to the recording server.
　録画サーバ１００は、ローカル環境（例えば、同じＬＡＮ上）にある監視用カメラ接続開始時にＩＰアドレス及びポート番号を用いた接続によって接続相手を特定でき、ユーザＩＤとパスワードで認証する。録画サーバ１００は、インターネット網を介する等、遠隔環境にある監視用カメラとの接続では、ルータのポート転送（ポートフォワーディングなどとも表現される）を用いることで、グローバルＩＰアドレス及びポート番号を用いて接続相手を特定する。 The recording server 100 can specify a connection partner by connecting using an IP address and a port number when starting a monitoring camera connection in a local environment (for example, on the same LAN), and authenticates with a user ID and a password. The recording server 100 uses a global IP address and port number by using router port forwarding (also expressed as port forwarding, etc.) when connecting to a surveillance camera in a remote environment such as via the Internet network. Identify the connection partner.
　図４は、実施形態の差分画像作成方法の第１実施例の端末（モバイル）の構成図である。モバイル端末２００は、メモリ２０２を伴うＣＰＵ２０１やデバイスドライバ等を有する制御・演算装置と、記憶装置２１０、データの送受信等を行う通信制御装置、表示装置としてのディスプレイ２０３、操作ボタンあるいはタッチパネル等の入出力装置を備えている。記憶装置２１０には、画像表示プログラム２１３やオペレーティングシステム２１４が格納されている。モバイル端末２００は、例えばスマートフォン等の携帯電話等であり、ＣＰＵ２０１が画像表示プログラム２１３をメモリ２０２にロードして実行することにより実施形態の画像表示が可能なコンピュータの機能を実現する。ＣＰＵ２０１は、通常のモバイル端末に搭載する演算処理装置であり、各種プログラムを実行し、各種制御等を行う。 FIG. 4 is a configuration diagram of the terminal (mobile) of the first example of the differential image creating method of the embodiment. The mobile terminal 200 includes a control / arithmetic apparatus having a CPU 201 with a memory 202, a device driver, and the like, a storage device 210, a communication control device for transmitting and receiving data, a display 203 as a display device, operation buttons, a touch panel and the like. An output device is provided. The storage device 210 stores an image display program 213 and an operating system 214. The mobile terminal 200 is, for example, a mobile phone such as a smartphone, and the CPU 201 loads the image display program 213 into the memory 202 and executes it, thereby realizing a computer function capable of displaying an image according to the embodiment. The CPU 201 is an arithmetic processing device mounted on a normal mobile terminal, executes various programs, and performs various controls.
　図５は、実施形態の差分画像作成方法の第１実施例の端末（ＰＣ）の構成図である。閲覧ＰＣ３００は、メモリ３０２を伴うＣＰＵ３０１やデバイスドライバ等を有する制御・演算装置と、ＤＲＡＭ等の主記憶装置やハードディスク等の補助記憶装置を有する記憶装置３１０と、ネットワークインターフェース３０４等の通信制御装置や表示装置としてのディスプレイ３０３、キーボード３０５、マウス３０６等で構成される入出力装置とを備えている。記憶装置３１０には、画像表示プログラム３１３やオペレーティングシステム３１４が格納されている。閲覧ＰＣ３００は、例えばデスクトップＰＣやノートＰＣ、タブレット端末等であり、ＣＰＵ３０１が画像表示プログラム３１３をメモリ３０２にロードして実行することにより実施形態の画像表示が可能なコンピュータの機能を実現する。ＣＰＵ３０１は、通常のＰＣに搭載する演算処理装置であり、各種プログラムを実行し、各種制御等を行う。 FIG. 5 is a configuration diagram of the terminal (PC) of the first example of the differential image creating method of the embodiment. The viewing PC 300 includes a control / arithmetic unit having a CPU 301 and a device driver with a memory 302, a storage device 310 having a main storage device such as a DRAM and an auxiliary storage device such as a hard disk, a communication control device such as a network interface 304, And an input / output device including a display 303 as a display device, a keyboard 305, a mouse 306, and the like. The storage device 310 stores an image display program 313 and an operating system 314. The browsing PC 300 is, for example, a desktop PC, a notebook PC, a tablet terminal, or the like. The CPU 301 loads the image display program 313 into the memory 302 and executes it, thereby realizing a computer function capable of displaying an image according to the embodiment. The CPU 301 is an arithmetic processing device mounted on a normal PC, executes various programs, and performs various controls.
　画像表示プログラムは、コンピュータに、録画サーバとの接続を行う端末接続機能、録画サーバから送信された画像を表示する画像表示機能を実現させるためのプログラムである。 The image display program is a program for realizing a terminal connection function for connecting to a recording server and an image display function for displaying an image transmitted from the recording server on a computer.
　画像表示プログラムは、より好ましい態様として、さらに、コンピュータに、ライブ画像表示中に、監視用カメラから録画サーバ－のカメラ画像取得間隔毎に、ライブ画像を録画サーバに要求するライブ画像要求機能、ライブ画像表示中に、巻戻し要求の入力を受け付け、巻戻しライブ画像を録画サーバに要求する巻戻し開始要求機能、巻戻しライブ画像表示中に、ライブ画像の要求間隔より短い一定時間（例えば０．２秒）毎に、巻戻しライブ画像を録画サーバに要求する巻戻し継続要求機能、巻戻しライブ画像表示中に、ユーザからの早送り要求の入力を受け付け、早送りライブ画像を録画サーバに要求する巻戻しライブ画像早送り開始要求機能、早送りライブ画像表示中に、ライブ画像の要求間隔より短い一定時間（例えば０．２秒）毎に、早送りライブ画像を録画サーバに要求する巻戻しライブ画像早送り継続要求機能をも実現させるためのプログラムである。 As a more preferable aspect, the image display program further includes a live image request function for requesting a live image to the recording server at every camera image acquisition interval of the recording server from the monitoring camera during live image display on the computer. A rewind start request function for accepting an input of a rewind request during image display and requesting a rewind live image from the recording server, and a fixed time (for example, 0. 0) shorter than the live image request interval during rewind live image display. Every 2 seconds), a rewind continuation request function for requesting a rewind live image to the recording server, while receiving a rewind live image, accepts a fast forward request input from the user, and requests a rewind live image to the recording server. During the live image fast-forward start request function and fast-forward live image display, a fixed time shorter than the live image request interval (for example, 0.2 seconds) Every is a program for also realizing the rewind live image fast-forward continuation request function for requesting fast forward a live image to the recording server.
　本実施例では、画像表示プログラムは、より好ましい態様として、さらに、コンピュータに、巻戻しライブ画像表示中に、ライブラリ画像要求の入力を受け付け、ライブラリ画像を録画サーバに要求するライブラリ画像開始要求機能、ライブラリ画像表示中に、カメラから録画サーバ－のカメラ画像取得間隔毎に、ライブラリ画像を録画サーバに要求するライブラリ画像継続要求機能をも実現させるためのプログラムである。ここでいう「ライブラリ画像」は、ライブ画像から過去へ巻き戻しての再生画像または過去の画像の早送り再生画像である。 In the present embodiment, as a more preferable aspect, the image display program further accepts an input of a library image request while displaying a rewind live image on the computer, and requests a library image from the recording server. This is a program for realizing a library image continuation request function for requesting a library image to the recording server at every camera image acquisition interval of the recording server from the camera during display of the library image. The “library image” here is a replay image obtained by rewinding the live image to the past or a fast-forward replay image of the past image.
　本実施例においては、録画サーバ１００と閲覧ＰＣ３００と差分検出装置４００とは、ともにパーソナルコンピュータとして構成され、通常のパーソナルコンピュータが有するクロック機能等を備えている。モバイル端末２００もクロック機能等を備えている。 In this embodiment, the recording server 100, the viewing PC 300, and the difference detection device 400 are all configured as personal computers, and have a clock function or the like that a normal personal computer has. The mobile terminal 200 also has a clock function and the like.
　本実施例の差分検出方法では、第１実施例の差分検出装置４００は、動画を撮影するカメラ群と静止画を撮影するカメラ群の各カメラ（監視用カメラ（動画用）７００・監視用カメラ（静止画用）７０１）に接続され、ルータ６００とインターネットを含むネットワーク５００とを介して録画サーバ１００に接続され、画像を保存する蓄積フォルダ４１１を有する差分検出装置であって、（１）録画サーバ１００からの監視用カメラ（動画用）７００又は監視用カメラ（静止画用）７０１との接続要求を受信する毎に、監視用カメラ（動画用）７００又は監視用カメラ（静止画用）７０１との接続を行うカメラ接続手段、（２）一定時間轟又は録画サーバ１００からの全画像要求を受信する毎に、差分検出装置４００が接続している監視用カメラ（動画用）７００或いは監視用カメラ（静止画用）７０１から全画像を取得し、全画像転送用の全画像とし、蓄積フォルダ４１１に全画像転送用の全画像を保存するとともに、録画サーバ１００に前記全画像転送用の全画像を転送する全画像転送手段、（３）一定時間中の所定時間毎又は録画サーバ１００からの差分画像要求を受信する毎に、差分検出装置４００が接続している監視用カメラ（動画用）７００或いは監視用カメラ（静止画用）７０１から全画像を取得し、差分画像転送用の全画像とする差分画像用全画像取得手段、（４）蓄積フォルダ４１１に保存されている全画像を抽出する抽出手段、（５）抽出手段で抽出された全画像と差分画像用全画像取得手段で取得された差分画像転送用の全画像との差分画像であって、抽出手段で抽出された全画像と差分画像用全画像取得手段で取得された差分画像転送用の全画像とが相違しないエリアをマスクエリアとした第１の差分画像を作成する第１の差分画像作成手段、（６）差分画像用全画像取得手段で取得された差分画像転送用の全画像から第１の差分画像との差分画像であって、第１の差分画像のマスクエリア以外については、差分画像用全画像取得手段で取得された差分画像転送用の全画像の対応部分とし、第１の差分画像のマスクエリアについては透明色情報を設定した第２の差分画像を作成する第２の差分画像作成手段、（７）録画サーバ１００に第２の差分画像を転送する差分画像転送手段、（８）蓄積フォルダ４１１に保存されている全画像を差分画像用全画像取得手段で取得された差分画像転送用の全画像に置き換える全画像置換手段、を設けてある。本実施例では、差分画像用全画像取得手段は、さらに、監視用カメラ（動画用）７００又は監視用カメラ（静止画用）７０１から全画像を取得した後、蓄積フォルダ４１１に保存されている全画像と画像の色構成を比較し、光量変化によるノイズを除去してから保存する。 In the difference detection method according to the present embodiment, the difference detection apparatus 400 according to the first embodiment includes a camera group that shoots a moving image and a camera group that shoots a still image (monitoring camera (for moving image) 700, monitoring camera). (For still image) 701), connected to the recording server 100 via the router 600 and the network 500 including the Internet, and having a storage folder 411 for storing images, and (1) recording Each time a connection request with the monitoring camera (for moving image) 700 or the monitoring camera (for still image) 701 is received from the server 100, the monitoring camera (for moving image) 700 or the monitoring camera (for still image) 701 is received. (2) for monitoring for which the difference detection device 400 is connected every time when a request for all images from the recording server 100 is received. All images are acquired from the camera (for moving images) 700 or the monitoring camera (for still images) 701, are transferred to all images, and are stored in the storage folder 411. 100, all image transfer means for transferring all images for all image transfer, (3) a difference detection device 400 is connected every time a predetermined time in a certain time or every time a difference image request from the recording server 100 is received. A difference image all-image acquisition unit that acquires all images from the monitoring camera (moving image) 700 or the monitoring camera (still image) 701 and sets all images for difference image transfer; (4) storage folder 411 (5) a difference image between the entire image extracted by the extracting unit and the entire image for differential image transfer acquired by the differential image all-image acquiring unit. , Extraction First difference image creation for creating a first difference image using a mask area as an area where all the images extracted by the means and all the images for difference image transfer obtained by the difference image all-image obtaining means are not different And (6) a difference image between the entire difference image transfer image acquired by the difference image all image acquisition unit and the first difference image, except for the mask area of the first difference image. A second difference that creates a second difference image in which transparent color information is set for the mask area of the first difference image, as a corresponding portion of all images for difference image transfer acquired by the image all-image acquisition means Image creation means, (7) difference image transfer means for transferring the second difference image to the recording server 100, and (8) difference obtained by the difference image all image acquisition means for all images stored in the storage folder 411. All for image transfer All-image replacement means for replacing with images is provided. In this embodiment, the difference image all-image acquisition unit further acquires all images from the monitoring camera (for moving image) 700 or the monitoring camera (for still image) 701 and then stores them in the accumulation folder 411. Compare all the images and the color composition of the images, remove the noise due to the change in the amount of light, and save.
　差分検出装置４００は、前述のハードウェア構成と差分検出プログラム４１３によって（１）～（８）の手段として機能する。 The difference detection device 400 functions as means (1) to (8) by the hardware configuration and the difference detection program 413 described above.
　本実施例の差分検出方法では、録画サーバ１００は、（１）差分検出装置との接続を行う差分検出装置接続手段、（２）一定時間（例えば１秒）毎に、接続した差分検出装置に画像を録画サーバに入力させる画像取得手段、（３）差分検出装置から取得した画像が差分画像かを判断する差分画像判断手段、差分検出装置から取得した全画像を保存する全画像保存手段、（４）差分検出装置から取得した差分画像をその一つ前の時点の全画像に合成して全画像として保存する差分画像変換手段、（５）取得又は合成した全画像を次回画像生成までメモリに保持する画像保持手段、（６）取得又は合成した全画像を一次データとして一次画像フォルダに蓄積する一次データ作成手段、（７）一次データを、一定時間分（例えば１０分分）毎に結合させタイムスタンプを付して動画形式で圧縮した二次データに変換する二次データ作成手段を設けてある。 In the difference detection method of the present embodiment, the recording server 100 includes (1) a difference detection device connection means for connecting to the difference detection device, and (2) a connected difference detection device every fixed time (for example, 1 second). Image acquisition means for inputting an image to a recording server, (3) difference image determination means for determining whether the image acquired from the difference detection apparatus is a difference image, all image storage means for storing all images acquired from the difference detection apparatus, 4) Difference image conversion means for combining the difference image acquired from the difference detection device with all the images at the previous time and storing it as all images, (5) The acquired or combined all images in the memory until the next image generation Image holding means for holding; (6) primary data creating means for storing all acquired or synthesized images as primary data in a primary image folder; and (7) binding primary data every predetermined time (for example, 10 minutes). Is allowed is provided with a secondary data generating means for converting the secondary data compressed video format time stamped.
　録画サーバ１００は、前述のハードウェア構成と撮影・収集・編集プログラムによって（１）～（７）の手段として機能する。また、録画サーバ１００は、前述のハードウェア構成と送信プログラムによって、端末へのライブ画像やライブラリ画像やそれらの画像の巻戻しや早送り等の画像送信手段として機能する。ここでいう「ライブラリ画像」は、ライブ画像から過去へ巻き戻しての再生画像または過去の画像の早送り再生画像を指す。 The recording server 100 functions as means (1) to (7) according to the hardware configuration and the shooting / collection / editing program described above. In addition, the recording server 100 functions as an image transmission unit such as a live image, a library image, and rewinding or fast-forwarding of these images to the terminal by the above-described hardware configuration and transmission program. The “library image” here refers to a playback image that is rewound from the live image to the past or a fast-forward playback image of the past image.
　また、端末２００、３００に、（１）監視権限のある１又は複数の監視用カメラ７００についてカメラ動画のライブ再生を録画サーバ１００に要求するライブ再生要求手段、（２）録画サーバ１００から送信された一次データを表示するライブ再生表示手段、（３）監視権限のある１又は複数の監視用カメラ７００について過去のカメラ動画の再生を録画サーバ１００に要求する過去映像再生要求手段、（４）録画サーバ１００から送信された二次データを表示する過去映像再生表示手段を設けてある。端末は、前述のハードウェア構成と画像表示プログラム２１３、３１３によって、（１）～（４）の手段として機能する。 Also, the terminal 200, 300 is (1) live playback requesting means for requesting the recording server 100 to perform live playback of camera moving images for one or more monitoring cameras 700 with the authority to monitor, and (2) transmitted from the recording server 100. Live reproduction display means for displaying primary data, (3) past video reproduction request means for requesting the recording server 100 to reproduce past camera moving images for one or a plurality of monitoring cameras 700 having the authority to monitor, and (4) recording. Past video reproduction display means for displaying secondary data transmitted from the server 100 is provided. The terminal functions as means (1) to (4) by the hardware configuration and the image display programs 213 and 313 described above.
　なお、カメラ画像データを圧縮して作成された一次データは、一定時間毎に録画サーバで二次データに変換されているので、端末は、ライブ画像の巻戻しや巻戻し後の早送り以外に、従来の録画再生装置で可能な、過去のカメラ画像の再生や巻戻し、早送りも可能である。高画質でありながらデータ量が小さく、巻戻しライブ合成画像や早送りライブ合成画像は、一次的な保存用に圧縮した一次データを使用するため、高画質ながらデータ量が小さい。頻繁には使用しない、二次データ作成単位以上に過去分の再生や早送り巻戻しは、圧縮変換した動画ファイルである二次データを使用するため、動画ながら保存に要するデータ量が小さくて済む。また、ライブ合成画像の巻戻し後の再生が可能となる。 Since the primary data created by compressing the camera image data is converted into secondary data by the recording server at regular time intervals, the terminal is not limited to rewinding live images or fast-forwarding after rewinding. It is also possible to reproduce, rewind, and fast forward past camera images, which is possible with a conventional recording / playback apparatus. Although the image quality is high, the data amount is small, and the rewind live composite image and fast-forward live composite image use primary data compressed for primary storage, so the data amount is small while high image quality. Since the secondary data, which is a moving image file that has been compressed and converted, is used for playback and fast forward rewinding more than the secondary data creation unit that is not frequently used, the amount of data required for storage can be reduced while moving images. Further, it becomes possible to reproduce the live composite image after rewinding.
　上述した監視用カメラと差分検出装置と録画サーバと端末とによって構成される監視用カメラシステムは、第１実施例の差分画像作成方法と、かかる差分画像作成方法を利用した第１実施例の画像転送方法と、かかる画像転送方法によって転送された画像の復元方法である第１実施例の画像復元方法を実現する監視カメラシステムであり、差分検出装置から録画サーバ－の送信データ量が小さく、ローカル環境以外に動画撮影用カメラを設けてもデータ遅延等の問題が生じにくく、また、録画サーバから端末への送信データ量も小さく、解像度が高く高画質ながらネットワークにかかる負荷が小さい監視システムを実現できる。すなわち、録画サーバ－の転送フレーム数を落とさず、また解像度も落とさずに、録画サーバへの転送容量の削減を可能とする。ネットワーク上に同時に流れるデータ量が少なくなるので、一般のインターネット回線などを通じてカメラ画像を録画サーバに流すことができ、また、一般のインターネット回線などにも、ネットワークカメラ情報を流すことができる。さらに、インターネット回線（ＷＡＮ）越しのクラウドサーバでのデータ蓄積を可能にする。 The above-described monitoring camera system including the monitoring camera, the difference detection device, the recording server, and the terminal includes the difference image creation method of the first embodiment and the image of the first embodiment using the difference image creation method. A surveillance camera system that realizes a transfer method and an image restoration method according to the first embodiment, which is a restoration method of an image transferred by the image transfer method. Even if a video camera is installed in addition to the environment, problems such as data delays are unlikely to occur, and the amount of data transmitted from the recording server to the terminal is small. it can. In other words, the transfer capacity to the recording server can be reduced without reducing the number of transfer frames of the recording server and without reducing the resolution. Since the amount of data that flows simultaneously on the network is reduced, a camera image can be sent to the recording server through a general Internet line, and network camera information can also be sent to a general Internet line. In addition, data can be stored in a cloud server over the Internet line (WAN).
（手順）
　実施形態の差分画像作成方法の第１実施例の差分画像作成手順について次に説明する。本手順では、上述した監視用カメラと差分検出装置と録画サーバと端末とによって構成される監視カメラシステムを使用する。 (procedure)
Next, the difference image creation procedure of the first example of the difference image creation method of the embodiment will be described. In this procedure, the monitoring camera system including the above-described monitoring camera, difference detection device, recording server, and terminal is used.
　図６は、実施形態の差分画像作成方法の第１実施例における監視用カメラから録画サーバへのデータ送信のイメージ図である。監視用カメラ（動画用）であるストリームＩＰカメラも、監視用カメラ（静止画用）である静止画ＩＰカメラも、ネットワークＨＵＢを介して差分検出装置に接続されており、差分検出装置は、ネットワークＨＵＢを介してインターネット網に接続されており、録画サーバであるサーバ機は、インターネット網に接続されている。 FIG. 6 is an image diagram of data transmission from the monitoring camera to the recording server in the first example of the differential image creating method of the embodiment. Both a stream IP camera that is a monitoring camera (for moving images) and a still image IP camera that is a monitoring camera (for still images) are connected to a difference detection device via a network HUB. The server machine that is connected to the Internet network via the HUB and is a recording server is connected to the Internet network.
　録画サーバから差分検出装置へ全画像要求や差分画像要求が送信され、差分検出装置は、録画サーバから受信した要求に基づいて監視用カメラ（動画用）や、監視用カメラ（静止画用）に画像要求する。差分検出装置からの要求に基づいて、監視用カメラ（動画用）はストリーム動画を差分検出装置に送信し、監視用カメラ（静止画用）は静止画像を差分検出装置に送信する。差分検出装置に送信されたストリーム動画や静止画像は、差分検出装置において、静止画像である全画像又は差分画像に編集して録画サーバに送信する。したがって、インターネット網には静止画像のみ流れることになり、しかも差分画像では全画像よりデータ量が小さくなるので、ネットワークへの負荷が小さくて済む。 An all-image request or a difference image request is transmitted from the recording server to the difference detection device, and the difference detection device sends a monitoring camera (for video) or a monitoring camera (for still image) based on the request received from the recording server. Request an image. Based on a request from the difference detection device, the monitoring camera (for moving image) transmits a stream moving image to the difference detection device, and the monitoring camera (for still image) transmits a still image to the difference detection device. The stream moving image and still image transmitted to the difference detection device are edited into all images or difference images that are still images and transmitted to the recording server in the difference detection device. Therefore, only a still image flows through the Internet network, and the data amount of the difference image is smaller than that of all images, so that the load on the network can be reduced.
　図７は、実施形態の差分画像作成方法の実施例における差分抽出方法及び実施形態の差分画像作成方法の実施例により作成された差分画像からの画像復元方法の説明図である。 FIG. 7 is an explanatory diagram of an image restoration method from the difference image created by the difference extraction method in the example of the difference image creation method of the embodiment and the example of the difference image creation method of the embodiment.
　本実施例の差分画像作成方法では、（１）第１の全画像（図７における「Ａ画像」）と、第１の全画像の次に連続して取得した第２の全画像（図７における「Ｂ画像」）に基づき、第１の全画像と第２の全画像の差分画像であって、第１の全画像と第２の全画像とが相違しないエリアをマスクエリアとした第１の差分画像（図７における「Ａ画像とＢ画像の差分」）を作成するステップ、（２）第２の全画像から第１の差分画像との差分画像であって、第１の差分画像のマスクエリア以外については第２の全画像の対応部分とし、第１の差分画像のマスクエリアについては透明色情報を設定した第２の差分画像（図７における「Ｂ画像から差分発生エリアを抽出」＝「差分画像」）を作成するステップ、を含む。図７における「Ａ画像とＢ画像の差分」の絵において、マスクエリアは黒く塗りつぶした部分である。図７における「Ｂ画像から差分発生エリアを抽出」の絵において、黒く塗りつぶした部分は、αチャンネルとして透明色を扱えるフォーマットに変換して透明色情報を設定したエリアである。これにより、差分画像を、αチャンネルを保持できかつ効率的に容量圧縮可能なフォーマットにすることで、画像容量を削減できる。 In the difference image creation method of the present embodiment, (1) a first all image (“A image” in FIG. 7) and a second all image (FIG. 7) acquired successively after the first all image. The first image having a mask area that is a difference image between the first all image and the second all image and that is not different between the first all image and the second all image. A difference image (“difference between A image and B image” in FIG. 7), (2) a difference image between the second whole image and the first difference image, Except for the mask area, the second differential image in which transparent color information is set for the mask area of the first difference image is used as the corresponding portion of the second entire image ("extract difference generation area from B image" in FIG. 7). = “Difference image”). In the picture “Difference between A image and B image” in FIG. 7, the mask area is a blackened portion. In the picture “extract difference occurrence area from B image” in FIG. 7, the blacked out area is an area in which transparent color information is set by converting into a format that can handle transparent colors as an α channel. Thus, the image capacity can be reduced by making the difference image into a format capable of holding the α channel and efficiently compressing the capacity.
　上述した差分画像作成方法によって作成された差分画像の復元方法では、第１の全画像（図７における「Ａ画像」）に第２の差分画像（図７における「Ｂ画像から差分発生エリアを抽出」＝「差分画像」）を合成するステップを含む。第１の全画像に第２の差分画像を重ね合わせると第２の全画像（図７における「Ａ画像に差分画像のみを描画」＝「Ｂ画
像」）を復元できる。 In the restoration method of the difference image created by the difference image creation method described above, the difference generation area is extracted from the second whole difference image (“A image” in FIG. 7) and the second difference image (“B image” in FIG. 7). "=" Difference image "). When the second difference image is superimposed on the first all image, the second all image (“Draw only the difference image on the A image” = “B image” in FIG. 7) can be restored.
　本実施例の差分画像作成方法では、特に定点で設置する監視カメラのシステムにおいては、差分が小さいので、転送容量の縮小に極めて効果的である。 The difference image creation method of the present embodiment is extremely effective in reducing the transfer capacity because the difference is small particularly in a surveillance camera system installed at a fixed point.
　本実施例の差分画像作成方法を含む監視カメラの画像転送方法は、動画を撮影するカメラ群と静止画を撮影するカメラ群と、カメラ群の各カメラ（監視用カメラ（動画用）・監視用カメラ（静止画用））に接続され画像を保存するフォルダを有する差分検出装置と、差分検出装置にルータ及びインターネットを介して接続された録画サーバと、を有する監視カメラシステムにおける画像転送方法であって、（１）一定時間毎又は録画サーバからの全画像要求を受信する毎に、差分検出装置が、カメラから全画像を取得し、全画像転送用の全画像とし、蓄積フォルダに全画像転送用の全画像を保存するとともに、録画サーバに全画像転送用の全画像を転送する全画像転送ステップを含み、かつ、一定時間中の所定時間毎又は録画サーバからの差分画像要求を受信する毎に、（２）差分検出装置が、前記カメラから全画像を取得し、差分画像転送用の全画像とする差分画像用全画像取得ステップと、（３）差分検出装置が、蓄積フォルダに保存されている全画像を抽出する抽出ステップと、（４）差分検出装置が、抽出ステップで抽出された全画像と差分画像用全画像取得ステップで取得された差分画像転送用の全画像との差分画像であって、抽出ステップで抽出された全画像と差分画像用全画像取得抽出ステップで取得された差分画像転送用の全画像とが相違しないエリアをマスクエリアとした第１の差分画像を作成する第１の差分画像作成ステップと、（５）差分検出装置が、差分画像用全画像取得ステップで取得された差分画像転送用の全画像から第１の差分画像との差分画像であって、第１の差分画像のマスクエリア以外については、差分画像用全画像取得ステップで取得された差分画像転送用の全画像の対応部分とし、第１の差分画像のマスクエリアについては透明色情報を設定した第２の差分画像を作成する第２の差分画像作成ステップと、（６）差分検出装置が、録画サーバに第２の差分画像を転送する差分画像転送ステップと、（７）差分検出装置が、蓄積フォルダに保存されている全画像を差分画像用全画像取得ステップで取得された差分画像転送用の全画像に置換える全画像置換ステップの（２）～（７）の各ステップを順に繰り返すことを含み、かつ、（８）録画サーバが、差分検出装置から取得した全画像を保存する全画像保存ステップ、（９）録画サーバが、差分検出装置から取得した差分画像をその一つ前の時点の全画像に合成して全画像として保存する差分画像変換ステップを含む。また、上述した全画像転送ステップ及び／または差分画像用全画像取得ステップにおいて、本実施例においては、差分検出装置及び／または録画サーバは、カメラから全画像を取得した後、蓄積フォルダに保存されている全画像と画像の色構成を比較し、光量変化による差分量を除去してから保存する。差分量の除去は、第１の全画像と第２の全画像との間のホワイトバランスの差を減らすために行うものであって、大きく画像の色構成が変更されたタイミング（屋外日中、屋外夜間、屋内蛍光灯、屋内照明なしなど）の数枚の静止画像を用いてホワイトバランスの差が少なくなる調整を行うものである。 The image transfer method of the surveillance camera including the difference image creation method of the present embodiment includes a camera group for capturing a moving image, a camera group for capturing a still image, and each camera of the camera group (monitoring camera (for moving image) / monitoring) An image transfer method in a surveillance camera system having a difference detection device connected to a camera (for still images) and having a folder for storing images, and a recording server connected to the difference detection device via a router and the Internet. (1) Every time when a request for all images from the recording server is received, the difference detection device acquires all images from the camera, sets all images for all image transfer, and transfers all images to the storage folder. All image transfer steps for storing all images for image transfer and transferring all images for image transfer to the recording server, and at a predetermined time in a certain time or from the recording server Every time a split image request is received, (2) the difference detection device acquires all images from the camera and sets them as all images for difference image transfer, and (3) a difference detection device. An extraction step of extracting all images stored in the accumulation folder; and (4) a difference detection device for transferring a difference image acquired in the all-images extracted in the extraction step and a difference image all-image acquisition step. The difference image from all the images of the first image is a mask area that is an area in which the entire image extracted in the extraction step and the entire image for differential image transfer acquired in the differential image all-image acquisition and extraction step are not different. A first difference image creating step for creating one difference image; and (5) a difference detection device configured to obtain a first difference image from all images for transfer of difference images acquired in the difference image all image acquisition step. Difference image In addition, the areas other than the mask area of the first difference image are the corresponding parts of all the images for transferring the difference image acquired in the difference image all-image acquisition step, and the mask area of the first difference image is transparent in color. A second difference image creation step for creating a second difference image in which information is set; (6) a difference image transfer step in which the difference detection device transfers the second difference image to the recording server; and (7) a difference. Steps (2) to (7) of the all-image replacement step in which the detection device replaces all images stored in the storage folder with all images for differential image transfer acquired in the difference image all-image acquisition step. (8) the recording server stores all images acquired from the difference detection device, and (9) the difference image acquired from the difference detection device by the recording server. It includes a difference image conversion step of combining the entire previous image and storing it as an entire image. In the above-described all image transfer step and / or difference image all image acquisition step, in the present embodiment, the difference detection device and / or recording server acquires all images from the camera and then saves them in the accumulation folder. All the images and the color configuration of the image are compared, and the difference amount due to the change in the amount of light is removed and stored. The removal of the difference amount is performed in order to reduce the difference in white balance between the first all images and the second all images, and the timing when the color configuration of the image is largely changed (outdoor daytime, Adjustment is performed to reduce the difference in white balance using several still images (outdoor night, indoor fluorescent lamp, no indoor lighting, etc.).
　撮影・収集・編集プログラムと、差分検出プログラムとを有する監視カメラシステムのプログラムは、これらのステップをコンピュータに実行させるものである。図を用いて、さらに具体的に説明する。 The surveillance camera system program having the photographing / collecting / editing program and the difference detection program causes the computer to execute these steps. This will be described more specifically with reference to the drawings.
　図８は、実施形態の差分画像作成方法の第１実施例における監視用カメラ（静止画用）と録画サーバとの接続から一次画像データ書き込みまでの手順概要を示すフロー図である。まず、録画サーバは、ＴＣＰ／ＩＰ方式で差分検出装置に接続を要求する。差分検出装置は、かかる接続要求を受けて監視用カメラ（静止画用）に接続要求をする。監視用カメラ（静止画用）は１又は複数台で構成され、各カメラと録画サーバは、差分検出装置を介して、ユーザＩＤ及びパスワードで認証を行う。差分検出装置が各カメラからの接続応答を受けて、録画サーバの認証用データベースを参照して認証に成功すると、接続状態となる。 FIG. 8 is a flowchart showing an outline of the procedure from connection of the monitoring camera (for still image) and recording server to writing of primary image data in the first example of the differential image creating method of the embodiment. First, the recording server requests connection to the difference detection device by the TCP / IP method. In response to the connection request, the difference detection apparatus issues a connection request to the monitoring camera (for still image). One or a plurality of monitoring cameras (for still images) are configured, and each camera and recording server authenticate with a user ID and a password via a difference detection device. When the difference detection apparatus receives a connection response from each camera and successfully authenticates with reference to the authentication database of the recording server, the connection state is established.
　次に、録画サーバは差分検出装置に、画像要求を行う。差分検出装置は、かかる画像要求を受けて、監視用カメラ（静止画用）に、静止画を撮影し撮影したカメラ画像をＪＰＥＧ形式で差分検出装置に送信することを要求（画像要求）する。監視用カメラ（静止画用）は、画像要求を受けて、スナップショット（静止画）を撮影して、撮影して得られた静止画をＪＰＥＧ形式でカメラ画像としてＴＣＰ／ＩＰ方式で差分検出装置に送信する。本実施例では、録画サーバ側で一定時間毎に二次データに動画圧縮変換するため、静止画を撮影するカメラに動画フォーマットの機能が備わっている必要がない。 Next, the recording server makes an image request to the difference detection device. Upon receiving such an image request, the difference detection device requests (image request) the monitoring camera (for still image) to take a still image and transmit the captured camera image to the difference detection device in JPEG format. The surveillance camera (for still image) receives an image request, shoots a snapshot (still image), and uses the TCP / IP method to detect a still image obtained by shooting as a camera image in JPEG format. Send to. In this embodiment, since the video recording server side performs video compression conversion to secondary data at regular intervals, the camera that captures a still image does not need to have a video format function.
　差分検出装置は、静止画のカメラ画像を受信すると、上述のようにノイズ除去を行い、録画サーバから全画像要求が送信されてから監視用カメラ（静止画用）から送信された画像（１枚目）かを判断する。１枚目である場合は、差分検出装置は、全画像転送用の全画像とし、蓄積フォルダに全画像転送用の全画像として保存するとともに、録画サーバに全画像転送用の全画像を転送する全画像転送を行う。録画サーバから差分画像要求が送信されてから監視用カメラ（静止画用）から送信された画像である（１枚目でない）場合は、差分検出装置は、差分画像転送用の全画像とし、蓄積フォルダに保存されている全画像との差分から、上述のようにして透明色情報を設定した第２の差分画像を作成し、録画サーバに第２の差分画像を転送する差分画像転送を行う。差分検出装置は、蓄積フォルダに保存されている全画像を上述した差分画像転送用の全画像（第２の全画像）に置き換える。差分検出装置は、録画サーバから画像要求が送信される毎にこれらのステップを繰り返す。 When receiving the camera image of the still image, the difference detection device performs noise removal as described above, and the image (one image) transmitted from the monitoring camera (for still image) after the all-image request is transmitted from the recording server. Eyes). In the case of the first image, the difference detection device stores all images for all image transfer, stores all images for all image transfer in the storage folder, and transfers all images for all image transfer to the recording server. Transfer all images. When the difference image request is transmitted from the recording server and is an image transmitted from the monitoring camera (for still image) (not the first image), the difference detection device stores and stores all images for the difference image transfer. From the difference from all the images stored in the folder, the second difference image in which the transparent color information is set is created as described above, and the difference image transfer is performed to transfer the second difference image to the recording server. The difference detection device replaces all images stored in the accumulation folder with the above-described all images for differential image transfer (second all images). The difference detection apparatus repeats these steps every time an image request is transmitted from the recording server.
　録画サーバは、差分検出装置から画像を取得すると、画像復元を行う。まずは、録画サーバは、取得した画像が差分画像かを判断し、取得した画像が差分画像の場合は、その一つ前の時点の全画像に合成して全画像を生成し、取得した画像が全画像の場合は、そのままとし、いずれの場合も、画像をメモリに展開し、次の画像生成まで保持する。そして、録画サーバは、保持した画像を一次データとして、一次画像フォルダに蓄積する。その後、録画サーバは、一次データを動画圧縮変換して二次映像フォルダに蓄積する。 When the recording server obtains an image from the difference detection device, it performs image restoration. First, the recording server determines whether or not the acquired image is a difference image. If the acquired image is a difference image, the recording server generates a whole image by combining all the images at the previous time point. In the case of all images, the image is left as it is. In either case, the image is developed in the memory and held until the next image generation. Then, the recording server accumulates the held image as primary data in the primary image folder. Thereafter, the recording server compresses and converts the primary data into a moving image and stores it in the secondary video folder.
　図９は、実施形態の差分画像作成方法の第１実施例におけるカメラ（動画撮影）と録画サーバとの接続から一次画像データ書き込みまでの手順概要を示すフロー図である。まず、録画サーバは、ＴＣＰ／ＩＰ方式で差分検出装置に接続を要求する。差分検出装置は、かかる接続要求を受けて監視用カメラ（動画用）に接続要求をする。監視用カメラ（動画用）は１又は複数台で構成され、各カメラと録画サーバは、差分検出装置を介して、ユーザＩＤ及びパスワードで認証を行う。差分検出装置が各カメラからの接続応答を受けて、差分検出装置は、監視用カメラ（動画用）に、動画を撮影し撮影したストリーム動画のカメラ画像を差分検出装置に送信することを要求（画像要求）する。 FIG. 9 is a flowchart showing an outline of the procedure from the connection of the camera (moving image shooting) and the recording server to the writing of the primary image data in the first example of the differential image creating method of the embodiment. First, the recording server requests connection to the difference detection device by the TCP / IP method. In response to the connection request, the difference detection apparatus issues a connection request to the monitoring camera (for moving image). One or a plurality of monitoring cameras (for moving images) are configured, and each camera and the recording server authenticate with a user ID and a password via a difference detection device. When the difference detection device receives a connection response from each camera, the difference detection device requests the monitoring camera (for moving image) to send a camera image of a streamed moving image taken to the difference detection device. Image request).
　監視用カメラ（動画用）は、画像要求を受けて、動画を連続撮影して、撮影して得られた動画をストリーム方式でＨ．２６４ベースラインプロファイルの動画のカメラ画像として差分検出装置に送信する。差分検出装置は、動画のカメラ画像を受信すると、接線応答を録画サーバに送信し、録画サーバの認証用データベースを参照して認証に成功すると、接続状態となる。 The surveillance camera (for moving images) receives image requests, continuously shoots moving images, and shoots the moving images in a stream format. H.264 baseline profile moving image as a camera image is transmitted to the difference detection device. When the difference detection device receives the camera image of the moving image, the difference detection device transmits a tangential response to the recording server, and enters the connection state when the authentication is successful with reference to the authentication database of the recording server.
　次に、録画サーバは差分検出装置に、画像要求を行う。差分検出装置は、かかる画像要求を受けて、受信済の動画のカメラ画像から所定の間隔で静止画像を抽出し、ノイズを除去する。差分検出装置は、受信済の動画のカメラ画像から抽出した１枚目の静止画像は、全画像転送用の全画像とし、蓄積フォルダに全画像転送用の全画像として保存するとともに、録画サーバに全画像転送用の全画像を転送する全画像転送を行う。２枚目以降の静止画像に対しては、差分検出装置は、差分画像転送用の全画像として蓄積フォルダに保存されている全画像との差分から、上述のようにして透明色情報を設定した第２の差分画像を作成し、録画サーバに第２の差分画像を転送する差分画像転送を行う。差分検出装置は、蓄積フォルダに保存されている全画像を上述した差分画像転送用の全画像（第２の全画像）に置き換える。差分検出装置は、録画サーバから画像要求が送信される毎にこれらのステップを繰り返す。 Next, the recording server makes an image request to the difference detection device. In response to such an image request, the difference detection device extracts still images from the received camera image of the moving image at predetermined intervals, and removes noise. The difference detection apparatus saves the first still image extracted from the camera image of the received moving image as all images for all image transfer, stores them as all images for all image transfer in the storage folder, and stores them in the recording server. All image transfer for transferring all images for all image transfer is performed. For the second and subsequent still images, the difference detection device sets the transparent color information as described above based on the difference from all images stored in the storage folder as all images for difference image transfer. A second difference image is created, and difference image transfer is performed to transfer the second difference image to the recording server. The difference detection device replaces all images stored in the accumulation folder with the above-described all images for differential image transfer (second all images). The difference detection apparatus repeats these steps every time an image request is transmitted from the recording server.
　録画サーバは、差分検出装置から画像を取得すると、画像復元を行う。まずは、録画サーバは、取得した画像が差分画像かを判断し、取得した画像が差分画像の場合は、その一つ前の時点の全画像に合成して全画像を生成し、取得した画像が全画像の場合は、そのままとし、いずれの場合も、画像をメモリに展開し、次の画像生成まで保持する。そして、録画サーバは、保持した画像を一次データとして、一次画像フォルダに蓄積する。その後、録画サーバは、一次データを動画圧縮変換して二次映像フォルダに蓄積する。 When the recording server obtains an image from the difference detection device, it performs image restoration. First, the recording server determines whether or not the acquired image is a difference image. If the acquired image is a difference image, the recording server generates a whole image by combining all the images at the previous time point. In the case of all images, the image is left as it is. In either case, the image is developed in the memory and held until the next image generation. Then, the recording server accumulates the held image as primary data in the primary image folder. Thereafter, the recording server compresses and converts the primary data into a moving image and stores it in the secondary video folder.
　図１０は、実施形態の差分画像作成方法の第１実施例における画像の変換方法を示すイメージ図である。図１０は、監視用カメラ（動画用）から取得した画像に基づいて差分検出装置が作成した全画像（Ｉフレーム）と差分画像（Ｗフレーム）を録画サーバに送信する状態を示しており、Ｗフレームは透明色を含む差分画像である。録画サーバの一次画像フォルダに蓄積される一次データは、全フレームがＩフレームであって、二次映像フォルダに蓄積される二次データは、一次データからＩフレームを２００フレームに１枚の割合まで減少させ、ＢフレームとＰフレームとを含む圧縮した動画ファイルである。 FIG. 10 is an image diagram showing an image conversion method in the first example of the difference image creation method of the embodiment. FIG. 10 shows a state in which all images (I frames) and difference images (W frames) created by the difference detection device based on images acquired from the monitoring camera (for moving images) are transmitted to the recording server. The frame is a difference image including a transparent color. The primary data stored in the primary image folder of the recording server is all I frames, and the secondary data stored in the secondary video folder is from the primary data to the ratio of one I frame to 200 frames. This is a compressed moving image file including B frames and P frames.
　本実施例は、防犯や管理等のための監視用カメラシステムに利用できる差分画像作成方法であって、撮影した画像やデータはサーバに転送され、蓄積される。これを用いた画像転送方法によって録画サーバに転送されたデータは、上述した画像復元方法により、端末で表示できる画像に復元される。 This embodiment is a differential image creation method that can be used in a surveillance camera system for crime prevention, management, etc., and captured images and data are transferred to a server and stored. The data transferred to the recording server by the image transfer method using this is restored to an image that can be displayed on the terminal by the image restoration method described above.
（効果）
　本実施例によれば、録画サーバ－の転送フレーム数を落とさず、また解像度も落とさずに、録画サーバ－の転送容量の削減を可能とする。 (effect)
According to the present embodiment, the transfer capacity of the recording server can be reduced without reducing the number of transfer frames of the recording server and without reducing the resolution.
　また、インターネット回線等を経由してクラウドサーバでのデータ蓄積も可能となる。単純な静止画像収集に比べて、フレーム数の増加、若しくは解像度の拡大が可能である。例えば、従来、Ｉフレームのみで１秒１コマの画像取得が限界であった場合でもＩフレームとＷフレームとすることで、１秒３コマの画像取得が可能となり、したがって、一定時間に対するフレーム数を上げることが可能となる。同じ条件で解像度を上げた場合は、１枚目は遅延が発生するものの、それ以降は差分となり容量が減るので、オンタイムをキープできるようになる。 In addition, it is possible to store data on the cloud server via the Internet line. Compared to simple still image collection, the number of frames can be increased or the resolution can be increased. For example, even if the acquisition of an image of 1 frame per second with the I frame alone has been the limit, it is possible to acquire an image of 3 frames per second by using the I frame and the W frame. Can be raised. When the resolution is increased under the same conditions, a delay occurs in the first image, but after that, the difference becomes a difference and the capacity decreases, so that the on-time can be kept.
　カメラから端末へデータを送信する際にパケット紛失が起きにくく、データ転送量が少なくて済む。したがって、インターネット回線（ＷＡＮ）越しのクラウドサーバでのデータ蓄積が可能となる。 When transmitting data from the camera to the terminal, packet loss is unlikely to occur and the data transfer amount is small. Therefore, data can be stored in the cloud server over the Internet line (WAN).
　なお、実施形態は、上記実施の形態に限定されず、その発明の趣旨を逸脱しない範囲で種々と変形実施が可能である。また、上記各実施の形態の構成要素を発明の趣旨を逸脱しない範囲で任意に組み合わせることができる。

  The embodiment is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the invention. In addition, the constituent elements of the above embodiments can be arbitrarily combined without departing from the spirit of the invention.

Claims (7)

  1. 　第１の全画像と、前記第１の全画像の後に取得した第２の全画像に基づき、前記第１の全画像と前記第２の全画像とが相違しないエリアをマスクエリアとした第１の差分画像を作成するステップ、
    　前記第１の差分画像のマスクエリア以外については第２の全画像の対応部分とし、第１の差分画像のマスクエリアについては透明色情報を設定した第２の差分画像を作成するステップ、
    　を含む差分画像作成方法。     Based on a first all image and a second all image acquired after the first all image, a first area in which the first all image and the second all image are not different is defined as a mask area. Creating a difference image of
    Creating a second difference image in which transparent color information is set for the mask area of the first difference image, except for the mask area of the first difference image, corresponding to the second whole image;
    A difference image creation method including:
  2. 　請求項１記載の差分画像作成方法によって作成された差分画像の復元方法であって、前記第１の全画像に前記第２の差分画像を合成するステップを含む画像復元方法。 A method for restoring a difference image created by the method for creating a difference image according to claim 1, further comprising a step of synthesizing the second difference image with the first whole image.
  3. 　第１の全画像と、前記第１の全画像の後に取得した第２の全画像に基づき、前記第１の全画像と前記第２の全画像とが相違しないエリアをマスクエリアとした第１の差分画像を作成する第１の差分画像作成手段と、
    　前記第１の差分画像のマスクエリア以外については第２の全画像の対応部分とし、第１の差分画像のマスクエリアについては透明色情報を設定した第２の差分画像を作成する第２の差分画像作成手段と、
    　を備える差分検出装置。     Based on a first all image and a second all image acquired after the first all image, a first area in which the first all image and the second all image are not different is defined as a mask area. First difference image creation means for creating a difference image of
    Other than the mask area of the first difference image, it is a corresponding part of the second whole image, and the second difference image for creating the second difference image in which transparent color information is set for the mask area of the first difference image Image creation means;
    A difference detection apparatus comprising:
  4. 　動画を撮影するカメラ群及び／又は静止画を撮影するカメラ群の各カメラに接続され、ネットワークを介して画像復元装置に接続されており、
    　一定時間毎又は前記画像復元装置からの全画像要求を受信する毎に、前記カメラから全画像を前記第１の全画像として取得し、前記画像復元装置に前記第１の全画像を転送する全画像転送手段と、
    　前記一定時間中の所定時間毎又は前記画像復元装置からの差分画像要求を受信する毎に、前記カメラから全画像を前記第２の全画像として取得する差分画像用全画像取得手段と、
    　を更に備え、
    　前記第１の差分画像作成手段は、前記全画像転送手段で取得された前記第１の全画像と前記差分画像用全画像取得手段で取得された前記第２の全画像とが相違しないエリアをマスクエリアとした第１の差分画像を作成し、
    　前記第２の差分画像作成手段は、前記第１の差分画像のマスクエリア以外については、前記差分画像用全画像取得手段で取得された前記第２の全画像の対応部分とし、前記第１の差分画像のマスクエリアについては透明色情報を設定した第２の差分画像を作成し、
    　前記画像復元装置に前記第２の差分画像を転送する差分画像転送手段を更に備える請求項３に記載の差分検出装置。     It is connected to each camera of the camera group that shoots moving images and / or the camera group that shoots still images, and is connected to an image restoration device via a network.
    Every time a constant image is received or every time a request for all images from the image restoration device is received, all images are acquired from the camera as the first all images, and all the first images are transferred to the image restoration device. Image transfer means;
    A differential image full image acquisition means for acquiring a full image from the camera as the second full image every predetermined time in the fixed time or each time a differential image request is received from the image restoration device;
    Further comprising
    The first difference image creating means includes an area where the first all images acquired by the all image transfer means and the second all images acquired by the difference image all image acquiring means are not different. Create a first difference image as a mask area,
    The second difference image creating means, except for the mask area of the first difference image, is a corresponding portion of the second all image acquired by the difference image all-image acquiring means, and the first difference image creating means For the mask area of the difference image, create a second difference image with transparent color information set,
    The difference detection device according to claim 3, further comprising difference image transfer means for transferring the second difference image to the image restoration device.
  5. 　請求項３に記載の差分検出装置によって作成された差分画像の復元方法であって、前記第１の全画像に前記第２の差分画像を合成する差分画像変換手段を備える画像復元装置。 4. A method for restoring a difference image created by the difference detection device according to claim 3, further comprising difference image conversion means for synthesizing the second difference image with the first all images.
  6. 　動画を撮影するカメラ群及び／又は静止画を撮影するカメラ群と、前記カメラ群のカメラに接続された差分検出装置と、前記差分検出装置にネットワークを介して接続された画像復元装置と、を有する監視カメラシステムにおける監視方法であって、
    　一定時間毎又は前記画像復元装置からの全画像要求を受信する毎に、前記差分検出装置が、前記カメラから全画像を第１の全画像として取得し当該取得した全画像を前記画像復元装置に前記第１の全画像を転送する全画像転送ステップを含み、
    かつ、
    　前記一定時間中の所定時間毎又は前記画像復元装置からの差分画像要求を受信する毎に、前記差分検出装置が、前記カメラから全画像を第２の全画像として取得する差分画像用全画像取得ステップと、
    　前記差分検出装置が、前記第１の全画像と前記第２の全画像とが相違しないエリアをマスクエリアとした第１の差分画像を作成する第１の差分画像作成ステップと、
    　前記差分検出装置が、前記第１の差分画像のマスクエリア以外については、前記差分画像用全画像取得ステップで取得された第２の全画像の対応部分とし、前記第１の差分画像のマスクエリアについては透明色情報を設定した第２の差分画像を作成する第２の差分画像作成ステップと、
    　前記差分検出装置が、前記画像復元装置に前記第２の差分画像を転送する差分画像転送ステップと、
    　の各ステップを順に繰り返すことを含み、
    かつ、
    　前記画像復元装置が、前記差分検出装置から取得した全画像を保存する全画像保存ステップと、
    　前記画像復元装置が、前記差分検出装置から取得した差分画像をその一つ前の時点の全画像に合成する差分画像変換ステップを含む監視方法。     A camera group for capturing a moving image and / or a camera group for capturing a still image, a difference detection device connected to a camera of the camera group, and an image restoration device connected to the difference detection device via a network, A surveillance method in a surveillance camera system comprising:
    The difference detection device acquires all images from the camera as a first all image every fixed time or every time a request for all images from the image restoration device is received, and the obtained all images are sent to the image restoration device. A full image transfer step of transferring the first full image,
    And,
    The difference image all-image acquisition in which the difference detection device acquires all images from the camera as a second all image every predetermined time in the fixed time or every time a difference image request is received from the image restoration device. Steps,
    A first difference image creating step in which the difference detection device creates a first difference image using a mask area as an area where the first all images and the second all images are not different from each other;
    The difference detection device, except for the mask area of the first difference image, corresponds to the second all image acquired in the difference image all image acquisition step, and the mask area of the first difference image A second difference image creating step for creating a second difference image in which transparent color information is set;
    A difference image transfer step in which the difference detection device transfers the second difference image to the image restoration device;
    Including repeating each step of
    And,
    The image restoration device stores all images acquired from the difference detection device, and a whole image storage step,
    A monitoring method including a difference image conversion step in which the image restoration device combines a difference image acquired from the difference detection device with all images at the previous time point.
  7. 　前記全画像転送ステップにおいて、前記第１の全画像を記憶装置に保存し、
    　さらに、前記全画像転送ステップ及び／または前記差分画像用全画像取得ステップにおいて、前記差分検出装置及び／または前記録画サーバは、前記カメラから全画像を取得した後、当該取得した全画像と前記記憶装置に保存されている全画像と間で画像の色構成を比較し、光量変化によるノイズを除去する請求項６に記載の監視方法。

          In the all image transfer step, the first all images are stored in a storage device;
    Further, in the all image transfer step and / or the difference image all image acquisition step, the difference detection device and / or the recording server acquires all images from the camera, and then stores the acquired all images and the storage. The monitoring method according to claim 6, wherein the color configuration of the image is compared with all the images stored in the apparatus, and noise due to a change in the amount of light is removed.

PCT/JP2017/008397 2016-05-25 2017-03-03 Difference image generation method, image restoration method, difference detection device, image restoration device, and monitoring method WO2017203789A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2016103896A JP6363130B2 (en) 2016-05-25 2016-05-25 Surveillance method, difference image creation method, image restoration method, and difference detection apparatus in surveillance camera system
JP2016-103896 2016-05-25

Publications (1)

Publication Number Publication Date
WO2017203789A1 true WO2017203789A1 (en) 2017-11-30

Family

ID=60411282

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/008397 WO2017203789A1 (en) 2016-05-25 2017-03-03 Difference image generation method, image restoration method, difference detection device, image restoration device, and monitoring method

Country Status (3)

Country Link
JP (1) JP6363130B2 (en)
TW (1) TW201742450A (en)
WO (1) WO2017203789A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110198475A (en) * 2018-11-09 2019-09-03 腾讯科技（深圳）有限公司 Method for processing video frequency, device, equipment and computer readable storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0916254A (en) * 1995-07-04 1997-01-17 Mitsubishi Heavy Ind Ltd Plant abnormality detecting device
JP2003299088A (en) * 2002-04-01 2003-10-17 Fuji Photo Film Co Ltd Image transmitting/receiving system, image transmitting apparatus and image receiving apparatus
JP2010141599A (en) * 2008-12-11 2010-06-24 Canon Inc Information processor and information processing method
JP2015008412A (en) * 2013-06-25 2015-01-15 グローリー株式会社 Moving body image extraction method and moving body image extraction device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0916254A (en) * 1995-07-04 1997-01-17 Mitsubishi Heavy Ind Ltd Plant abnormality detecting device
JP2003299088A (en) * 2002-04-01 2003-10-17 Fuji Photo Film Co Ltd Image transmitting/receiving system, image transmitting apparatus and image receiving apparatus
JP2010141599A (en) * 2008-12-11 2010-06-24 Canon Inc Information processor and information processing method
JP2015008412A (en) * 2013-06-25 2015-01-15 グローリー株式会社 Moving body image extraction method and moving body image extraction device

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110198475A (en) * 2018-11-09 2019-09-03 腾讯科技（深圳）有限公司 Method for processing video frequency, device, equipment and computer readable storage medium
CN110198475B (en) * 2018-11-09 2022-02-25 腾讯科技（深圳）有限公司 Video processing method, device, equipment, server and readable storage medium

Also Published As

Publication number Publication date
JP2017212566A (en) 2017-11-30
TW201742450A (en) 2017-12-01
JP6363130B2 (en) 2018-07-25

Similar Documents

Publication Publication Date Title
JP5854194B2 (en) Surveillance camera system and monitoring method
JP6238255B2 (en) Surveillance method and video segmentation apparatus using surveillance camera system
CN110519616B (en) Video distribution method, distribution node, scheduling center and storage medium
JP2010154377A (en) Content sharing system and method
JP6363130B2 (en) Surveillance method, difference image creation method, image restoration method, and difference detection apparatus in surveillance camera system
WO2017169149A1 (en) Surveillance camera system and surveillance method
RU2697702C1 (en) Image compression method, image recovery method, image compression device and image recovery device
JP2015056788A (en) Image monitoring system and image transmission method
JP6446006B2 (en) Surveillance camera system, moving image browsing method and moving image combination method in surveillance camera system
JP2018174573A (en) Monitoring method, difference image creating method, image restoring method, and difference detecting apparatus in monitoring camera system
WO2021095598A1 (en) Information processing device, information processing method, information processing program, terminal device, terminal device control method, and control program
JP6284172B1 (en) Screen image transfer method, image restoration method, screen image transfer system, image restoration system, screen image transfer program, image restoration program, image compression method, image compression system, image compression program
US11190812B2 (en) Communication apparatus, control method for controlling the same, and recording medium for the same
JP6357188B2 (en) Surveillance camera system and surveillance camera data storage method
JP5896438B2 (en) Surveillance camera system and monitoring method
WO2018123078A1 (en) Monitoring camera system
WO2021117679A1 (en) Terminal device, control program, information processing device, information processing program, and information processing system
JP2018110404A (en) Screen image transfer method, image decompression method, screen image transfer system, image decompression system, screen image transfer program, image decompression program, image compression method, image compression system, and image compression program

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17802391

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 17802391

Country of ref document: EP

Kind code of ref document: A1