GB2525466A

GB2525466A - Imaging system and method

Info

Publication number: GB2525466A
Application number: GB1501661.1A
Authority: GB
Inventors: Michael Faulks
Original assignee: LIVE LINK CAMERA Ltd
Current assignee: LIVE LINK CAMERA Ltd
Priority date: 2014-03-06
Filing date: 2015-02-02
Publication date: 2015-10-28
Also published as: GB201501661D0; GB201403964D0

Abstract

An image transmission device comprising an input for receiving captured video image data, wireless data communication over a over a mobile phone network 300, the transmission of the image data in real time over the data connection to a remote server 318, the calculation or receipt of data representative of the latency and / or speed of the data connection 310, 312, and the adjustment of the amount and / or quality of image data 314 to be transmitted over the data connection depending on the calculated latency and / or speed. Data packets may be sent and received from the remote server and the time taken for the transmission and response from the server recorded and used to calculate the latency of the network 310. A modem may be used to establish a wireless data connection and may provide data of the connections type and / or speed 312. The image data may be converted into and transmitted as still frame images. Another invention relates to an image transmission system where the image data is transmitted as individual still image frames.

Description

IMAGING SYSTEM AND METHOD

This invention relates generally to an imaging system and method and, more particularly, to a method and system for streaming live image data to a central server for live viewing and/or storage for future retrieval, as required.

Video cameras are in prevalent use in many different environments and for many different purposes. An example of the prevalence of cameras is the wide usage of cameras for security by both public and private institutions.

In a scenario where an area needs to be continuously be monitored, cameras are often more suited for the job than humans, since images are recorded onto a hard medium and are viewable by others as long as the medium is maintained.

For example, closed circuit television (CCTV) surveillance systems are becoming widespread, whereby cameras are mounted at intervals within or around an area to be monitored, and video image data is transmitted back to a control centre for viewing and/or storage for future retrieval as, for example, evidence of a criminal act having been committed within the monitored area.

However, with conventional methods of CCTV installation, for example when installing CCTV cameras mounted on masts in car parks or other open outdoor sites, there is a requirement to dig up tracts of land to facilitate the laying of electrical power cables, and CCTV cables required to transmit the video data to a remote control centre, and to transmit control signals from the control centre to enable an operator to adjust the aim and the lens system of the individual cameras. This installation process is extremely disruptive to the public, time consuming and expensive. It is also a major obstacle to the expansion of CCTV security systems in the prevention of crime. In addition, each camera is usually mounted in an exposed position at the top of a specially designed mast, so that its presence is obvious to the public.

Integrated Internet cameras have been developed, which include a video output for sending a standard video signal, wherein the digital image files are transmitted as video images to the video output. In this manner, any images transmitted over the Internet or otherwise may also be supplied to, e.g., a local monitor, recording device, or CCIV network.

With a wireless internet connection, the imaging device connects via a local network to a router and the router connects to the Internet. The router is allocated a public IP address by the ISP (Internet Service Provider) and bridges that IP address to all locally connected computing devices. In this way, all local computing devices connect over the fixed IP address and are therefore connected directly to the Internet. This IP address can also be used for incoming connections which are received by the router and can then be directed to the required local computing device. Thus, a wireless CCTV or other monitoring system can be effectively configured in areas where a reliable wireless Internet connection is available for use.

With the increasing use of mobile image capturing devices, such as Internet and 3G-enabled cameras, and mobile phones and tablets which have integral image capturing functionality, it would be desirable to provide a wireless image streaming system which enables live image data to be effectively and efficiently streamed in real time via a data connection over a mobile phone network, thus enabling the system to be reconfigured and deployed as and where required, without restriction.

With a mobile phone data connection, the mobile (or data card modem) connects to the mobile phone company and the mobile phone company connects to the internet. Although the mobile phone company has a public IP address, the mobile phone device only has a local network IP address allocated by the mobile phone company's network. In this way, all mobile phone type devices connect over a variable internal IP address and are therefore not connected directly to the internet. As this IP address is a local address it cannot be used for incoming connections.

There are several consequences to this when attempting to stream live data to a data centre.

1) There is no reverse path connectivity -ie the mobile device does not have a public IP address and therefore cannot accept incoming connections from the server. Note that it is possible to purchase fixed IP SIM cards however they have limited data connectivity and are very expensive.

2) Connection times can be slow -the connection process requires a) connection to the mobile phone network b) connection to the mobile phone data network c) connection to the internet. Connection times can vary significantly and are not predictable.

3) Data transfer times can be slow -the data throughput depends on a) the type of signal ie 2G,3G,4G, b) the strength on the signal, c) how busy the mobile phone network and data networks are and d) any speed throttling imposed by the mobile phone company. This gives huge variation in speed -even during a call. Walking round a corner or behind a building can cause the signal to drop from 4G/3G to 2G.

The present invention seeks to address at least some of the issues outlined above to provide a method and system for remote imaging and image data transmission.

Thus, as stated above, one of the problems with using a mobile phone network to stream live images is the unstable and varying speed and latency of the data connection. It is obviously essential that live images are received by the control centre in real time -images which are delayed are of little value when real time action is required in response.

In accordance with a first aspect of the present invention, there is provided an image transmission device comprising an input for receiving captured video image data, means for establishing a wireless data connection over a mobile phone network, means for transmitting said image data in real time over said data connection to a remote server, means for calculating or receiving data representative of the latency of said network and/or speed of said data connection, and means for adjusting the amount and/or quality of data to be transmitted over said data connection depending on said calculated latency and/or speed thereof The device beneficially includes means configured to transmit a data packet over said data connection to said remote server, means configured to receive, from said remote server, response data transmitted in response to receipt of said data packet, and means configured to record the time elapsed between transmission of said data packet and receipt of said response data from said remote server. The device preferably comprises a clock and means configured to synchronise said clock to a clock of said remote server in response to a timing signal received therefrom.

In a first exemplary embodiment of the invention, the device may comprise means configured to calculate the latency of the network using data representative of said time elapsed between transmission of said data packet and receipt of said response data from said remote server. In an alternative exemplary embodiment of the invention, the device may comprise means configured to transmit, to said remote server, data representative of said time elapsed between transmission of said data packet and receipt of said response data from said remote server, and means configured to receive data representative of said latency of said network from said remote server.

Preferably, said means for establishing a wireless data connection over a mobile phone network comprises a modem, and said image transmission device further comprises means configured to interrogate said modem to obtain data therefrom representative of the type and/or speed of said data connection.

Beneficially, said video image data is converted into, or captured, and transmitted in the form of individual still image frames.

In order to provide the maximum amount of usable information to the central server, the first aspect of the present invention adjusts the amount of data sent from the device to the server depending on the network latency and connection speed. For example, it is better to display a low quality, smaller picture with a slow connection because this will enable a quick enough frame update for the operator at the central server to view the captured images in real time. A larger image can be sent over a slow connection, but this would take longer and would increase the time lapse between events occurring and what is seen by the operator. Thus, the device periodically sends a packet to the server and times how long it takes to receive a response, thereby enabling the network latency and connection speed to be calculated. For this to be useful, the precise timings of the outward and inward transmission of the packet and response respectively are required, which in turn requires the clocks in the camera and the server to be synchronised. Thus, the server is configured to maintain a time signal to which the device can synchronise.

In accordance with a second aspect of the present invention, there is provided an image transmission device comprising an input for receiving captured live video image data, means for establishing a wireless data connection over a mobile phone network, means for transmitting said image data in real time over said data connection to a remote server, wherein said live video image data is transmitted in the form of individual image frames.

Internet cameras, which use wireless Internet data connections, tend to record live video data, which is then compressed and transmitted as a compressed video file. However, this can have a negative effect on the quality of the image data and the present invention therefore provides much higher quality images, which may be preferable for evidence gathering, for example. Furthermore, recording individual images, rather than a video stream, allows the quality and size of each frame to be altered dynamically depending on the connection speed calculated. Thus, the transmitted images can then be displayed as they are received by the central server, in real time, and permits freezing and zooming on images with the maximum possible resolution permitted by the current connection speed. The server can be configured to re-scale the images before reassembling all the images received during a session back into a video for archive purposes.

Preferably, the image transmission device includes a global positioning system (GPS) for obtaining data representative of the absolute geographical location thereof.

In a preferred embodiment, a permanent connection is maintained between the image transmission device and the server, so that a signal can be sent to a selected image transmission device from the server to actuate image data transmission, and such image data transmission can thus be commenced substantially immediately upon receipt of the signal, thereby avoiding any unnecessary delay which would be caused by establishing the required data connection. An additional advantage of this is that the transmission devices can update the server as to their precise geographical location by periodically transmitting GPS data from their integrated GPS system.

It will be appreciated by a person skilled in the art that an image transmission device according to some exemplary embodiments of the invention could be provided as a plug-in or bolt-on accessory to a conventional portable image capture device. In this case, the image transmission device would be communicably coupled to the image capture device by, for example, its USB output, its HDMI output or via Wi-fi, for example. For image capture devices that only have SD cards, a SD-Wi-fi card could be provided. However, in alternative exemplary embodiments, the image transmission device could be integrated into the portable image capture device and the present invention extends to a portable image capture device in which the image transmission device is integrated therein, and configured to feed captured video image data to the input of the image transmission device.

The present invention extends further to an imaging system comprising one or more image transmission devices in accordance with the first or second aspect of the invention, and a server for receiving image data from said one or more image transmission devices, wherein said server is configured to receive location data from said one or more image transmission devices representative of the absolute location(s) thereof, and to display data representative of said location data.

The server beneficially comprises means configured to enable a user to select one or more of said image transmission devices, and transmit a signal thereto to initiate transmission of image data therefrom.

The present invention extends still further to an imaging system comprising one or more image transmission devices in accordance with the first or second aspect of the invention, either integrated in or connected to a respective image capture device, and a server for receiving image data from said one or more image transmission devices, wherein said server is configured to support Internet connectivity to enable connection thereto, via the Internet, by a remote computing device for viewing said image data.

These and other features and advantages of the present invention will become apparent to a person skilled in the art from the following exemplary embodiments which is described below, by way of examples only and with reference to the accompanying drawings, in which: Figure la is a schematic block diagram of an imaging system according to an exemplary embodiment of the present invention; Figure lb is a schematic block diagram of an image transmission device according to an exemplary embodiment of the invention; Figure 2 is a schematic block diagram of an image capture device according to an exemplary embodiment of the present invention; Figure 3 is a schematic block diagram illustrating some of the principal image streaming functionality of a processor for the device of Figure 2; Figure 4 is a flow diagram illustrating some of the principal steps of an image streaming function performed within the system of Figure 1; Referring to Figure 1 a of the drawings, an imaging system according to an exemplary embodiment of the present invention comprises a plurality of mobile communications devices 10 which include respective video image capture devices incorporating or connected to respective image transmission devices. The system further comprises a central remote server 12 which is configured to support Internet connectivity, thus allowing access to image data stored thereon from a remote computing terminal via a conventional web browser 14.

In use, live video images are processed within an image transmission device integrated within, or connected to, the image capture device 10 and transmitted as individual image frames to the server 12 via a data connection over a mobile phone network.

The image frames received by the server 12 can be displayed in real time for viewing by an operator, and also reassembled into an AVI movie for storage and future retrieval, if required. The server provides a geographical indication of all connected image capture devices within an area, thus enabling an operator to select one or more of the devices from which to receive image data at any time, depending on what is required to be viewed.

If stored image data is required to be accessed remotely, this can be achieved over the Internet by means of the web browser on a remote computing terminal.

Referring to Figure 2 of the drawings, a video image capture device according to an exemplary embodiment of the present invention comprises a mobile phone data connection interface 102, including a SIM card, a display 103, a camera lens and image capture circuitry 104, a processor 106, and a GPS module 108. All of these features may be provided as an integral part of a mobile communications or computing device, or within a dedicated unit, as required by the application of the system.

The functionality of the data connection interface 102, the camera lens and image capture circuitry 104 and the GPS module 108 will be well understood by a person skilled in the art, and will not be described further herein. The present invention is not intended to be in any way limited by the precise nature of this functionality. Indeed, referring to Figure lb of the drawings, the image capture device may be a conventional camera to which is connected an image transmission device 210 including a processor board and software 211, a wi-fi interface 212, a modem and antenna module 213 and a battery 214. The device 214 may also include a charger input 215. The image transmission device 210 would be communicably coupled to an image capture device (not shown) by, for example, its LJSB output, its HDMI output or via Wi- fi, for example. For image capture devices that only have SD cards, a SD-Wi-fi card could be provided.

Referring to Figure 3 of the drawings, the processor 106 of the video image capture device 100 or the processor 211 of the image transmission device 210, includes a module 200 for calculating network latency and connection speed (or receiving data representative thereof), an image processing module 202 for receiving captured live image data and converting it into individual high resolution image frames, and a data processing module 204 for adjusting the amount and/or quality of data in each image frame prior to transmission thereof via the data connection interface over a mobile phone network connection.

Referring to Figure 4 of the drawings, the processing steps performed by the processor in order to facilitate live image "streaming" from an image capture device 10 (or an image transmission device 210) to the server 12 will now be described.

At step 300, a data connection is established by a user between the device 10, 210 and the server 12 by dialling (or causing the device to dial) a predefined service centre number. In response, the device connects to the data network and, when the call is connected, the device is connected to the server at step 302. At step 304, periodically, for example every 5 seconds, the GPS position of the device is obtained from the GPS module and data representative thereof is transmitted over the data connection to the server.

This occurs for the duration of maintenance of the data connection between the device and the server, irrespective of whether image data transmission is taking place, so as to continuously provide an indication to the server of the current location of the device. In order to end the data connection, the user (or the server) must actively disconnect the call. Otherwise, it will be maintained continuously, and available for immediate use as required.

Thus, an operator at the server can be provided with a map which indicates the presence and position in an area of all image capture devices or image transmission devices which are connected to the server, and thus available for image "streaming" as required. In one configuration, the operator can cause a signal to be transmitted to any one or more selected cameras (or image transmission devices connected to cameras) to cause that device to start transmitting image data, captured at their location, to the server.

Returning to Figure 4 of the drawings, the user or the remote server may actuate image data transmission at step 306. As stated above, there are many things which can have a negative effect on the network latency and data connection speed. The data throughput depends on a) the type of signal ie 2G,3G,4G, b) the strength on the signal, c) how busy the mobile phone network and data networks are and d) any speed throttling imposed by the mobile phone company.

Thus, at step 308, when image data is required to be transmitted over the data connection, a data packet is transmitted to the server, in response to receipt of which, the server returns response data, including a timing signal which enables the device clock to be synchronised to the server clock. The processor can, therefore, calculate at step 310 the latency of the network and the end-to-end timings from the precise time elapsed between transmitting the data packet and receiving the response. In an alternative exemplary embodiment, data representative of the time elapsed between transmission of the data packet and the response from the server can be returned by the device processor to the server, where the network latency is calculated and data representative thereof is then returned to the device processor at step 310.

At step 312, the modem is interrogated for the data connection type (2G13G14G) and connection speed.

At step 314, if necessary, based on the calculated network latency and data connection speed, a new image size and quality, together with a new frame rate, is calculated in order to optimally match these properties with the network latency and connection speed previously determined, i.e. to optimise the real time transmission of the best possible quality of image data. This can be done by the processor in the device, or may again be performed by the server (in response to a timing signal from the device) and data representative of the new image size, quality and frame rate returned to the device processor.

In any event, data representative of the calculated image size, quality and frame rate is either transmitted to or received from the server, and at step 318, the resultant image frames are transmitted to the server in real time.

Steps 308 to 318 are repeated until the streaming function is terminated by the user or the server, or the "call" providing the data connection is disconnected, by the user or the server or for any other reason.

It will be apparent to a person skilled in the art that modifications and variations can be made to the described embodiments without departing from the scope of the invention as defined by the appended claims.

Claims

CLAIMSAn image transmission device comprising an input for receiving captured video image data, means for establishing a wireless data connection over a mobile phone network, means for transmitting said image data in real time over said data connection to a remote server, means for calculating or receiving data representative of the latency and/or speed of said data connection, and means for adjusting the amount and/or quality of image data to be transmitted over said data connection depending on said calculated latency and/or speed thereof.
2. A device according to claim 1, including means configured to transmit a data packet over said data connection to said remote server, means configured to receive, from said remote server, response data transmitted in response to receipt of said data packet, and means configured to record the time elapsed between transmission of said data packet and receipt of said response data from said remote server.
3. A device according to claim 2, comprising a clock and means configured to synchronise said clock to a clock of said remote server in response to a timing signal received therefrom.
4. A device according to claim 2 or claim 3, comprising means configured to calculate the latency of the network using data representative of said time elapsed between transmission of said data packet and receipt of said response data from said remote server.
5. A device according to claim 2 or claim 3, comprising means configured to transmit, to said remote server, data representative of said time elapsed between transmission of said data packet and receipt of said response data from said remote server, and means configured to receive data representative of said latency of said network from said remote server.
6. A device according to any preceding claim, wherein said means for establishing a wireless data connection over a mobile phone network comprises a modem, and said device further comprises means configured to interrogate said modem to obtain data therefrom representative of the type and/or speed of said data connection.
7. A device according to any preceding claim, wherein said video image data is converted into, or captured, and transmitted in the form of individual still image frames.
8. An image transmission device comprising an input for receiving captured live video image data, means for establishing a wireless data connection over a mobile phone network, means for transmitting said image data in real time over said data connection to a remote server, wherein said live video image data is converted into, and transmitted in the form of, individual still image frames.
9. A device according to any preceding claim, comprising GPS means for obtaining data representative of the absolute geographical location thereof, and transmitting said data to said remote server.
10. A device according to claim 9, comprising means configured to transmit said image data together with said data representative of the absolute geographical location of said device to said remote server.
11. A device according to any preceding claim, comprising means configured to establish and maintain a permanent data connection between said device and said remote server.
12. A device according to any preceding claim, configured to commence transmission of image data over said data in 1-, connection upon receipt of a signal from said remote server.
13. A device according to any of the preceding claims comprising an external input configured to be communicably coupled to an output of an image capture device.
14. An image transmission device substantially as herein described with reference to the accompanying drawings.
15. A portable image capture device comprising means for capturing video image data and having an output communicably coupled to an image transmission device according to any of the preceding claims.
16. An imaging system comprising one or more image transmission devices according to any of claims 1 to 14, and a server for receiving image data from said one or more image transmission devices, wherein said server is configured to receive location data from said one or more image transmission devices representative of the absolute location(s) thereof, and to display data representative of said location data.
17. An imaging system according to claim 16, comprising means configured to enable a user to select one or more of said image transmission devices, and transmit a signal thereto to initiate transmission of image data therefrom.
18. An imaging system according to claim 16 or claim 17, wherein said server is configured to support Internet connectivity to enable connection thereto, via the Internet, by a remote computing terminal for viewing said image data.
19. An imaging system comprising one or more image transmission devices according to any of claims 1 to 14, and a server for receiving image data from said one or more image transmission devices, wherein said server is configured to support Internet connectivity to enable connection thereto, via the Internet, by a remote computing terminal for viewing said image data.
20. An imaging system substantially as herein described with reference to the accompanying drawings.