CN113596317A

CN113596317A - Live-action shot image security method, terminal and system

Info

Publication number: CN113596317A
Application number: CN202010389796.XA
Authority: CN
Inventors: 杨筑平; 周跃平
Original assignee: Shenzhen Jinlan Hanyuan Technology Co ltd
Current assignee: Shenzhen Jinlan Hanyuan Technology Co ltd
Priority date: 2020-04-30
Filing date: 2020-04-30
Publication date: 2021-11-02
Anticipated expiration: 2040-04-30
Also published as: CN113596317B

Abstract

A method, a terminal and a system for saving live-action shot images relate to the technical field of image media, shooting equipment, network systems and electronic commerce. The method comprises the steps of shooting images in real time on a real scene and ensuring the reality and credibility of the image scene, wherein corresponding to a main frame section of the real scene shot image, a differential frame section of a differential focusing synchronous shot image is recorded as a depth of field fingerprint of the image scene, the image scene correlation with the corresponding main frame section, which is mutually verified in space and time, is existed, more than one section of depth of field fingerprint is reserved as an evidence for identifying the reality and credibility of the real scene shot image, multiple lenses can cooperate in labor, field geographic environment information is recorded in a correlated manner, and a strategy instruction determines a differential focusing strategy; the terminal realizes the method; the system provides management image evidence access service for the terminal and even executes automatic comparison and analysis of related authentication algorithms. The method and the terminal system are matched, the image shooting counterfeiting perfection trust mechanism can be broken, the real and credible identification problem of the real shooting image scene is solved, and the method is suitable for face recognition and electronic commerce.

Description

Live-action shot image security method, terminal and system

Technical Field

The invention relates to the technical field of image media, shooting equipment, network systems and electronic commerce.

Background

Images, including photographs and videos. Video is at least purely pictorial and typically also contains audio. The live-action shot image, referred to as a live-action shot image for short, is an image shot in a real scene and actually shot in the real scene, and is different from an image of an artificial scene such as a drawing, an animation, a fairy tale, a mythology, a game and the like created by human creation, and is also different from a data image created by editing and modifying mainly in a later stage. The images are typically captured in a file format such as BMP, JPG, PNG, AVI, MOV, WMV, MPEG, RM, etc. The existing image format file only has pure image meaning and has no identifiable scene-related authenticity evidence, and the existing software tools, including adopting VR (virtual reality), AR (augmented reality) and AI (artificial intelligence) high technologies, can be arbitrarily edited and modified to lose authenticity. For example, photo retouching, embellishment retouching; image tampering, namely changing heads and faces of people, wherein the image freshness is increased by opening mouths and blinking, and even peeling by one key; the image is fictitious, so that a fake and fake image can be manufactured; a phenomenon, event, occurring in one place can be said to occur in another place; the high-quality products planted and cultured in good environments can be displayed by the fake images of products in non-native areas. The image may then be used to fraud, fooling the common sense that people believe is true. People always want to face the real situation of knowing the real situation in the daily life of communication, transaction and life. Nowadays, the information society is widely developed, a large number of images are shot, the spread is wide and rapid, and a proper technical means is urgently needed to ensure the reality and credibility of the shot images. The guarantee of trueness and credibility of the image is to guarantee trueness and credibility of the image scene, for example, characters, flowers and watermelons are really in the real scene of shooting, but not endorsements of social attributes (identity, credit and the like) of the characters, the quality of the flowers (plant flowers or plastic flowers) and the planting (green or non-green) of the watermelons, which needs more non-technical concrete evidence. In the prior art, the authenticity and the reliability of the image are difficult to generally identify, and the universal automatic identification is difficult to add. The image can not be self-certified, and only can be certified by relying on the certification service of a third-party trust organization, the basic idea is to control the real image shooting process and immediately upload the image to a server for storage and fixation. However, this presents two challenges for counterfeiting: firstly, replaying a picture to shoot and make a fake image, wherein a shooter firstly makes a fake image and then plays the fake image again to shoot the fake image by using a shooting terminal for uploading; and secondly, shooting of virtual signals is fake, a photographer virtually forms an image shooting virtual terminal, virtual image source electric signals are input, and the image real shooting uploading process is controlled in a simulation mode. For the two cheating conditions, the existing server side cannot be easily known at present, so that the trust mechanism has the technical defect that the technical defect needs to be made up.

Disclosure of Invention

The invention aims to overcome the two challenge problems starting from three aspects of a shooting method, terminal equipment and system service, and break through image shooting counterfeiting, thereby overcoming the technical defects, improving the trust mechanism and solving the identification problem of trueness and credibility of an actually shot image scene.

The invention provides a real-scene shot image preservation method, which is used for shooting an image in real time in a real scene and ensuring the reality and credibility of the image scene. Taking a picture in real time, taking a picture as one image data, taking a video (video) as image frame sequence (which may contain audio) data, and taking a picture as a freeze frame in the video image frame sequence. When actually shooting an image, a photographer faces a specific subject object (a person, an animal, a scenery, and other scenes) which is mainly shot in a real scene, and determines a shooting distance (i.e., a focusing distance) of a focusing plane (a scene light point plane perpendicular to a main optical axis of a lens) by focusing (or focusing). The focusing plane may change in distance with the movement of the photographer and/or the object, and therefore needs to be focused appropriately thereto. High-grade shooting terminals (such as smart phones and digital cameras) can automatically recognize shot objects (such as faces and flowers) and automatically focus, and a photographer can manually focus (such as pointing to a certain position in a display screen). The subject frame is a video frame captured on a subject focus plane. The frame segment is a segment of a continuous frame sequence that is generated and arranged in time series, and includes one or more frames. According to the image shooting principle, the scenery within a certain depth distance range respectively in front of and behind the focusing plane can be clearly imaged, the distance in front of the focusing plane is called the front field depth, the distance behind the focusing plane is called the rear field depth, the rear field depth is larger than the front field depth, and the sum of the front field depth and the rear field depth is called the field depth. The main body of a clear image of a frame is actually the superposed image of all visible scenery light spots constrained by the aperture within the depth of field space range of the focusing plane. The existing mathematical formula for calculating the depth of field is known, and the numerical value of the depth of field is related to the allowable circle of confusion diameter, the focal length of the lens, the shooting aperture value of the lens and the focusing distance. The differential focusing is focusing performed on a focusing plane (referred to as a differential focusing plane) divided by determining a focusing distance by setting a distance difference increment based on the focusing distance of the subject focusing plane.

When a certain main frame section of an image is actually shot, a differential focusing evidence obtaining process is started, a differential focusing plane is used as a reference and is related to the main focusing plane, the differential focusing plane longitudinally passes through and can exceed the foreground depth and/or the back depth of field of the main focusing plane, the image frame is synchronously shot at a specific focusing moment to serve as a differential frame of an image scene, and a differential frame sequence shot in a focusing change period is the differential frame section. The differential frame segments need to be tagged, including the start position (relative time scale or frame sequence number) and duration (time scale or frame number) of the corresponding body frame segment in time sequence, to enable retrieval of the associated playback. The subject focus plane and the differential focus plane have respective depths of field (including foreground depth and background depth), and the depths of field of the two focus planes are generally cross-correlated. In the space range with the overlapped depth of field, all the scenery can be clearly imaged on the main frame and the differential frame. In terms of visual effect, in the rear depth of field spatial range of the main body focusing plane, when the differential focusing plane gradually zooms away from the main body focusing plane in a distant direction along a photographer until the differential focusing plane exceeds a rear boundary of the rear depth of field (or even the depth of field of the differential focusing plane is separated from the rear depth of field of the main body focusing plane), the image of the differential frame section reflects that on one hand, an originally clear scenery on the main body frame is gradually reduced to lose details and become blurred and have a hallucinogenic sense (become blurred by the clearness), and on the other hand, the originally blurred scenery on the main body frame gradually presents more details and becomes clear and enlarged and has a hallucinogenic sense (become clear by the blur); when the differential focusing plane is reversely and gradually zoomed to stride from the back depth of field of the main body focusing plane and approach to the main body focusing plane, the images of the differential frame section reversely reflect the illusion and the illusion of the scenery on the main body frame respectively. Similarly, in the foreground deep space range of the main body focusing plane, when the differential focusing plane gradually leaves the main body focusing plane along the photographer in a near direction until the differential focusing plane exceeds the front boundary of the front depth of field (or even the depth of field of the differential focusing plane can reach the front depth of field which is separated from the main body focusing plane), the image of the differential frame section reflects that on one hand, the originally clear scenery on the main body frame is gradually reduced to lose details and become fuzzy and have a sense of illusion, and on the other hand, the originally fuzzy scenery on the main body frame gradually presents more details and becomes clear and amplified and has a sense of illusion; when the differential focusing plane reversely and gradually spans from the foreground of the main body focusing plane to the main body focusing plane, the images of the differential frame section reversely reflect the illusion and the illusion of the scenery on the main body frame respectively. The difference frame image obtained by difference focusing shooting reflects the image change characteristics of different depth levels of the main frame image in the same scene actually, and has the fingerprint significance of the image scene space in the depth direction. When the frame rate of the differential frame is fast enough, one or more sequences of the differential frame, i.e. differential frame segments, can be captured for a main frame, and the differential frame segments are viewed as depth fingerprints of the image scene of the main frame. A typical example is the taking of a picture in conjunction with the taking of a depth fingerprint of a segment of the image scene. For the video image, in a certain main frame segment, the difference frame segment shot synchronously is used as the depth-of-field fingerprint of the image scene of the main frame segment. The frame rate of the depth fingerprint (i.e. the differential frame rate) may be the same (one differential frame corresponds to one main frame) or different (a plurality of differential frames correspond to one main frame, or one differential frame corresponds to a plurality of main frames), and the former is higher than the latter for easier comparison and identification. Therefore, the depth-of-field fingerprints and the subject frame segments corresponding to the covered neighbors (including the covered and the neighboring) can be mutually verified in time sequence and in the depth level of the spatial scene, the image scene correlation which can be objectively identified exists, and the depth-of-field fingerprints which are more than one segment can be used as evidence for identifying the true reliability of the real-shot image scene.

The basic technical logic concept of the invention is to record and adopt images with the same scene and different depths of field to correlate and verify the real reliability of the photographed image scene. When the real-shot image is played, the main frame section and the corresponding depth-of-field fingerprint are synchronously played in a correlated manner, and the scene vision related similarity condition of the two images is observed, compared and judged, so that the real reliability of the scene of the real-shot image can be identified. By statistically averaging (or weighting) the number of segments or frames of the depth-of-field fingerprint comparison with the related similarity, the real reliability of the scene of the real-shot image can be quantified, for example, a specific percentage value is calculated. It is noted that the aforementioned illusive-in and illusive-out visual effect of the depth of field fingerprint is essentially an optical zoom sampling zoom-out transformation, which is not exactly imaginary by performing digital scaling and even mathematical interpolation on the image pixels of the subject frame segment, and is extremely difficult and impractical to implement under the constraint of lacking depth of field background detail substitution. Therefore, the problem of image counterfeiting can be solved: firstly, for shooting and counterfeiting of a replay picture, a fake scene including a frame of a play screen can be exposed through a focusing process of changing a depth of field fingerprint, and the picture is found to lack a corresponding illusion-vision effect only through pixel scaling transformation, so that the shooting and counterfeiting of the replay picture can be known; secondly, for the details of the virtual signal shooting counterfeiting and the unreal-in and unreal-out visual effects of the depth of field fingerprints, the counterfeiting algorithm and the execution process thereof are extremely high in complexity and difficult to operate, and therefore the implementation cost and the difficulty degree of the image counterfeiting can be greatly improved.

The live-action shot image preservation method is further characterized in that the difference frame and the main body frame are generated by respective lens division cooperative parallel shooting under the condition of multi-lens shooting, and the depth-of-field fingerprints and the corresponding main body frame segments are kept consistent in time sequence and are output in a correlated mode. Under the condition of a single lens, the difference frame and the main frame are shot by the same lens, and can only be shot in a serial time-sharing mode, and the depth-of-field fingerprint and the corresponding main frame section are in a serial alternative relation and are regarded as a covering adjacency relation. When multiple lenses are used for shooting, the optical characteristics of the lenses can be used for division and cooperation, the related depth of field fingerprints are synchronously shot while the shooting of the main frame section is ensured, the depth of field fingerprints and the corresponding main frame section are in a parallel covering adjacency relation, and the time sequence consistency is kept so that related output can be realized.

The method for preserving the live-action shot image further considers that visual interference to a photographer is avoided, and is characterized in that the depth of field fingerprint is automatically and covertly recorded during shooting and is not displayed in a main visual observation window. However, this is not necessarily limiting, and the photographer may autonomously select to open or close the auxiliary window to display the depth fingerprint or to mask the display.

The method for saving the live-action shot image further considers the auxiliary evidence based on the environmental information of the shot scene, and is characterized in that the field depth fingerprints are also associated and recorded with the on-site geographic environmental information during shooting. Such geographic context information includes, but is not limited to: longitude, latitude, elevation, moving direction and moving speed of geographic position positioning; temperature, humidity of the environment; date, time of day; the motion sensor records data. The visual error caused by the shooting can be distinguished and eliminated by comparing the moving direction and the moving speed during shooting. By means of natural common sense and geographic information system, the authenticity and credibility of the photographed image can be identified with reference.

The method for preserving the live-action shot image further considers the strategy and flexibility of differential focusing, and is characterized in that the depth of field fingerprint receives a strategy instruction to determine the differential focusing strategy during shooting. Such policy instructions may come from user selection of settings, and in particular, from external communication inputs. The differential focusing strategy includes the starting time (or frame number), duration (or frame number), frame frequency, and focusing control (direction, focal length, and change) of the captured depth fingerprint.

The live-action shot image preservation method comprises the above various considerations and further considers transmission preservation, and is characterized in that the depth of field fingerprint can be instantly transmitted to a server through a network communication mode to be stored in a related mode so as to fix evidence. The centralized uploading server is used for saving, and is an effective method for preserving images and fixed evidences. The association storage means that the depth fingerprint is stored in association with the body frame segment, and since the depth fingerprint is marked at the time of shooting, the association with the body frame segment is continuously maintained. The field depth fingerprints are uploaded and stored instantly so as to guarantee fixed evidence preferentially. The body frame segment is generally also uploaded in time, but in order to ensure the response efficiency of the system, a cache lag uploading mode can also be adopted.

The invention also provides a live-action shot image security terminal which is provided with more than one lens and is internally provided with a functional module for shooting the image in real time. Examples of such live-action image security terminals include, but are not limited to: digital cameras, video cameras, surveillance cameras, cell phones (mobile phones), tablet computers, and the like.

The invention further provides a live-action image security system which has a network communication connection function and is configured with a server for transmitting and storing images, and the live-action image security system is characterized in that the system provides an image management evidence access service for the live-action image security terminal, interactively receives live-action images which are transmitted by the terminal and contain depth of field fingerprints, and secures and stores evidence at the server side. The interactive receiving process is automatically carried out on line. Basic measures for storing and fixing evidence at a server side are guaranteed, including service identity authentication, access authority control, interruption residual storage, protection period setting, and even digital signature execution on an image file containing a depth of field fingerprint to guarantee the integrity and consistency of data for subsequent exchange, transmission and verification.

The live-action shot image security system is further characterized in that the interactive receiving process also sends a strategy instruction to the terminal computer from the server side so as to determine the differential focusing strategy of the depth-of-field fingerprint when the terminal shoots, and the server side controls and checks the differential focusing strategy. The server side leads the zoom strategy when the terminal shoots to be random and unpredictable, so that the terminal can be prevented from using the focusing strategy for counterfeiting, particularly the counterfeiting caused by virtual signal shooting, and even a powerful and efficient virtual terminal is difficult to immediately reconstruct the depth of field fingerprint corresponding to the randomly specified main frame section.

The system for saving the live-action shot image is further considered to achieve the automation of identification, and is characterized in that the server further executes a relevant identification algorithm process to compare and analyze the image scene correlation between the depth-of-field fingerprint and the corresponding main frame segment, so as to automatically identify the real reliability of the live-action shot image scene. The identification algorithm needs to use an AI image recognition processing technology, and can be continuously trained and perfected to improve the accuracy and the efficiency.

The live-action shot image security system is further considered to support face recognition application, and is characterized in that the server further assists the face recognition authentication process, and the real credibility of the live-action shot image scene is used as a constraint condition of identity authentication. In order to prevent deceiving face recognition by using AI face changing, the true credibility of the identified real shot image scene is used as a constraint condition of identity authentication, if the conclusion is reliable, the face recognition result is confirmed, otherwise, the face recognition result is denied, and even the face recognition is refused to be executed.

The system for protecting the live-action shot image further considers the practical and necessary modification requirement of the image file, and is characterized in that the server further assists the terminal to carry out controlled post-editing and manufacturing on the existing live-action shot image on the premise of authorizing and protecting the image scene. For example, editing operations of synthesizing captions, dubbing music and adding titles and tails must pass authorization and cannot destroy the real credibility of the image scene.

The system for saving the live-action shot images comprises various considerations, and further mainly considers the application of the electronic commerce application, and is characterized in that the server further intervenes in the operation process of the electronic commerce, pushes and displays the live-action shot images containing the depth-of-field fingerprints or identified through the depth-of-field fingerprints to the clients, and saves endorsements for the image scenes. The client can select the real shot image containing the depth of field fingerprint to authenticate itself, or select the real shot image authenticated by the depth of field fingerprint by a trust system (a third party organization), and the system declares and guarantees that the provided real shot image is real and credible in scene. The system performs digital signatures on live image files, and is also a clear representation of the secure endorsement for image scenes.

The invention has the advantages that the live-action shot image security method is provided, the depth of field fingerprint obtained by differential focusing shooting is taken as the evidence for identifying the true credibility of the live-action shot image scene, the live-action shot image security terminal for realizing the method and the live-action shot image security system connected with the terminal and providing the management image evidence service are also provided, the two challenge problems are overcome through the cooperation of the method, the equipment and the system, the image shooting counterfeiting is broken, the trust mechanism is improved, the true credibility identification problem of the live-action shot image scene can be solved, the method is particularly suitable for the application scene of face identification and electronic commerce, and the method can be applied to the proof of other affairs (such as investigation, law, notarization and the like).

Drawings

FIG. 1 is a schematic diagram of spatial correlation of differential focusing of a real-shot image according to the present invention. The numbering names are compared and listed in Table-1.

FIG. 2 is a timing diagram of differential focusing of a real-shot image according to the present invention. The numbering names are compared and listed in Table-2.

FIG. 3 is a diagram of a differential focusing boundary according to an embodiment of the present invention.

Fig. 4 is a schematic diagram of a system configuration according to an embodiment of the present invention.

Fig. 5 is a schematic diagram of a functional module of a user terminal according to an embodiment of the present invention.

Fig. 6 is a schematic diagram of a server function module configuration according to an embodiment of the present invention.

TABLE-1 reference table for numbering names in FIG. 1

TABLE-2 number name comparison Table in FIG. 2

Numbering	Name (R)	Remarks for note
			T	Time axis
10	Real shooting image start time	First frame shooting time
			11	Differential frame start time	Shooting time of first frame in differential frame section
12	Differential frame stop moment	End frame capture time in a differential frame segment
			13	End time of real shooting image	Last frame shot time
20	First frame of real shooting image
			21	Second frame of real shot image
22	Body frame segment	Sequence of subject frames in live-action image
			220	Body frame segment header frame
221	End frame of body frame segment
			23	End frame of real shooting image
30	Differential frame segment	A sequence of differential frames corresponding to the body frame segments, i.e., a depth of view fingerprint.
			300	Differential frame segment header frame
301	Differential frame segment end frame

Detailed Description

In the first aspect, the live-action image preservation method provided by the invention is fully understood.

Referring to table 1 in fig. 1, the spatial correlation of the differential focusing of the real-shot image according to the present invention is understood. On the main optical axis (horizontal dotted line), the relationship of each element and the size chain is analyzed. The subject focus distance 100 of the subject focus plane 10 is the sum of the object distance 400 and the image distance 410; at this focus distance, the subject within the subject depth of field 11 (solid rectangular box) can be clearly imaged as a subject frame 12; the longitudinal (in the direction of the main optical axis) depth of the subject depth 11 includes and is the sum of the subject front depth 110 and the subject rear depth 111. The focus distance of the differential focus plane (front) 20 is forward on the basis of the subject focus distance 100, i.e., the differential focus distance (front) increment 200; at this focus distance, the scene within the differential depth of field (front) 21 (dashed rectangle) can be clearly imaged as a differential frame (front) 22; the longitudinal depth of the differential depth of field (front) 21 also includes a front depth of field and a back depth of field (not shown). The focus distance of the differential focus plane (back) 30 is set back on the basis of the subject focus distance 100, i.e., the differential focus distance (back) increment 300; at this focus distance, the scene within the differential depth of field (back) 31 (dashed rectangle) can be clearly imaged as a differential frame (back) 32; the longitudinal depth of the differential depth of field (rear) 31 also includes a front depth of field and a rear depth of field (not shown). The foregoing three depths of field are depicted as rectangular boxes only to facilitate understanding of their extent and their mutual spatial positional relationship, with the upper and lower boundaries being effectively limited to the field angle of the lens. It will be readily appreciated that after the subject focus plane 10 is set, the differential focus plane 20/30 can be adjusted longitudinally (as indicated by the horizontal double arrow), either forward or backward, with the visual effect that as long as the depths of field of the two intersect: scenes in the overlapping portion of space that are clearly imaged in both the subject frame 12 and the differential frame 22/32; the scenes in the non-overlapped part space are imaged in the differential frame 22/32 to be blurred (lack of detail) if the images in the main frame 12 are sharp (detail is clear), and are imaged in the differential frame 22/32 to be sharp if the images in the main frame 12 are blurred; moreover, the difference in focal distance is also reflected in the change in the imaging scale of the scene. Therefore, there is a spatial mutual-corroborative image scene correlation between the differential frame and the main frame. Looking at the interior of the shooting terminal on the right side of the figure, the main body focusing distance 100 of the main body focusing plane 10 can be changed by moving the lens back and forth, and the increment 200/300 is added to the focusing distance of the differential focusing plane 20/30; presetting the diameter 500 of the imaging diffusion circle on the photosensitive film 50, and changing the focal depth 510 by adjusting the focal length and the aperture (not shown) of the lens 40, including a front focal depth 511 and a rear focal depth 512; thereby determining a subject depth of field 11, including a subject front depth of field 110 and a subject back depth of field 111, and also determining a differential depth of field 21/31; the light 60 emitted from the scene point on the main body focusing plane 10 is focused on the focal plane 41 through the lens 40 and imaged on the photosensitive film 50; the subject frame 12 and the differential frame 22/32 are both images on the film 50, but in the single-shot condition they can only be imaged asynchronously (time-shared) on the same film, whereas in the multi-shot condition they can be imaged synchronously (simultaneously) on different films.

Referring to table 2 in fig. 2, the timing relationship of the differential focusing of the live-shot image of the present invention can be understood. A real-shot image frame sequence is drawn below the time axis T as a boundary; the corresponding depth fingerprint is shown above, and although only one segment is shown, it should be understood that the depth fingerprint may have multiple segments, each of which is a difference frame segment. Generating a first frame 20 of a real image, a second frame 21 of the real image, and a subsequent frame sequence of a subject frame, a section 22 of the subject frame at a time 11, and a subsequent frame sequence at a predetermined frame rate from a start time 10 of the real image, and generating a last frame 23 of the real image until an end time 13 of the real image; the body frame segment 22 includes a body frame segment first frame 220 and a subsequent body frame sequence until the body frame segment end frame 221 at time 12. From the differential frame start time 11, a differential frame segment 30 is generated, which includes a differential frame segment first frame 300 and a subsequent differential frame sequence, until the differential frame stop time 12, a differential frame segment last frame 301 is generated. In terms of time sequence, the differential frame segment 30 and the main frame segment 22 are generally segmented and correspond to each other, specifically: when the frame frequencies of the two frames are the same, the two frames correspond to one-to-one frames; when the frame rates are different, one-to-many or many-to-one frames correspond. Therefore, there is a temporal mutual verified image scene correlation between the difference frame segment and the main frame segment.

Therefore, the depth of field fingerprint and the corresponding body frame segment have the image scene correlation of space-time mutual evidence, and more than one segment of depth of field fingerprint is reserved to be used as evidence for identifying the real reliability of the real shot image scene.

In a second aspect, a system configuration of an embodiment of the present invention is planned.

See fig. 4. The user terminal 1 draws only one, actually more than one, and it includes: a recording unit 11, a playing unit 12, a display 13, a motion sensing unit 14 and an application unit 15. The camera unit 11, which expresses the basic live-action image security method of the present invention, inputs images and positioning data, which are obtained real-time data, and outputs a data stream of live-action images and depth fingerprints (including the real-time positioning data), which includes associated logic between them, and uploads the data stream to the server 2 through a communication network (indicated by a right arrow), and can also receive a policy instruction (indicated by a left arrow) from the server 2 to determine a differential focusing policy during shooting; a playing unit 12 for receiving the real-time audio/video file 3 from the server 2; a display 13, including a playing window 131 and a map window 132, for respectively presenting the image and the locus track on the map associated with the identifier; the playing unit 12 and the display screen 13 are combined to realize image positioning playing, including the selectable playing of depth-of-field fingerprints; the motion sensing unit 14 is an auxiliary unit, and is indicated by a dashed line box as optional, and can provide emergency event data, start the shooting by the shooting unit 11, and drive the application unit 15 to send an application message (e.g., alarm) to the server; the application unit 15 implements the functions of the system applied to the user terminal (e.g. e-commerce). The server 2 includes: an image library unit 21, a user registry 22, an intelligent analysis unit 23 and an application unit 24. The image library unit 21 receives the live image and depth fingerprint data stream transmitted from the user terminal 1, stores the data stream as a set of live image files 211/212, sends a policy instruction to the shooting unit 11 of the terminal 1, and sends a live image file 3 in response to the playing unit 12; a user registry 22 for registering information registered by the user terminal 1 for receiving the service; the intelligent analysis unit 23 executes an artificial intelligent correlation identification algorithm process in the background, compares and analyzes the image scene correlation between the depth-of-field fingerprint and the corresponding body frame section for the real shot image and the depth-of-field fingerprint data stream or the real shot image file transmitted from the user terminal 1, and automatically identifies the real reliability of the real shot image scene; the application unit 24 serves the user in response to the notification of the intelligent analysis unit 23 or the reception of the message transmitted from the user terminal 1. The real-shot image file 3 includes: video file 31, and associated file 32. The video file 31 stores digitized data of a video, and is image coded data for a photograph and a frame coded (which may include audio coded) data sequence for a video; the data content stored by the associated file 32, including the depth of view fingerprint 321, the positioning data 322, and the digital signature 323; a depth-of-field fingerprint 321 storing the differential focusing frame segment and the corresponding relationship with the main frame segment; the positioning data 322 stores the positioning coordinates of the geographic position where the image is shot, wherein the image is a place for the picture, and the video is a place sequence of a moving track in the shooting process; the digital signature 323 stores digital signature data for the video file 31 and digital signature data for the associated file.

The embodiment of the invention is realized by expanding and improving the existing cloud storage service platform, and is called as real-time image shooting service. The cloud storage service platform has mature scale commercial application, such as a hundred-degree cloud disk, and generates a place function for uploaded photos, but only generates a city classification mark according to a registration place of an uploading source, and the cloud storage service platform is not a real-time shooting service and has no identification function of truthful and credible image scenes.

In a third aspect, the live-action image security terminal provided by the invention is implemented.

The existing high-grade mobile phone (such as P40) is adopted as the terminal equipment for live-action image shooting, and the terminal equipment is provided with a plurality of cameras, so that the aim of comprehensively improving the shooting performance and quality is originally fulfilled. The following two steps of design development are required.

Firstly, modifying a mobile phone operating system. The Android operating system is open-source and can be modified as required. Accordingly, modifications need to be made at two levels: the first layer is on the device driving layer of the kernel, modifies a camera (lens) driving program, on the basis of keeping the conventional shooting driving, expands and supports differential focusing shooting, optimizes multi-lens resource combination, drives at least one lens with optical zooming performance for differential focusing shooting, and ensures that the sensitization generation of the main frame image and the differential frame image is synchronous (namely, simultaneous generation). The camera driving program can accurately drive the lens motor to rotate and move to a position according to a given image distance so as to support shooting of different object distances or focusing distances. And the second layer is that the function of image shooting and a calling interface thereof are modified on a basic service layer and/or a program frame layer, on the basis of keeping the calling of the conventional shooting function, the calling of the differential focusing shooting function is expanded and supported on the basis of a differential focusing shooting drive, and the input parameters comprise a main focusing distance and a differential focusing distance increment for calling an upper application program. Function calling is realized according to given object distance and the following image distance calculation formula

And calculating the image distance, and calling a camera driver to perform focusing shooting. The upper image distance calculation formula is composed of the following Gaussian imaging formula

The result is deduced. Where u is the object distance, v is the image distance, and f is the focal length.

In the second step, a mobile phone Application (APP), called a real-time camera, is created, which is described with reference to fig. 5 and 4. The user terminal 1 is the APP, wherein the functional module, real shooting 11, includes shooting 111 and shooting 112, and implements the shooting unit 11 in fig. 4; a live broadcast 12 implementing the play unit 12 and the display 13 in fig. 4; an application 13 implementing the application unit 15 in fig. 4; a transmission 14 for realizing information communication transmission between the user terminal 1 and the server 2 in fig. 4; and setting 15, configuring the terminal operating environment and the control strategy. The main functional modules are implemented as follows.

The real shot 11 prompts two menu options: photographing and shooting (namely shooting video). And selecting and executing the corresponding function according to the user operation. The function module comprises two sub-modules, wherein the two sub-modules are used for measuring the focus distance of a shot object by infrared rays and then calculating the formula according to the depth of field

And calculating the numerical value of the depth of field. Where δ is the allowable circle diameter, F is the lens focal length, F is the shooting aperture value of the lens, L is the focus distance, Δ L1 is the foreground depth, Δ L2 is the back depth of field, and Δ L is the depth of field. As shown in fig. 1, the focus distance 100 is the sum of the object distance 400 and the image distance 410, i.e., L ═ u + v.

See fig. 3, to determine the differential focus limits. To maintain the image scene correlation of the differential frame and the subject frame, the distance between the differential focus plane (front/back) 20/30 and the subject focus plane 10, i.e. the differential focus distance increment (front/back) 200/300, should be the maximum absolute value when they are in series with the depth of field of the three frames. As can be seen from the above depth-of-field calculation formula, the subject depth of field 11 and the differential depth of field (front/rear) 21/31 are not equal to each other due to the difference in focal distance. To improve efficiency and simplify the calculation, only the value Δ L of the subject depth of field 11 is calculated, and the value of the differential depth of field (front/rear) 21/31 is replaced with Δ L approximation as the maximum absolute value of the differential focus distance increment. Then, taking the subject focusing distance L as a base point 0, the differential focusing limit is two half-open range sections: [ - Δ L, 0) and (0, Δ L ]. The value range intervals of the differential focusing distance Lx are respectively as follows: [ L-. DELTA.L, L ] and (L, L +. DELTA.L ].

Photographing 111: the functions of an operating system are called, photo image data are read and generated, GPS/Beidou positioning data are read, and a data stream of real photographing pictures and depth of field fingerprints including the positioning data is combined. In order to ensure the picture shooting quality, the original shooting function is still adopted for calling, in the selected focusing distance, when a shutter (which can be a touch screen action) is pressed, a stop-motion picture is firstly shot, then according to the differential frame number and the differential focusing increment sequence determined by the strategy instruction, the original shooting function is called for shooting to asynchronously generate a differential frame sequence, namely a depth of field fingerprint, corresponding to one picture.

Image pickup 112: and calling an operating system function, periodically (the frequency is determined by the frame frequency) reading and generating video image frame data, and periodically (the frequency is determined by a preset positioning frequency, such as 1 time/second) reading GPS/Beidou positioning data to combine into a data stream of a real shot image and a depth of field fingerprint which comprise positioning data. And executing the differential focusing shooting function call to synchronously generate a differential frame sequence, namely a depth of field fingerprint, according to the differential frame starting time, the differential frame stopping time (the differential frame number is also determined) and the differential focusing increment determined by the strategy command at the same frame frequency.

And (5) real broadcasting 12: opening an image file and a follow file in the live image file, checking the integrity and the validity of the live image file, including verifying a digital signature, and analyzing a depth of field fingerprint and positioning data from the follow file; opening a playing window, playing an image by using the existing playing plugin, selectively opening a non-main window to be related to play the depth of field fingerprint and compare and check the depth of field fingerprint, opening a map window in a related manner, and marking a positioning point track by using the map plugin; responding to and processing external pause, play, rewind, fast-forward, slow-forward and skip-forward operations, and adjusting image play and map positioning according to the change of scale positions on a time axis; and operations such as zooming, steering, showing and hiding, separating, superposing and the like of the window are processed, and image playing and map positioning are adjusted in response.

Application 13: the basic application is cloud storage, extensible electronic commerce, face recognition and the like.

And (4) transmission: and realizing a trusted communication protocol, and connecting and transmitting between the terminal and the server. The transmission content comprises the following steps: the method comprises the steps of actually shooting images, a depth-of-field fingerprint data stream (containing positioning data), strategy instructions, application messages and management information.

Setting 15: setting application control parameters, including setting policy instructions, but when receiving the policy instructions sent by the server, controlling the policy instructions in priority by the latter.

In a fourth aspect, the present invention provides a live-action image security system.

The module functions of the server are defined as explained with reference to fig. 6 in relation to fig. 4. Server 2 is a corresponding implementation of server 2 in fig. 4. The functional module transmits 21 a communication protocol for realizing connection between the server and the terminal; a storage 22 and a signature verification 23 for managing the image file library unit 21 in fig. 4; an intelligent analysis 24 implementing the intelligent analysis unit 23 in fig. 4; an application 25 implementing the application unit 24 in fig. 4; a user registry 26, implementing the user registry 22 management in FIG. 4; the service management 27 manages the policy parameters for the operation of the system. The main functional modules are implemented as follows.

And (3) transmission 21: and realizing a trusted communication protocol, and connecting and transmitting between the server and the terminal. The transmission content comprises the following steps: the method comprises the steps of actually shooting images, a depth-of-field fingerprint data stream (containing positioning data), strategy instructions, application messages and management information.

The storage 22: storing the live-shooting image and the depth-of-field fingerprint data stream (containing positioning data) received from the terminal into live-shooting image files, wherein each live-shooting image file comprises an image file in a standard format and a follow-up file in a self-defined format, and the live-shooting image files and the follow-up file are related and brought into an image library for management; or retrieving files from the library, extracting and generating a live image and a depth of field fingerprint data stream, and sending the live image and the depth of field fingerprint data stream to the user terminal; and the user can decide and operate to execute positioning data desensitization elimination on the designated positioning real shot image file. The user access operation needs to pass identity authentication. The system implements a mandatory protection period for the real-shot image files stored in the library, and the mandatory protection period cannot be deleted.

Signature verification 23: and executing digital signature on the photographed image file by adopting an asymmetric cryptographic algorithm RSA, and providing signature verification service for a verification request of a user terminal. The method comprises the steps of executing full-text digital signature on a live image file, executing all data ranges of a follow file except for a region for storing the digital signature and a check sum field, generating 128-byte signature data by executing the digital signature, saving the signature data to a digital signature field in a follow file header, then calculating the check sum of data (all data before the check sum field) in the file header, and saving the check sum field in the follow file header.

Intelligent analysis 24: and executing an artificial intelligent correlation identification algorithm process in a background, comparing and analyzing the image scene correlation between the depth-of-field fingerprint and the corresponding main frame section for the photographed image and the depth-of-field fingerprint data stream or the real photographed image file transmitted from the user terminal, and automatically identifying the real reliability of the real photographed image scene.

Application 25: the basic application is cloud storage, extensible electronic commerce, face recognition and the like.

User registration 26: user registration information is managed.

The service management 27: managing the strategy parameters of the system operation, managing the service mode, and listing the intelligent analysis into the value added service for compensation.

In a fifth aspect, several refinements of embodiments of the present invention are contemplated.

Firstly, a special format of a real shot image file is defined, a follow file is abandoned, and a real shot image, a depth of field fingerprint, positioning data and a digital signature are stored in a centralized mode. And secondly, receiving the shooting parameters including the main body focusing distance of the terminal by the server, calculating the depth of field at the server, randomly controlling the differential focusing distance through a strategy instruction, and recording, receiving, checking and checking to prevent the terminal from being false. And thirdly, the research and improvement of the related identification algorithm are perfected, and the comparison efficiency of automatic analysis is improved.

By the embodiment, the invention can be applied to an electronic commerce platform, match remote on-site transaction for users of both transaction parties through video interaction, and can also enable the same seller to synchronously face a plurality of buyers (typical auction), so that the transaction places can be visually and objectively understood, the image scene of the transaction background is ensured to be real, credible and safe, and fraud is avoided.

The above-described embodiments should not be construed as limiting the full scope of the invention, nor should they be construed as limiting the scope of the invention.

Claims

1. A real-scene shot image preservation method is used for shooting an image in real time of a real scene and ensuring the reality and credibility of the image scene, and is characterized in that a difference frame section of a difference focusing synchronous shot image is recorded corresponding to a main frame section of the real-scene shot image and is used as a depth of field fingerprint of the image scene, so that the depth of field fingerprint and the corresponding main frame section have image scene correlation of space-time mutual verification, and more than one section of depth of field fingerprint is reserved as evidence for identifying the reality and credibility of the real-scene shot image scene.

2. A live-action image preservation method as claimed in claim 1, wherein said difference frame and said main frame are generated by respective lens division cooperative parallel shooting under the condition of multi-lens shooting, and said depth-of-field fingerprint is associated with and output from the corresponding main frame segment with time sequence consistent.

3. A live action image preservation method according to claim 1 or 2, wherein said depth fingerprint is automatically hidden from being displayed in a main visual observation window at the time of shooting.

4. A live-action image preservation method according to claim 1 or 2, wherein said depth-of-field fingerprint is associated with geographical environment information of a recording site at the time of shooting.

5. A live-action image preservation method according to claim 1 or 2, wherein said depth-of-field fingerprint is subjected to a strategy command to determine a differential focusing strategy during shooting.

6. A live-action image preservation method according to any one of claims 1 to 5, wherein said depth-of-field fingerprint can be instantly transmitted to a server via network communication to be associated with the preserved and fixed evidence.

7. A live-action image security terminal, configured with more than one lens, having a function module for real-time image capturing therein, characterized in that the function module implements the live-action image security method according to any one of claims 1 to 6.

8. A live-action image security system with network communication connection function, configured to include a server for transmitting and storing images, characterized in that, the live-action image security terminal according to claim 7 is provided with an image evidence access management service, interactively receives live-action images including depth of field fingerprints transmitted by the terminal, and secures and stores the evidence at the server side.

9. A live-action image security system as claimed in claim 8, wherein the interactive receiving process further sends a policy instruction from the server to the terminal to determine a differential focusing policy of the depth fingerprint when the terminal takes the image, and controls and checks the differential focusing policy at the server.

10. A live-action image security system as claimed in claim 8 or 9, wherein said server further executes a correlation identification algorithm process to compare and analyze the image scene correlation between said depth-of-field fingerprint and the corresponding body frame segment, so as to automatically identify the authenticity of the live-action image scene.

11. The live-action image security system according to claim 10, wherein the server further assists an authentication process of face recognition, and uses the real reliability of the live-action image scene as a constraint condition of identity authentication.

12. A live-action image security system as claimed in claim 8 or 9, wherein said server further assists the terminal in performing controlled post-editing of the existing live-action image under authorization to secure the image scene.

13. A live-action image security system as claimed in any one of claims 8 to 12, wherein said server further intervenes in the e-commerce process to push and display live-action images including or authenticated by depth fingerprints to clients for image scene security endorsements.