CN108259764A - Video camera, image processing method and device applied to video camera - Google Patents
Video camera, image processing method and device applied to video camera Download PDFInfo
- Publication number
- CN108259764A CN108259764A CN201810259849.9A CN201810259849A CN108259764A CN 108259764 A CN108259764 A CN 108259764A CN 201810259849 A CN201810259849 A CN 201810259849A CN 108259764 A CN108259764 A CN 108259764A
- Authority
- CN
- China
- Prior art keywords
- image
- processing
- video camera
- camera
- photographing module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 11
- 238000012545 processing Methods 0.000 claims abstract description 162
- 230000003287 optical effect Effects 0.000 claims abstract description 42
- 238000011282 treatment Methods 0.000 claims abstract description 42
- 238000000034 method Methods 0.000 claims description 31
- 238000012937 correction Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 5
- 239000011159 matrix material Substances 0.000 description 21
- 230000004927 fusion Effects 0.000 description 19
- 238000010586 diagram Methods 0.000 description 18
- 239000013598 vector Substances 0.000 description 15
- 238000004458 analytical method Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 9
- 230000001815 facial effect Effects 0.000 description 6
- 238000013473 artificial intelligence Methods 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000013519 translation Methods 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 239000000203 mixture Substances 0.000 description 3
- 230000000877 morphologic effect Effects 0.000 description 3
- 238000006748 scratching Methods 0.000 description 3
- 230000002393 scratching effect Effects 0.000 description 3
- 241000251468 Actinopterygii Species 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000011221 initial treatment Methods 0.000 description 2
- 230000005291 magnetic effect Effects 0.000 description 2
- 238000007500 overflow downdraw method Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000003032 molecular docking Methods 0.000 description 1
- 238000003012 network analysis Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 230000001902 propagating effect Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
Abstract
The embodiment of the present application discloses video camera, image processing method and device applied to video camera.The video camera includes camera unit and processing unit, wherein, camera unit includes multiple photographing modules, and photographing module includes pick-up lens and optical sensor；Processing unit includes processing chip, and processing unit is electrically connected with camera unit, and processing unit carries out Combined Treatment for receiving each photographing module the image collected, to receiving by the collected multiple image of each photographing module.This embodiment improves video camera to the analyzing and processing ability and data-handling efficiency of image.
Description
Technical field
The invention relates to camcorder technology fields, and in particular to video camera, the image procossing applied to video camera
Method and device.
Background technology
In existing camcorder technology, video camera generally includes the Optical devices such as camera lens, aperture, shutter, utilizes the optics
Device acquires external image.
Usually it is connected in existing camera apparatus with processing equipment, and collected external image is sent to processing
Equipment equipment for processing analyzes and processes image data, its own generally can not carry out data to the image collected
Analysis.
Invention content
The embodiment of the present application proposes video camera, image processing method and device applied to video camera.
In a first aspect, the embodiment of the present application provides a kind of video camera, the camera photography unit and processing unit,
In, camera unit includes multiple photographing modules, and photographing module includes pick-up lens and optical sensor；Processing unit includes processing
Chip, processing unit are electrically connected with camera unit, and processing unit is for receiving each photographing module the image collected, to receiving
Combined Treatment is carried out by the collected multiple image of each photographing module.
In some embodiments, processing chip is used for：It is right based on the relative parameter between the multiple photographing modules demarcated
The multiple image received is corrected；Combined Treatment is carried out to the image after correction.
In some embodiments, processing unit further includes control chip, and control chip is used for connection processing chip and each light
Sensor.
In some embodiments, control chip is used to receive each optical sensor the image collected, and to each optical sensor
The image collected is spliced；Processing chip is used to handle spliced image.
In some embodiments, camera unit includes at least one of following：Binocular camera shooting module, panoramic shooting module.
In some embodiments, control chip provides after being used for the image coded treatment for shooting the camera unit received
To processing chip.
Second aspect, the embodiment of the present application provide a kind of image processing method applied to video camera, and video camera includes
Camera unit and processing chip, camera unit include multiple photographing modules, and photographing module includes pick-up lens and light sensing
Device；Processing chip is electrically connected with camera unit, and this method includes：Receive each photographing module the image collected；To what is received
Combined Treatment is carried out by the collected multiple image of each photographing module.
In some embodiments, method further includes：Based on the relative parameter between the multiple photographing modules demarcated, docking
The multiple image received is corrected；Combined Treatment is carried out to the image after correction.
In some embodiments, video camera further includes control chip, and control chip passes for connection processing chip and each light
Sensor carries out Combined Treatment to receiving by the collected multiple image of each photographing module, including：Control chip is received to light
The image that the image of sensor acquisition obtains after being spliced；The image obtained after splicing is handled.
The third aspect, the embodiment of the present application provide a kind of image processing apparatus applied to video camera, and video camera includes
Camera unit and processing chip, camera unit include multiple photographing modules, and photographing module includes pick-up lens and light sensing
Device；Processing chip is electrically connected with camera unit, which includes：Receiving unit, is configured to receive each photographing module and collects
Image；Processing unit is configured to carry out Combined Treatment by the collected multiple image of each photographing module to receiving.
In some embodiments, device is further used for：Based on the relative parameter between the multiple photographing modules demarcated,
The multiple image received is corrected；Combined Treatment is carried out to the image after correction.
In some embodiments, video camera further includes control chip, and control chip passes for connection processing chip and each light
Sensor, processing unit are further used for：Receive the image obtained after the image that control chip acquires optical sensor splices；
The image obtained after splicing is handled.
Fourth aspect, the embodiment of the present application provide a kind of video camera, including：Multiple photographing modules, photographing module include
Pick-up lens and optical sensor；One or more processing chips；Storage device, for storing one or more programs, when one
A or multiple programs are performed by one or more processing chips so that one or more processing chips are realized as appointed in second aspect
The method for embodiment of anticipating.
5th aspect, the embodiment of the present application provide a kind of computer readable storage medium, are stored thereon with computer journey
Sequence is realized when the program is executed by processor such as the method for any embodiment in second aspect.
Video camera provided by the embodiments of the present application, image processing method and device applied to video camera, usually setting are taken the photograph
As unit and processing unit, while the processing chip for handling image is set in processing unit so that video camera can be with
The image received is analyzed and processed, the analyzing and processing ability of image and data processing are imitated so as to improve video camera
Rate.
Description of the drawings
By reading the detailed description made to non-limiting example made with reference to the following drawings, the application's is other
Feature, objects and advantages will become more apparent upon：
Fig. 1 shows the structure diagram for the video camera that the application provides；
Fig. 2 schematically illustrates the structure diagram of a binocular camera shooting module of the application offer；
Fig. 3 shows the structure diagram for another video camera that the application provides；
Fig. 4 schematically illustrates the structure diagram of a panoramic shooting module of the application offer；
Fig. 5 schematically illustrates the structure diagram of a binocular panoramic shooting module of the application offer；
Fig. 6 shows the flow chart of one embodiment of the image processing method applied to video camera according to the application；
Fig. 7 is the structure diagram according to one embodiment of the image processing apparatus applied to video camera of the application；
Fig. 8 is adapted for the hardware architecture diagram of the video camera for realizing the embodiment of the present application.
Specific embodiment
The application is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched
The specific embodiment stated is used only for explaining related invention rather than the restriction to the invention.It also should be noted that in order to
Convenient for description, illustrated only in attached drawing and invent relevant part with related.
It should be noted that in the absence of conflict, the feature in embodiment and embodiment in the application can phase
Mutually combination.The application is described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
Fig. 1 shows the structure diagram for the video camera that the application provides.
In the present embodiment, as shown in Figure 1, video camera 100 can include camera unit 11 and processing unit 12.At this
In, camera unit 11 includes multiple photographing modules 111, which includes pick-up lens 1111 and optical sensor
1112。
In the present embodiment, camera unit 11 can include two photographing modules, 111, three photographing modules 111 etc..At this
In, which can need to set according to application scenarios.For example, if video camera setting is when gate inlet
It obtains along the bigger visual angle on direction perpendicular to the ground, multiple photographing modules 111 can be arranged side by side along gravity direction；In order to
It obtains along the bigger visual angle on direction parallel to the ground, above-mentioned photographing module 111 can side by side be set along direction parallel to the ground
It puts.Herein, photographing module 111 can include pick-up lens 1111 and optical sensor 1112.The pick-up lens 111 for example may be used
To include wide-angle lens, telephoto lens, fish eye lens, common single eyeglass camera lens etc..The optical sensor 1112 can for example include
Infrared light transducer[sensor, visible light sensor, RGB-IR sensors etc..Herein, the image example which can obtain
Such as can be face-image, the ambient enviroment image outside automatic driving vehicle, landscape image, images of items.
In the present embodiment, processing unit 12 is electrically connected with camera unit 11, and processing unit 12 can be used for receiving camera shooting
Each 111 the image collected of photographing module in unit 11, to receiving by several collected figures of each photographing module 111
As carrying out Combined Treatment.Herein, Combined Treatment can be to handle multiple image together.Herein, the Combined Treatment is for example
It can include to fusion treatment, splicing of multiple image etc..
In the present embodiment, processing unit 12 includes processing chip 121.Herein, the processing chip 121 can be based on
The processing chip of artificial intelligence (AI, Artificial Intelligence).Such as GPU is integrated in the processing chip 121
Processors such as (Graphics Processing Unit, graphics processors).Optionally, it can also be integrated in processing chip 121
There are DSP (Digital Signal Processing, Digital Signal Processing), FPGA (Field-Programmable Gate
Array, field programmable gate array) etc. functions.According to the needs of scene, the processing core for including different processing functions may be selected
Piece.
In the present embodiment, above-mentioned processing chip 121 can be used for receiving each photographing module the image collected, then right
What is received carries out Combined Treatment, such as image co-registration processing by the collected multiple image of each photographing module.Graphics Fusion leads to
Often include the fusion of signal rank, pixel scale fusion, the other fusion of feature level etc..Above-mentioned fusion method at different levels for example can be with
Merged including the image co-registration based on wavelet transformation, laplacian pyramid etc..
Above-mentioned Combined Treatment can also be the image procossing that the business demand of the practical application scene based on video camera performs,
Such as combine the image that multiple camera units acquire and carry out the processing such as recognition of face, Face datection, target following.As an example,
In recognition of face scene, Combined Treatment can extract feature respectively to several facial images received, then will extraction
The Fusion Features gone out；In target following scene, Combined Treatment can be the position coordinates for extracting target in each image, into
And calculate position of the target in world coordinate system.
In the present embodiment, it after processing chip 121 carries out Combined Treatment to the multiple image that photographing module 111 acquires, connects
Processing analysis can be carried out to image based on various artificial intelligence scenes by, and obtain final handling result.
As an example, processing chip 121 can be applied to recognition of face, processing chip 121 can clap video camera at this time
The facial image taken the photograph carries out analysis and identification etc., judges whether the face-image is matched face-image.Know when applied to face
It, can be including GPU, DSP etc. in above-mentioned processing chip 121 when other.It is various for recognition of face that it was carried can include
Model etc..The model is used for such as can be for for carrying out the convolutional neural networks of face characteristic extraction, Recognition with Recurrent Neural Network
Carry out active shape model (Active Shape Model) of Three-dimensional facial reconstruction technology etc..Above-mentioned each model can be based on
Great amount of samples carries out training in advance.By taking ASM as an example, application of the processing chip 121 in recognition of face is specifically explained
It states.ASM is built upon on the basis of points distribution models.When carrying out face matching, trained ASM can be utilized to taking the photograph
The face-image got as unit carries out facial characteristics identification, according to the face feature vector of the face-image, determines and pre-
Whether the characteristic value between the face feature vector first stored is less than predetermined threshold value, so as to be less than the facial image of predetermined threshold value
It is determined as matched face-image.Herein, which can train as follows：Obtain what is manually marked in advance first
The shape vector of each image in the sample set of feature point image；Then the feature point image in sample set is carried out
Normalized obtains average face shape's vector；Then it is determined using the Principal Component Analysis Algorithm of standard every in sample set
Covariance matrix between the shape vector of one feature point image and average face shape's vector, so as to by each image
Shape vector is represented by average shape vector and change in shape parameter；Finally, the characteristic point pair of each image pattern is calculated
In feature vector of the local gray level model as local feature region adjustment.Finally, the ASM models after being trained.
As an example, processing chip 121 can be used for automatic Pilot field.In automatic Pilot field, processing chip 121
In can be including GPU, DSP, FPGA etc..Its carried such as various models can be included, feedback network.It is for example, above-mentioned
Camera unit can be acquired the external environment of automatic driving vehicle, and collected external environment is provided to processing core
Piece 121.After processing chip 121 analyzes external environment using processing such as model, feedback networks, obtain controlling automatic Pilot
The control instruction of vehicle.The instruction is such as can include turn command, lane change instruction, assisted instruction, deceleration instruction, so as to carry
The automatic Pilot ability of high automatic driving vehicle.It is specifically illustrated by taking feedback network as an example.It can be pre- in processing chip 121
Motion-vector is first set.Processing chip 121, which can be directed to photographing module and send external environment image, be analyzed that (such as identification is outer
The content of portion's ambient image), and according to the hard-over speed of vehicle, vehicle the information of vehicles such as spacing by above-mentioned motion-vector
The tarnsition velocity that can control vehicle respectively, point vector of the turning radius of vehicle are decomposed into, then using this point vector as control
System instruction.Then, above-mentioned electronic equipment according to the control instruction to the driving parameters of vehicle (such as front wheel angle speed, vehicle
Travel speed) it is adjusted.The driving parameters after adjustment are compared with current environment simultaneously, so as to continue to control instruction
It adjusts, until the control instruction can be adapted for environment nowadays, and automatic for adjusting using the control instruction as above-mentioned
Drive the control instruction of the driving parameters of vehicle.
As an example, processing chip 121 can be also used for other image processing fields.What it was carried can for example include
Convolutional neural networks, full convolutional neural networks etc..It is illustrated for scratching diagram technology, processing chip 121 can will image mould
The image that block is got is input to full convolutional neural networks, obtains needing the classification image for the image for scratching figure.Then, processing chip
121 can will be set as prospect gray value in obtained classification image for characterizing the gray value of target object, will be used for table
The gray value of object except sign target object is set as background gray levels, generates initial treatment image.Then, processing chip
121 pairs of initial treatment images carry out Morphological scale-space, obtain morphological images.Then, such as figure cuts algorithm to morphological images
It is handled, and the gray value of the foreground image in treated figure is reduced in pending image at corresponding position
Gray value obtains scratching figure image.
Optionally, at least one binocular camera shooting module can be included in camera unit 11.In other words, multiple photographing modules
Two photographing modules 111 with identical direction combine composition binocular camera shooting module in 111.As shown in Fig. 2, Fig. 2 shows
The structure diagram for showing binocular camera shooting module of example property.By setting binocular camera shooting module, the scene that is taken can be obtained
Depth of view information so that processing unit preferably can be analyzed the scene being taken using depth of view information so that
Analysis result is more accurate.
Optionally, the processing chip 121 in above-mentioned processing unit 11 is also based on the multiple photographing modules 111 demarcated
Between relative parameter, the multiple image received is corrected, then to after correction image carry out Combined Treatment.Specifically
, above-mentioned processing chip 121 can first demarcate binocular camera shooting module.Herein, the number demarcated is needed to image mould
The parameter of block is such as can include the Intrinsic Matrix of each photographing module, distortion factor matrix in binocular camera shooting module.The mark
Fixed method is identical with the reference scaling method of known monocular-camera.Herein, it is also necessary to left in binocular camera shooting module
Relativeness between right photographing module coordinate system is demarcated namely to binocular camera shooting mould spin matrix in the block, translation square
Battle array, eigenmatrix and basis matrix calibration.Herein, it is determined in binocular camera shooting module using spin matrix and translation matrix
The relative position relation of the two photographing module coordinate systems in left and right.Specially can the coordinate under left photographing module be transformed into the right side to take the photograph
The coordinate under left photographing module is transformed into as the coordinate under module or by the coordinate under right photographing module.Eigenmatrix is double
Left and right coordinate system is mutually filled on the matrix changed namely description left and right photographing module picture screen between corresponding points in mesh photographing module
Relationship.And basis matrix adds the relevant information of camera Intrinsic Matrix in photographing module for eigenmatrix, is used for describing
Same object is projected in the relationship under pixel vectors on the capture apparatus picture screen of left and right.Then, processing chip 121 is to calibration
Good binocular camera shooting module is corrected.Herein, binocular camera shooting module is corrected is in order to will be in binocular camera shooting module
The plane of image captured by two photographing modules in left and right is set on same plane.Specifically, left and right two can be utilized to take the photograph
As the spin matrix of module rotates captured image so that the same point in each image projects to two photographing modules
During the plane of delineation, same a line in two pixel coordinate systems, so as to fulfill the correction of binocular camera shooting module.Finally, it is above-mentioned
Processing chip 121 in processing unit can carry out fusion treatment to receiving by the collected multiple image of photographing module.
The method of the fusion treatment can refer to foregoing teachings, and details are not described herein.
Optionally, infrared light projection instrument is also provided in above-mentioned video camera 100, is further included at least in camera unit 11
One infrared photography module and one can be by optical modules.At least one infrared photography mould optical sensor in the block is visible ray
Sensor, the sensor of the visible optical module is visible light sensor.Above-mentioned infrared light projection instrument projects red to subject
Outer light pattern.Above-mentioned infrared photography module for capture with infrared light pattern subject infrared image, it is above-mentioned can
Optical module is seen for obtaining the visible images of subject.Processing chip 121 can to infrared image and with infrared figure
As having the visible images of identical reference object to carry out fusion treatment, the figure for including more rich imaging band information is formed
Picture.
With continued reference to Fig. 3, it illustrates the structure diagrams of another embodiment of video camera that the application provides.
As shown in figure 3, in the present embodiment, video camera 300 includes camera unit 31 and processing unit 32.Camera unit
31 include multiple photographing modules 311, and photographing module 311 includes pick-up lens 3111 and optical sensor 3112.Processing unit 32
Including processing chip 321.Wherein, the concrete structure and Fig. 1 of pick-up lens 3111, optical sensor 3112 and processing chip 321
Shown video camera 100 is identical, and details are not described herein.
In the present embodiment, unlike embodiment shown in FIG. 1, in the present embodiment, processing unit 32 further includes control
Coremaking piece 322.Control chip 322 is used to connect the optical sensor 3112 of each photographing module 311 and processing chip 321.In this reality
It applies in example, control chip 322 can be used for providing with the external USB interface control for carrying out information exchange, to receiving for video camera 300
311 the image collected of each photographing module arrived carries out format conversion, coded treatment etc..
In the present embodiment, above-mentioned control chip 322 can also carry out splicing to multiple image.Image mosaic is for example
Image mosaic can be carried out including the use of OpenCV, utilizes ORB (Oriented FAST and Rotated BRIEF, quick spy
The extraction of sign point and description) algorithm progress image mosaic.In image mosaic technology, it is preferred multiple image is carried out feature point extraction and
Matching, then carries out image registration, that is to say and convert image to be spliced into the same coordinate system.It then, will be after conversion coordinate
Image mosaic together, and fusion is weighted to brightness, pixel etc. for splicing intersection in every two images, finally obtained
Spliced image.The image that existing image mosaic technology acquires multiple camera units of video camera can be used to spell
It connects, details are not described herein.
In the present embodiment, can include unlike embodiment shown in Fig. 2, in camera unit 31 at least one complete
Scape photographing module, as shown in figure 4, Fig. 4 schematically illustrates the structure diagram of panoramic shooting module, two of which camera lens
Head is reversed in the same direction, and two pick-up lens can be the fish eye lens that field angle is more than 180 °, and two image
The optical axis of camera lens can be overlapped.In this way, at least two in multiple photographing module 311a combine composition panoramic shooting mould
Block.By setting panoramic shooting module so that imaging angle bigger, the contents screen that can make the scene being taken are richer
Richness, so that processing unit is more comprehensive to the analysis result of image.
In the present embodiment, above-mentioned control chip 322 can carry out stitching portion to panoramic shooting module the image collected
Reason, and image is provided to processing chip 321 by treated.
Optionally, at least one binocular panoramic shooting module can be included in camera unit 31.In other words, multiple camera shootings
Mould in the block at least four combines composition binocular panoramic shooting module.As an example, the binocular panoramic shooting module packet
Four photographing modules are included, as shown in figure 5, Fig. 5 schematically illustrates the structure diagram of binocular panoramic shooting module.In this pair
In mesh panoramic shooting module, the pick-up lens of two of which photographing module 311b and the camera lens of other two photographing module 311b
Head is respectively facing opposite direction.For forming a binocular camera shooting mould between two photographing module 311b in the same direction
Block.So as to so that a binocular panoramic shooting module is formed between the photographing module of two groups of pick-up lens in the opposite direction.Pass through
Binocular panoramic shooting module in camera unit is set, causes visual angle more while the scene depth for increasing captured object
Widen the richness for extensively, improving image content, so as to be more conducive to analysis of the processing unit to image data.
From figure 3, it can be seen that unlike embodiment shown in FIG. 1, the processing unit 32 of this implementation increases control
Coremaking piece 322 and camera unit 31 increase at least one panoramic shooting module, while control chip 322 that can be taken the photograph to panorama
As the collected figure progress splicing of module, so that captured visual angle is broader, the rich of image content is improved
Fu Du, so as to be more conducive to analysis of the processing unit to image data.
With continued reference to Fig. 6, it illustrates an implementations of the image processing method applied to video camera according to the application
The flow 600 of example.
The video camera includes camera unit and processing chip, and camera unit includes multiple photographing modules, photographing module packet
Pick-up lens and optical sensor are included, processing chip is electrically connected with camera unit.This is applied to the image processing method of video camera,
Include the following steps：
Step 601, photographing module the image collected is received.
In the present embodiment, each photographing module the image collected can for example include infrared light image, it is seen that light image
Deng.Can be for example ambient enviroment image, the landscape figure outside face-image, automatic driving vehicle with the image collected
Picture, images of items etc..Processing chip can receive each photographing module the image collected.
Step 602, Combined Treatment is carried out by the collected multiple image of photographing module to receiving.
In the present embodiment, processing chip can carry out Combined Treatment to the multiple image received.The Combined Treatment example
Such as can be that image co-registration is handled.The Combined Treatment can for example include the fusion treatment to multiple image.Graphics Fusion is usual
Including the fusion of signal rank, pixel scale fusion, the other fusion of feature level etc..Above-mentioned fusion method at different levels can for example wrap
Include the image co-registration based on wavelet transformation, laplacian pyramid fusion etc..
In the present embodiment, processing chip can also to the infrared image that receives and with the infrared image captured by
The identical visible images of object carry out fusion treatment, so as to obtain three-dimensional image.
It in the present embodiment, then can be with after the multiple image that processing chip acquires photographing module carries out Combined Treatment
Processing analysis is carried out based on various artificial intelligence scenes to image, obtains final handling result.
As an example, processing chip can be applied to field of face identification, processing chip can shoot video camera at this time
Facial image carry out analysis and identification etc., judge whether the face-image is matched face-image.
As an example, processing chip can be used for automatic Pilot field.In automatic Pilot field, processing chip can be right
After external environment is using the processing such as model, feedback network analysis, the control instruction for controlling automatic driving vehicle is obtained.This refers to
Enable such as can include turn command, lane change instruction, assisted instruction, deceleration instruction, so as to improve automatic driving vehicle from
Dynamic driving ability.
In some optional realization methods of the present embodiment, processing chip is also based on the multiple camera shooting moulds demarcated
Relative parameter between block is corrected the multiple image received, then carries out Combined Treatment to the image after correction.Tool
Body, above-mentioned processing chip can first demarcate binocular camera shooting module.Herein, the number photographing module demarcated is needed
Parameter such as can include the Intrinsic Matrix of each photographing module, distortion factor matrix in binocular camera shooting module.The calibration
Method it is identical with the reference scaling method of known monocular photographing module.Herein, it is also necessary to left in binocular camera shooting module
Relativeness between right photographing module coordinate system is demarcated namely to binocular camera shooting mould spin matrix in the block, translation square
Battle array, eigenmatrix and basis matrix calibration.Herein, it is determined in binocular camera shooting module using spin matrix and translation matrix
The relative position relation of the two photographing module coordinate systems in left and right.Specially can the coordinate under left photographing module be transformed into the right side to take the photograph
The coordinate under left photographing module is transformed into as the coordinate under module or by the coordinate under right photographing module.Eigenmatrix is double
Left and right coordinate system is mutually filled on the matrix changed namely description left and right photographing module picture screen between corresponding points in mesh photographing module
Relationship.And basis matrix adds the relevant information of camera Intrinsic Matrix in photographing module for eigenmatrix, is used for describing
Same object is projected in the relationship under pixel vectors on the photographing module picture screen of left and right.Then, processing chip is to demarcating
Binocular camera shooting module be corrected.Herein, binocular camera shooting module is corrected be in order to will in binocular camera shooting module it is left
The plane of image captured by right two photographing modules is set on same plane.Specifically, the camera shooting of left and right two can be utilized
The spin matrix of module rotates captured image so that the same point in each image projects to two photographing module figures
During image plane, same a line in two pixel coordinate systems, so as to fulfill the correction of binocular camera shooting module.Finally, above-mentioned place
Processing chip in reason unit can carry out fusion treatment to receiving by the collected multiple image of photographing module.The fusion
The method of processing can refer to foregoing teachings, and details are not described herein.
In some optional realization methods of the present embodiment, the control unit of above-mentioned video camera further includes control chip,
The control chip is used for connection processing chip and optical sensor.Above-mentioned processing chip can also be used to receive control chip to light sensing
The image that the image of device acquisition obtains after being spliced, and the image obtained after splicing is handled.It such as can be to splicing
The image obtained afterwards carries out data analysis, so as to obtain analysis result.As an example, in specific application scenarios, such as nothing
In the application scenarios of people's retail shop, processing chip can detect the track of shopper according to spliced full images, so as to sentence
The consumer behavior of disconnected shopper is preferably to optimize unmanned shop.Processing chip can also identify barrier according to spliced image
Hinder object etc..
Image processing method provided by the embodiments of the present application applied to video camera passes through each photographing module to receiving
The multiple image of acquisition carries out Conjoint Analysis processing, can improve video camera to the analyzing and processing ability of image and data processing
Efficiency.
With further reference to Fig. 7, as the realization to method shown in above-mentioned each figure, this application provides one kind to be applied to camera shooting
One embodiment of the image processing apparatus of machine, the device embodiment are corresponding with embodiment of the method shown in Fig. 3.
The video camera includes camera unit and processing chip, and camera unit includes multiple photographing modules, photographing module packet
Pick-up lens and optical sensor are included, processing chip is electrically connected with camera unit.
As shown in fig. 7, the image processing apparatus 700 applied to video camera described in the present embodiment includes：Receiving unit
701, it is configured to receive each photographing module the image collected；Processing unit 702 is configured to receiving by each camera shooting
The collected multiple image of module carries out Combined Treatment.
In the present embodiment, applied in the image processing apparatus of video camera 700：Receiving unit 701 and processing unit 702
Specific processing and its caused technique effect can be respectively with reference to the step 601 and the phase of step 602 in 6 corresponding embodiment of figure
It speaks on somebody's behalf bright, details are not described herein.
In some optional realization methods of the present embodiment, the image processing apparatus 700 applied to video camera is further
For：Based on the relative parameter between the multiple photographing modules demarcated, the multiple image received is corrected；To correction
Image afterwards carries out Combined Treatment.
In some optional realization methods of the present embodiment, video camera can also include control chip, and control chip is used
In connection processing chip and each optical sensor, processing unit 702 is further used for：Receive what control chip acquired optical sensor
The image that image obtains after being spliced；The image obtained after splicing is handled.
Image processing apparatus provided by the embodiments of the present application applied to video camera passes through each photographing module to receiving
The multiple image of acquisition carries out Conjoint Analysis processing, can improve video camera to the analyzing and processing ability of image and data processing
Efficiency.
Below with reference to Fig. 8, it illustrates suitable for being used for realizing the hardware architecture diagram of the video camera of the embodiment of the present application.
The structure of video camera shown in Fig. 8 is only an example, and the function and use scope of the embodiment of the present application should not be brought and appointed
What is limited.
As shown in figure 8, video camera 800 include pick-up lens 801, camera 802, optical sensor 803, optical sensor 804,
Processing chip 805 and memory 806.Wherein, include image processor GPU in processing chip 805.Pick-up lens 801, camera shooting
First 802, optical sensor 803, optical sensor 804, processing chip 805 and memory 806 are connected with each other by bus 807.It is defeated
Enter/export (I/O) interface 808 and be also connected to bus 807.Here, it may be implemented as computer journey according to the present processes
Sequence, and be stored in memory 806.Processing chip 805 in video camera 800 by call stored in memory 806 it is upper
Computer program is stated, to implement the image analysis function limited in the present processes.
Particularly, in accordance with an embodiment of the present disclosure, it may be implemented as computer above with reference to the process of flow chart description
Software program.For example, embodiment of the disclosure includes a kind of computer program product, including being carried on computer-readable medium
On computer program, which includes for the program code of the method shown in execution flow chart.In such reality
It applies in example, which can be downloaded and installed from network by communications portion 809 and/or from detachable media
810 are mounted.When the computer program is performed by the image processing unit (GPU) in processing chip 805, perform the application's
The above-mentioned function of being limited in method.It should be noted that computer-readable medium described herein can be computer-readable
Signal media or computer readable storage medium either the two arbitrarily combine.Computer readable storage medium is for example
It may be-but not limited to-system, device or the device or arbitrary of electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor
Above combination.The more specific example of computer readable storage medium can include but is not limited to：It is led with one or more
The electrical connection of line, portable computer diskette, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type can
Program read-only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device,
Magnetic memory device or above-mentioned any appropriate combination.In this application, computer readable storage medium can be any packet
Contain or store the tangible medium of program, which can be commanded the either device use or in connection of execution system, device
It uses.And in this application, computer-readable signal media can include propagating in a base band or as a carrier wave part
Data-signal, wherein carrying computer-readable program code.Diversified forms may be used in the data-signal of this propagation,
Including but not limited to electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can be with
Any computer-readable medium other than computer readable storage medium, which can send, propagate or
Person transmission for by instruction execution system, device either device use or program in connection.Computer-readable Jie
The program code included in matter can be transmitted with any appropriate medium, including but not limited to：Wirelessly, electric wire, optical cable, RF etc.
Deng or above-mentioned any appropriate combination.
Flow chart and block diagram in attached drawing, it is illustrated that according to the system of the various embodiments of the application, method and computer journey
Architectural framework in the cards, function and the operation of sequence product.In this regard, each box in flow chart or block diagram can generation
The part of one module of table, program segment or code, the part of the module, program segment or code include one or more use
In the executable instruction of logic function as defined in realization.It should also be noted that it in some implementations as replacements, is marked in box
The function of note can also be occurred with being different from the sequence marked in attached drawing.For example, two boxes succeedingly represented are actually
It can perform substantially in parallel, they can also be performed in the opposite order sometimes, this is depended on the functions involved.Also it to note
Meaning, the combination of each box in block diagram and/or flow chart and the box in block diagram and/or flow chart can be with holding
The dedicated hardware based system of functions or operations as defined in row is realized or can use specialized hardware and computer instruction
Combination realize.
Being described in unit involved in the embodiment of the present application can be realized by way of software, can also be by hard
The mode of part is realized.Described unit can also be set in the processor, for example, can be described as：A kind of processor packet
Include receiving unit and processing unit.Wherein, the title of these units does not form the limit to the unit in itself under certain conditions
It is fixed, for example, receiving unit is also described as " unit for receiving each photographing module the image collected ".
As on the other hand, present invention also provides a kind of computer-readable medium, which can be
Included in device described in above-described embodiment；Can also be individualism, and without be incorporated the device in.Above-mentioned calculating
Machine readable medium carries one or more program, when said one or multiple programs are performed by the device so that should
Device：Receive each photographing module the image collected in video camera；To receiving by several collected figures of photographing module
As carrying out Combined Treatment；Wherein, video camera includes camera unit and processing chip, and camera unit includes multiple photographing modules,
Photographing module includes pick-up lens and optical sensor；Processing chip is electrically connected with camera unit.
The preferred embodiment and the explanation to institute's application technology principle that above description is only the application.People in the art
Member should be appreciated that invention scope involved in the application, however it is not limited to the technology that the specific combination of above-mentioned technical characteristic forms
Scheme, while should also cover in the case where not departing from foregoing invention design, it is carried out by above-mentioned technical characteristic or its equivalent feature
The other technical solutions for arbitrarily combining and being formed.Such as features described above has similar work(with (but not limited to) disclosed herein
The technical solution that the technical characteristic of energy is replaced mutually and formed.
Claims (14)
1. a kind of video camera, including camera unit and processing unit, wherein,
The camera unit includes multiple photographing modules, and photographing module includes pick-up lens and optical sensor；
The processing unit includes processing chip, and the processing unit is electrically connected with the camera unit, and the processing unit is used
In receiving photographing module the image collected, Combined Treatment is carried out by the collected multiple image of photographing module to receiving.
2. video camera according to claim 1, wherein, the processing chip is used for：
Based on the relative parameter between the multiple photographing modules demarcated, the multiple image received is corrected；
Combined Treatment is carried out to the image after correction.
3. video camera according to claim 1, wherein, the processing unit further includes control chip, the control chip
For connecting the processing chip and optical sensor.
4. video camera according to claim 3, wherein, the control chip is used to receive the collected figure of optical sensor
Picture, and optical sensor the image collected is spliced；
The processing chip is used to handle spliced image.
5. video camera according to claim 1, wherein, the camera unit includes at least one of following：Binocular camera shooting mould
Block, panoramic shooting module.
6. video camera according to claim 3, wherein, the control chip is used to clap the camera unit received
The processing chip is provided to after the image coded treatment taken the photograph.
7. a kind of image processing method applied to video camera, the video camera includes camera unit and the processing chip,
The camera unit includes multiple photographing modules, and photographing module includes pick-up lens and optical sensor；The processing chip with
The camera unit electrical connection, the method includes：
Receive photographing module the image collected；
Combined Treatment is carried out by the collected multiple image of photographing module to receiving.
8. according to the method described in claim 7, wherein, the method further includes：
Based on the relative parameter between the multiple photographing modules demarcated, the multiple image received is corrected；
Combined Treatment is carried out to the image after correction.
9. method according to claim 7 or 8, wherein, the video camera further includes control chip, and the control chip is used
In connecting the processing chip and optical sensor, described pair receive combined by the collected multiple image of photographing module
Processing, including：
Receive the image obtained after the image that the control chip acquires optical sensor splices；
The image obtained after splicing is handled.
10. a kind of image processing apparatus applied to video camera, video camera includes camera unit and the processing chip, described
Camera unit includes multiple photographing modules, and the photographing module includes pick-up lens and optical sensor；The processing chip with
The camera unit electrical connection, described device include：
Receiving unit is configured to receive photographing module the image collected；
Processing unit is configured to carry out Combined Treatment by the collected multiple image of photographing module to receiving.
11. device according to claim 10, wherein, described device is further used for：
Based on the relative parameter between the multiple photographing modules demarcated, the multiple image received is corrected；
Combined Treatment is carried out to the image after correction.
12. the device according to claim 10 or 11, wherein, the video camera further includes control chip, the control core
For connecting the processing chip and optical sensor, the processing unit is further used for piece：
Receive the image obtained after the image that the control chip acquires optical sensor splices；
The image obtained after splicing is handled.
13. a kind of video camera, including：
Multiple photographing modules, photographing module include pick-up lens and optical sensor；
One or more processing chips；
Storage device, for storing one or more programs,
When one or more of programs are performed by one or more of processing chips so that one or more of processing cores
Piece realizes the method as described in any in claim 7-9.
14. a kind of computer-readable medium, is stored thereon with computer program, wherein, it is real when described program is executed by processor
The now method as described in any in claim 7-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810259849.9A CN108259764A (en) | 2018-03-27 | 2018-03-27 | Video camera, image processing method and device applied to video camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810259849.9A CN108259764A (en) | 2018-03-27 | 2018-03-27 | Video camera, image processing method and device applied to video camera |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108259764A true CN108259764A (en) | 2018-07-06 |
Family
ID=62746492
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810259849.9A Pending CN108259764A (en) | 2018-03-27 | 2018-03-27 | Video camera, image processing method and device applied to video camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108259764A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109274939A (en) * | 2018-09-29 | 2019-01-25 | 成都臻识科技发展有限公司 | A kind of parking lot entrance monitoring method and system based on three camera modules |
| CN110717445A (en) * | 2019-10-09 | 2020-01-21 | 清华大学 | Front vehicle distance tracking system and method for automatic driving |
| CN111311742A (en) * | 2020-03-27 | 2020-06-19 | 北京百度网讯科技有限公司 | Three-dimensional reconstruction method, three-dimensional reconstruction device and electronic equipment |
| TWI704408B (en) * | 2018-08-22 | 2020-09-11 | 宏碁股份有限公司 | Omnidirectional camera apparatus and image mapping/combining method thereof |
| CN113923330A (en) * | 2021-10-11 | 2022-01-11 | 中国人民解放军63921部队 | Multi-target measurement system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102355545A (en) * | 2011-09-20 | 2012-02-15 | 中国科学院宁波材料技术与工程研究所 | 360-degree real-time panoramic camera |
| JP2017050819A (en) * | 2015-09-04 | 2017-03-09 | Kddi株式会社 | Generation device for panoramic image data, generation method and program |
| CN106878687A (en) * | 2017-04-12 | 2017-06-20 | 吉林大学 | A kind of vehicle environment identifying system and omni-directional visual module based on multisensor |
| CN106926800A (en) * | 2017-03-28 | 2017-07-07 | 重庆大学 | The vehicle-mounted visually-perceptible system of multi-cam adaptation |
| CN107423667A (en) * | 2017-04-12 | 2017-12-01 | 杭州奥腾电子股份有限公司 | A kind of method of prediction barrier on car body |
| CN107483911A (en) * | 2017-08-25 | 2017-12-15 | 秦山 | A kind of signal processing method and system based on more mesh imaging sensors |
-
2018
- 2018-03-27 CN CN201810259849.9A patent/CN108259764A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102355545A (en) * | 2011-09-20 | 2012-02-15 | 中国科学院宁波材料技术与工程研究所 | 360-degree real-time panoramic camera |
| JP2017050819A (en) * | 2015-09-04 | 2017-03-09 | Kddi株式会社 | Generation device for panoramic image data, generation method and program |
| CN106926800A (en) * | 2017-03-28 | 2017-07-07 | 重庆大学 | The vehicle-mounted visually-perceptible system of multi-cam adaptation |
| CN106878687A (en) * | 2017-04-12 | 2017-06-20 | 吉林大学 | A kind of vehicle environment identifying system and omni-directional visual module based on multisensor |
| CN107423667A (en) * | 2017-04-12 | 2017-12-01 | 杭州奥腾电子股份有限公司 | A kind of method of prediction barrier on car body |
| CN107483911A (en) * | 2017-08-25 | 2017-12-15 | 秦山 | A kind of signal processing method and system based on more mesh imaging sensors |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI704408B (en) * | 2018-08-22 | 2020-09-11 | 宏碁股份有限公司 | Omnidirectional camera apparatus and image mapping/combining method thereof |
| CN109274939A (en) * | 2018-09-29 | 2019-01-25 | 成都臻识科技发展有限公司 | A kind of parking lot entrance monitoring method and system based on three camera modules |
| CN110717445A (en) * | 2019-10-09 | 2020-01-21 | 清华大学 | Front vehicle distance tracking system and method for automatic driving |
| CN110717445B (en) * | 2019-10-09 | 2022-08-23 | 清华大学 | Front vehicle distance tracking system and method for automatic driving |
| CN111311742A (en) * | 2020-03-27 | 2020-06-19 | 北京百度网讯科技有限公司 | Three-dimensional reconstruction method, three-dimensional reconstruction device and electronic equipment |
| CN113923330A (en) * | 2021-10-11 | 2022-01-11 | 中国人民解放军63921部队 | Multi-target measurement system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108259764A (en) | Video camera, image processing method and device applied to video camera | |
| Wang et al. | 360sd-net: 360 stereo depth estimation with learnable cost volume | |
| CN104835117B (en) | Spherical panorama generation method based on overlapping mode | |
| CN112950667B (en) | Video labeling method, device, equipment and computer readable storage medium | |
| US20180012411A1 (en) | Augmented Reality Methods and Devices | |
| JP4825980B2 (en) | Calibration method for fisheye camera. | |
| WO2022165809A1 (en) | Method and apparatus for training deep learning model | |
| CN109685913B (en) | Augmented reality implementation method based on computer vision positioning | |
| CN108961327A (en) | A kind of monocular depth estimation method and its device, equipment and storage medium | |
| WO2023011013A1 (en) | Splicing seam search method and apparatus for video image, and video image splicing method and apparatus | |
| CN110060230B (en) | Three-dimensional scene analysis method, device, medium and equipment | |
| EP4178194A1 (en) | Video generation method and apparatus, and readable medium and electronic device | |
| CN105488777A (en) | System and method for generating panoramic picture in real time based on moving foreground | |
| CN110276831A (en) | Constructing method and device, equipment, the computer readable storage medium of threedimensional model | |
| CN115330594A (en) | Target rapid identification and calibration method based on unmanned aerial vehicle oblique photography 3D model | |
| KR101715781B1 (en) | Object recognition system and method the same | |
| WO2023280082A1 (en) | Handle inside-out visual six-degree-of-freedom positioning method and system | |
| CN108564654B (en) | Picture entering mode of three-dimensional large scene | |
| CN114358112A (en) | Video fusion method, computer program product, client and storage medium | |
| CN108122199A (en) | The original image color method of adjustment and device of a kind of panorama camera | |
| CN116665077A (en) | Power transmission line detection shooting method and system based on AI (advanced identification) recognition technology | |
| CN116012609A (en) | Multi-target tracking method, device, electronic equipment and medium for looking around fish eyes | |
| CN115565155A (en) | Training method of neural network model, generation method of vehicle view and vehicle | |
| CN115115713A (en) | Unified space-time fusion all-around aerial view perception method | |
| CN115359067A (en) | Continuous convolution network-based point-by-point fusion point cloud semantic segmentation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180706 |
|
| RJ01 | Rejection of invention patent application after publication |