CN104243950A - Method and apparatus for real-time conversion of 2-dimensional content to 3-dimensional content - Google Patents
Method and apparatus for real-time conversion of 2-dimensional content to 3-dimensional content Download PDFInfo
- Publication number
- CN104243950A CN104243950A CN201410240725.8A CN201410240725A CN104243950A CN 104243950 A CN104243950 A CN 104243950A CN 201410240725 A CN201410240725 A CN 201410240725A CN 104243950 A CN104243950 A CN 104243950A
- Authority
- CN
- China
- Prior art keywords
- pixel
- picture frame
- subset
- motion vector
- speed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000006243 chemical reaction Methods 0.000 title abstract description 6
- 239000013598 vector Substances 0.000 claims abstract description 68
- 230000033001 locomotion Effects 0.000 claims abstract description 66
- 238000004364 calculation method Methods 0.000 claims description 74
- ORQBXQOJMQIAOY-UHFFFAOYSA-N nobelium Chemical compound [No] ORQBXQOJMQIAOY-UHFFFAOYSA-N 0.000 description 39
- 238000004891 communication Methods 0.000 description 21
- 230000008859 change Effects 0.000 description 6
- 238000004590 computer program Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000009466 transformation Effects 0.000 description 4
- 230000008447 perception Effects 0.000 description 3
- 210000004556 brain Anatomy 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000000712 assembly Effects 0.000 description 1
- 238000000429 assembly Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013213 extrapolation Methods 0.000 description 1
- 230000002070 germicidal effect Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000000523 sample Substances 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Landscapes
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Image Analysis (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention relates to a method and an apparatus for real-time conversion of a 2-dimensional content to a 3-dimensional content. Various aspects of the method and apparatus for video processing may include a computing device communicably coupled to an external device. The computing device may be operable to determine an average vertical velocity and an average horizontal velocity of a subset of pixels in an image frame and determine a depth value for each pixel of the subset of pixels based on calculated motion vectors of the pixel of the subset of pixels, the average vertical velocity of the subset of pixels and the average horizontal velocity of the subset of pixels.
Description
To the cross reference of related application
The application is referred to the U.S. Patent application that the sequence number submitted on March 28th, 2012 is 13/432,065, and its full content is incorporated herein by reference.
The application is referred to the U.S. Patent application that the sequence number submitted on February 28th, 2011 is 13/036,656, and its full content is incorporated herein by reference.
Technical field
The various embodiments of present disclosure relate to Content Transformation.More specifically, the various embodiments of present disclosure relate to the method and apparatus 2 dimension (2D) contents being converted in real time 3 dimensions (3D) content.
Background technology
Must the relevant problem for the treatment of effeciency the producer of consumer electronics device does not affect disposal ability during attempting utilizing available limited storage space.Especially, during two-dimensional content is converted to three-dimensional content in real time, consumer electronics device needs high disposal ability and large storage space.Therefore, there is the demand to efficient 2D to 3D contents conversion system.
By more such have the present disclosure of setting forth with reference to accompanying drawing in the remainder of the application some in system, other limitation that is conventional and traditional method and shortcoming will become obvious for a person skilled in the art.
Summary of the invention
Method and apparatus for 2 dimension contents are converted to 3 dimension contents is in real time illustrated in the accompanying drawings at least one substantially, and/or by reference to the accompanying drawings at least one be described, as more fully set forth in claim.
By coming together to read the detailed description below present disclosure with accompanying drawing, can recognize the feature and advantage of these and other of present disclosure, in the accompanying drawings, identical Reference numeral represents identical part all the time.
Accompanying drawing explanation
Fig. 1 is the block diagram of the network environment for real-time 2D to 3D conversion of the embodiment illustrated according to present disclosure.
Fig. 2 is the block diagram of the calculation element of embodiment according to present disclosure.
Fig. 3 illustrates the picture frame of the embodiment according to present disclosure.
Fig. 4 A to Fig. 4 C illustrates the exemplary plot according to the pixel column in the picture frame of the embodiment of present disclosure.
Fig. 5 illustrates the flow chart for the illustrative steps that provides real-time 2D to 3D to change at calculation element place according to the embodiment of present disclosure.
Fig. 6 illustrates the flow chart for the illustrative steps that provides real-time 2D to 3D to change at calculation element place according to another embodiment of present disclosure.
Embodiment
Various execution mode can found for providing in the method for real-time conversion of 2 dimension contents to 3 dimension contents and/or equipment.
The illustrative aspects of present disclosure can comprise the calculation element for Video processing.In an embodiment, calculation element can determine average vertical speed and/or the Mean Horizontal Velocity of the pixel subset in picture frame.Based on the motion vector calculated be associated with the pixel in pixel subset, average vertical speed and Mean Horizontal Velocity, the depth value of each pixel in pixel subset can be determined.
Motion vector algorithm may the motion vector of pixel in the region of homogeneous in calculating chart picture frame.For further calculating, the motion vector corresponding with this pixel in the region of homogeneous can be considered.In an embodiment, the motion vector corresponding with picture frame can be provided as input.The motion vector corresponding with one or more pixel may be unknown, or may be undefined in given input.This pixel with unknown motion vector and/or undefined motion vector may not have predetermined motion vector, thus, and may not included in for not included in the pixel subset of depth calculation.
In an embodiment, based on the average vertical pixel speed of the motion vector calculated be associated with the pixel in pixel subset, picture frame and the average level pixel speed of picture frame, the depth value of each pixel in pixel subset can be determined.Average vertical pixel speed and average horizontal pixel speed can be calculated based on the pixel in picture frame with predetermined motion vector.Based on the depth value be associated with the pixel of picture frame, calculation element can produce the stereo pairs for picture frame.Stereo pairs can comprise left image and right image, and in left image, each pixel in picture frame can offset left specific value, and in right image, each pixel in picture frame can offset to the right specific value.The depth value that specific value can be associated with each pixel in same picture frame is proportional.In an embodiment, pixel subset can belong to a line in picture frame or multirow pixel.
In an embodiment, picture frame can be two dimension (2D) picture frame.In an embodiment, picture frame can be two dimension (2D) picture frame of the pixel resolution with minification.In an embodiment, calculation element can receive picture frame from one or two external device (ED) and/or local storage.It is one or more that calculation element can comprise in television set, smart phone, flat computer, laptop computer and/or desktop PC.It is one or more that external device (ED) can comprise in video player, Set Top Box, personal video recorder, video camera and/or Cloud Server.
Fig. 1 is the block diagram of the network environment for real-time 2D to 3D conversion of the embodiment illustrated according to present disclosure.With reference to Fig. 1, network environment 100 is shown.Network environment 100 can comprise calculation element 102, one or more external device (ED) 104 (such as, video player 104a, Set Top Box 104b, personal video recorder 104c, Cloud Server 104d, video camera 104e) and communication network 106.However, present disclosure is not limited to this, and can adopt other external device (ED) when not limiting the scope of the disclosure.
Calculation element 102 can comprise can with external device (ED) (such as 104a) direct communication or suitable logic, circuit, interface and/or the code that communicate via communication network 106.In an embodiment, calculation element 102 external device (such as 104a) may can send instruction and order, and/or receives instruction and order from this external device (ED).Calculation element 102 also may can play three-dimensional (3D) content.The example of calculation element 102 can include, but not limited to television set, smart phone, flat computer, laptop computer and/or desktop PC.However, present disclosure is not limited to this, and can adopt other calculation element when not limiting the scope of the disclosure.
External device (ED) (such as 104a) can comprise can with calculation element 102 direct communication or suitable logic, circuit, interface and/or the code that communicate via communication network 106.In an embodiment, external device (ED) (such as 104a) may can send instruction and order to calculation element 102, and/or receives instruction and order from this calculation element 102.In another embodiment, external device (ED) (such as 104a) can provide the media content can asked by calculation element 102.
Communication network 106 can comprise medium, and by this medium, the calculation element 102 in network environment 100 can intercom mutually with external device (ED) (such as 104a).Communication network 106 can be realized by one or more communication protocol, described communication protocol comprises, but be not limited to, Wireless Fidelity (Wi-Fi), radio universal serial bus (WUSB), local area network (LAN) (LAN), ZigBee, transmission control protocol and Internet Protocol (TCP/IP), Ethernet, cable network, satellite network, DLNA (DLNA) home network and/or Bluetooth communication protocol.Various assemblies in network environment 100 can be connected with communication network 106 according to various wired and wireless communication protocol (such as, TCP/IP, User Datagram Protoco (UDP) (UDP), ZigBee, infrared (IR), IEEE802.11a, IEEE802.11b, IEEE802.11g and/or IEEE802.11n communication protocol).
When operating, calculation element 102 and external device (ED) (such as 104a) can be switched on, and can activate connection between calculation element 102 and external device (ED) (such as 104a) via communication network 106.Calculation element 102 can based on the request that can be transmitted by calculation element 102, from external device (ED) (such as 104a) receiving media content.Media content can comprise one or more picture frames that can form 2D content.Calculation element 102 can calculate the motion vector of all pixels in the one or more picture frames received.Calculation element 102 can carry out calculation of motion vectors based on one or more algorithms.Calculation element 102 can be determined and the degree of depth that one or more pixel is associated based on the motion vector calculated.
In addition, calculation element 102 can produce stereo pairs corresponding to picture frame based on the depth map calculated.Stereo pairs comprises left image and right image.Stereo pairs forms the corresponding 3D content of the 2D picture frame that transmitted by external device (ED) (such as 104a).
In an embodiment, 3D content can be shown on the display screen be associated with calculation element 102.In an embodiment, the 3D content produced by calculation element 102 (such as, smart phone) can be sent to any display unit (such as, monitor) for display.
In an embodiment, 2D content can be following in one or more: the film stored in the removable dish in video player 104a, the Live TV programs transmitted by Set Top Box 104b, the media content recorded by personal video recorder 104c, the media content stored in Cloud Server 104d, the video segment (footage) etc. transmitted by video camera 104e.
Fig. 2 is the block diagram of the calculation element of embodiment according to present disclosure.Element in composition graphs 1 carrys out key-drawing 2.With reference to Fig. 2, calculation element 102 is shown.Calculation element 102 can include, but not limited to processor 202, transceiver 204, memory 206, communication interface 208 and input and output (I/O) device 210.Memory 206 can include, but not limited to camera application 212, operating system 214, motion vector estimator 216, depth estimator 218, view data 220, data estimator 222, stereo-picture generator 224 and out of Memory 226.
Processor 202 can be coupled communicatedly with memory 206, transceiver 204 and I/O device 210.Transceiver 204 can be coupled with communication network 106 communicatedly via communication interface 208.
Processor 202 can comprise suitable logic, circuit, interface and/or the code that can operate for performing the one group of instruction be stored in memory 206.Processor 202 can be realized based on one or more processor technology as known in the art.The example of processor 202 can be based on the processor of X86, risc processor, asic processor, cisc processor or any other processor.
Memory 206 can comprise suitable logic, circuit, interface and/or the code that can operate for storing the one group of instruction received.Memory 206 can be based on, but be not limited to, and random access memory (RAM), read-only memory (ROM), hard disk drive (HDD), storage server and/or secure digital (SD) card realizes.
Transceiver 204 can comprise suitable logic, circuit, interface and/or the code that can operate for communicating with external device (ED) (such as 104a) via communication interface 208.In an embodiment, transceiver 204 can operate for external device (ED) (such as, 104a) direct communication.Transceiver 204 can realize for supporting the known technology with the wired or wireless communication of communication network 106.
I/O device 210 can comprise and can operationally be coupled to receive user with processor 202 and input and suitable logic, circuit, interface and/or the code that show 3D content.The example of input unit can include, but not limited to touch pad, track pad, transducer, camera and/or microphone.The example of output device can include, but not limited to liquid crystal display (LCD) and/or light-emitting diode (LED) display.However, present disclosure is not limited thereto, and can adopt other I/O device 210 when not limiting the scope of the disclosure.
When operating, 2D content can be sent to calculation element 102 by external device (ED) (such as, 104a).Processor 202 can receive the 2D content transmitted by external device (ED) (such as, 104a).In an embodiment, processor 202 can receive 2D content from memory 206.Camera application 212 can comprise and can preferably perform the various function of calculation element 102 and the program command of operation by processor 202.Operating system 214 can control and coordinate rudimentary (low-level) function of calculation element 102.Motion vector estimator 216 can control and coordinated movement of various economic factors vector estimation procedure.This can comprise determines the motion vector corresponding with each pixel of picture frame, or receives the motion vector information corresponding with picture frame.In an embodiment, motion vector estimator 216 can determine the average motion vector of picture frame, and uses identical average motion vector for the picture frame subsequently of predetermined quantity.
Depth estimator 218 can control and coordinate depth estimation procedure, to determine and the depth value that each pixel in picture frame is associated.View data 220 can comprise one or more picture frame.Data estimator 222 can comprise information or the data of any type for performing depth estimation procedure.Stereo-picture generator 224 can control and coordinate stereo-picture production process, to produce the stereo pairs corresponding with picture frame.Stereo-picture production process can adopt the result of depth estimation procedure to produce stereo pairs.Out of Memory 226 can comprise other adequate information of the operation for calculation element 102.In order to understand present disclosure better, deposit example shown in Figure 3.
Fig. 3 illustrates the picture frame of the embodiment according to present disclosure.Element in composition graphs 1 and Fig. 2 carrys out key-drawing 3.With reference to Fig. 3, picture frame 302 is shown.Picture frame 302 can comprise (being collectively referred to as object 306 hereinafter) and the background 304 such as object 306a, object 306b, object 306c that the side that illustrates at arrow D1, D2, the D3 etc. by correspondence respectively moves up.Pixel subset (such as, 308a, 308b, 308c) is corresponding with the pixel column in picture frame 302.Object 306a moves along the X direction as shown in arrow D1.Object 306a only has speed along the X direction.Object 306b is mobile along X-Y plane diagonal as shown in arrow D2.Object 306b have along the X direction with the speed of both Y-directions.Object 306c moves along the Y direction as shown in arrow D3.Object 306c only has speed along the Y direction.Processor 202 can the motion vector of multiple pixels in calculating chart picture frame 302.In addition, processor 202 can be determined and the depth value that each pixel in multiple pixel is associated.
In an embodiment, processor 202 can perform identification by once the pixel subset (such as, 308a, 308b, 308c etc.) processed from picture frame 302 by step.Pixel subset (such as, 308a) can belong to a line in picture frame 302 or multirow.
In an embodiment, processor calculates the mean pixel speed of picture frame in the X direction (hereinafter referred to as " averV
x") and the mean pixel speed of picture frame in the Y direction (hereinafter referred to as " averV
y").Processor 202 can operate the depth value for carrying out each pixel in calculating chart picture frame 302 based on the motion vector of each pixel in picture frame 302 and average pixel speed.In an embodiment, processor 202 can use following formula (1) to calculate the depth value corresponding with each pixel in picture frame 302:
Depth＝abs(V
X-averV
X)+abs(V
Y-averV
Y).............(1)
Wherein, V
xand V
ythe pixel speed in the x-direction and the z-direction in picture frame 302 respectively, and averV
xand averV
ymean pixel speed in the x-direction and the z-direction respectively.In an embodiment, motion vector algorithm possibly cannot calculate the motion vector corresponding with one or more pixel.In this case, the motion vector of this pixel may not included in the calculating of mean pixel speed.In an embodiment, the mean pixel speed calculated for depth value can be the mean pixel speed of pixel subset (such as, 308a).Pixel subset (such as, 308a) can comprise can the pixel of compute depth value.
Most of pixel with approximately uniform velocity vector is called as " background pixel ", and remaining pixel is called as " foreground pixel ".When the averVx in the formula (1) above being used for by the average speed of background pixel, averVy (instead of average frame speed), improve beholder's perceived quality of depth map.
In an embodiment, the motion vector of some pixels in picture frame 302 may be unknown.This may occur in texture and may to lack or in faint region (such as, cloudless sky).Described algorithm removes this defect by adopting the human capabilitiy of the degree of depth by filling in homogenous area from the region extrapolation degree of depth with good texture.
In an embodiment, the average motion vector of previous picture frame can be applicable to current picture frame.The average speed be associated with previous picture frame can be used to estimate the calculating of the depth value of current image frame.This can reduce calculating needed for the estimation of the depth value be associated with picture frame 302 and time.In addition, with regard to film, average motion vector may seldom change between frames.Average motion vector may change significantly between different plots, but in this case, beholder may ignore a frame or multiframe with the incorrect degree of depth, because human brain can need at least 3 in film to form the depth perception based on parallax to 4 frames.
The stereo pairs corresponding with picture frame 302 can be produced.Stereo pairs can comprise two images, that is, left image and right image.Left image can be produced by the particular value pixel in picture frame 302 being offset the degree of depth that is associated with each pixel left proportional.Right image can be produced by the particular value pixel in picture frame 302 being offset the degree of depth that is associated with each pixel to the right proportional.The 3D that stereo pairs forms the 2D picture frame received represents.
The 3D that stereo pairs forms the 2D picture frame received represents.Stereo pairs can correspond to the 3D content produced from the 2D content transmitted by external device (ED) (such as 104a).In addition, 3D content can be shown on the display screen be associated with calculation element 102.The method probes into the Germicidal efficacy of the three-dimensional perception about the mankind.Particularly, the method adopts the observation that only can define depth map accurately for the region in picture frame 302 with good texture.In addition, the method adopts beholder may to ignore when new film plot starts the observation of the jump of only mean depth in a frame.
Fig. 4 A to Fig. 4 C illustrates the exemplary plot according to the pixel column in the picture frame 302 of the embodiment of present disclosure.Element in composition graphs 3 carrys out key-drawing 4A to Fig. 4 C.Fig. 4 A illustrates and the pixel column in the picture frame 302 of the location overlap of object 306a.Each pixel in pixel groups 404 corresponds to object 306a, and only has speed along the X direction.Fig. 4 B illustrates and the pixel column in the picture frame 302 of the location overlap of object 306b.Each pixel in pixel groups 406 corresponds to object 306b, and have along the X direction with the speed of Y-direction.Fig. 4 C illustrates and the pixel column in the picture frame 302 of the location overlap of object 306c.Each pixel in pixel groups 408 corresponds to object 306c, and only has speed along the Y direction.Each pixel in pixel groups 402 corresponds to background.
In an embodiment, the motion vector corresponding with each pixel in picture frame 302 can be calculated dividually for X-direction and Y-direction.The motion vector calculated provides the information of the speed about each pixel in the X-direction in picture frame and Y-direction.In an embodiment, motion vector algorithm may can not provide the information of the reliability of the motion vector about each pixel.This problem can by calculating local texture, then using local texture to solve as motion vector reliability figure.The pixel with the absolute value of the spatial gradient more than threshold value can be considered to have reliable motion vector.Processor 202 can carry out the mean pixel speed in the X-direction of calculating chart picture frame 302 and Y-direction based on the motion vector calculated.The degree of depth of each pixel can be determined respectively based on the mean pixel speed in the X-direction of the speed in the X-direction of each pixel corresponded in pixel column (such as, 308a) and Y-direction and picture frame 302 and Y-direction.The formula (1) of previous description can be used for determining and the depth value that each pixel in pixel column (such as, 308a) is associated.The pixel only with the velocity vector of good definition participates in the calculating of average speed and depth value.The degree of depth with the pixel of the speed of good definition is designated as and equals zero (are background pixels as them).When watching 3D film, the degree of depth is assigned to these pixels of from the veined nearest region of tool, it being carried out to interpolation by human brain.
In an embodiment, processor 202 can operate the depth map for carrying out each pixel in calculating chart picture frame 302 based on the motion vector of pixel, the average vertical speed of background pixel and Mean Horizontal Velocity.Processor 202 can produce based on depth map the stereo pairs corresponding to picture frame 302.Stereo pairs can comprise left image and right image.Left image can be produced by foreground pixel being moved to the left particular value.Right image can be produced by the particular value that moved right by foreground pixel.The depth value that particular value can be associated with each pixel in same foreground pixel is proportional.
The stereo pairs comprising left image and right image forms the 3D content corresponding to picture frame 302.The stereoeffect of 3D content can be displayed on the I/O device 210 of calculation element 102.In an embodiment, 3D content can be sent to other display unit be associated with calculation element 102 by calculation element 102.Such as, 3D content can be sent to display unit (such as, monitor) by calculation element 102 (such as, smart phone).Therefore, calculation element 102 can be used to realize real-time 2D to 3D Content Transformation.
In an embodiment, proportionality coefficient can be used for percentage regulation allowance to adapt to the distance of beholder from screen.This can cause producing better 3D content from the 2D picture frame of correspondence.
In an embodiment, the size of picture frame 302 can be reduced the predefined factor (such as, in each direction twice) by processor 202 while calculation of motion vectors and/or depth map.When the depth map calculated with the direct image to original size compares, that calculate the frame of minification, that then original size frame is got back in application depth perception may be that human eye cannot be distinguished.However, present disclosure is not limited to this, and can adopt other the predefined factor when not limiting the scope of the disclosure.In addition, for the calculating of motion vector, processor 202 can adopt the luminance component of the pixel in the picture frame of picture frame 302 or minification.In an embodiment, for the calculating of motion vector, processor 202 can adopt the luminance component with the pixel of the texture in predefined scope in the picture frame of minification.
In an embodiment, processor 202 can in memory 206 storage pixel subset (such as, 308a), to calculate the degree of depth that is associated with each pixel.Therefore, the needs to full frame memory can be overcome, because memory 206 can a time point place storing one row pixel.Thus, can by method disclosed herein realize to memory use remarkable minimizing.In addition, the pixel resolution of downscaled images frame and the quantity reducing the pixel in a line can reduce the memory requirement for calculating further.In an embodiment, processor 202 can operate the luminance component of the pixel in the picture frame of the minification of the texture had in predefined scope, thus reduces calculating and memory requirement further.
According to another embodiment, computing time can be reduced by performing calculating concurrently to a line or multirow.However, present disclosure is not limited to this, and can adopt other technology for reducing computing time when not limiting the scope of the disclosure.
Said method can be stored on the chip that is associated with calculation element 102, to provide 2D to 3D Content Transformation.In addition, the method described in present disclosure can with motion vector calculation chip portfolio, to provide the self-contained unit that can be used for providing 2D to 3D Content Transformation.
Fig. 5 illustrates the flow chart for the illustrative steps that provides real-time 2D to 3D to change at calculation element place according to the embodiment of present disclosure.With reference to Fig. 5, method 500 is shown.The element of composition graphs 1, Fig. 2, Fig. 3 and Fig. 4 carrys out describing method 500.
Illustrative steps is started in step 502.In step 504, calculation element 102 can receive the 2D content transmitted by external device (ED) (such as, 104a).Processor 202 can receive the picture frame 302 corresponding with the 2D content transmitted by external device (ED) (such as, 104a).In step 506, the picture frame 302 corresponding to 2D content can be reduced the predefined factor, to obtain the picture frame with lower pixel resolution by processor 202.In step 508, processor 202 can calculate the motion vector with the pixel of the texture of more than threshold value in the picture frame with lower pixel resolution.In step 510, processor 202 can select the first row pixel had in the picture frame of lower pixel resolution.
In step 512, processor 202 can calculate the mean pixel speed of the pixel of the row of selection.In step 514, processor 202 can determine the depth value of each pixel in the row selected based on the mean pixel speed of the pixel of the row selected.In step 516, whether the row that processor 202 inspection is selected is last column of picture frame 302.Select capable be not last column when, present treatment proceeds to step 518.In step 518, processor 202 is selected next line and is proceeded to step 512.When the row selected is last column, present treatment proceeds to step 520.In step 520, processor can produce the stereo pairs corresponding with picture frame 302.Stereo pairs can be produced based on the depth value be associated with each pixel in picture frame 302.Method 500 terminates in step 522.
Fig. 6 illustrates the flow chart for the illustrative steps that provides real-time 2D to 3D to change at calculation element place according to another embodiment of present disclosure.With reference to Fig. 6, method 600 is shown.The element of composition graphs 1, Fig. 2, Fig. 3 and Fig. 4 carrys out describing method 600.
Illustrative steps is started in step 602.In step 604, calculation element 102 can receive the 2D content transmitted by external device (ED) (such as, 104a).Processor 202 can receive the picture frame 302 corresponding with the 2D content transmitted by external device (ED) (such as, 104a).In step 606, the picture frame 302 corresponding to 2D content can be reduced the predefined factor, to obtain the picture frame with lower pixel resolution by processor 202.In step 608, processor 202 can calculate the motion vector with the pixel of the texture of more than a certain threshold value in the picture frame with lower pixel resolution.In step 610, processor 202 can calculate the mean pixel speed of the picture frame received.
In step 612, processor 202 can select the first row pixel had in the picture frame of lower pixel resolution.In step 614, processor 202 can determine the depth value of each pixel in the row selected.In step 616, whether the row that processor 202 inspection is selected is last column of picture frame 302.Select capable be not last column when, present treatment proceeds to step 618.In step 618, processor 202 is selected next line and is proceeded to step 614.When the row selected is last column, present treatment proceeds to step 620.In step 620, processor can produce the stereo pairs corresponding with picture frame 302.Stereo pairs can be produced based on the depth value be associated with each pixel in picture frame 302.Method 600 terminates in step 622.
According to the embodiment of present disclosure, the equipment for Video processing can comprise calculation element 102 (Fig. 1), and this calculation element 102 can be coupled communicatedly with external device (ED) (such as, 104a, Fig. 1).Calculation element 102 can comprise one or more processor and/or circuit, such as, processor 202 (Fig. 2) can operate average vertical speed for determining the pixel subset (such as, 308a) in picture frame 302 (Fig. 3) and Mean Horizontal Velocity.One or more processor and/or circuitry operative are used for based on pixel subset (such as, the Mean Horizontal Velocity of the motion vector calculated of each pixel 308a), the average vertical speed of pixel subset and pixel subset is to determine the depth value of each pixel in pixel subset (such as, 308a).In an embodiment, one or more processor and/or circuitry operative are used for based on pixel subset (such as, the average level pixel speed of the motion vector calculated of each pixel 308a), the average vertical pixel speed of picture frame 302 and picture frame 302 is to determine the depth value of each pixel in pixel subset (such as, 308a).Average vertical pixel speed and average horizontal pixel speed can be determined based on the pixel in picture frame with predetermined motion vector.One or more processor and/or circuitry operative are used for producing based on the depth value of the determination of each pixel in picture frame 302 stereo pairs of picture frame 302.
One or more processor and/or circuit, such as, processor 202, can operate the stereo pairs for producing picture frame 302 based on depth map.One or more processor and/or circuit, such as, processor 202, can operate for producing left image and right image.Left image can be produced by each pixel in picture frame 302 is moved to the left particular value, and right image can be produced by the particular value that each pixel in picture frame 302 moved right.The depth value that the particular value corresponding with each pixel in picture frame 302 can be associated with same pixel is proportional.
One or more processor and/or circuit, such as, processor 202, can operate for receiving picture frame 302 from one or two in external device (ED) 104 (Fig. 1) and/or local storage (such as, 204, Fig. 2).It is one or more that external device (ED) 104 can comprise in video player 104a, Set Top Box 104b, personal video recorder 104c, Cloud Server 104d and/or video camera 104e.It is one or more that calculation element 102 can comprise in television set, smart phone, flat computer, laptop computer and/or desktop PC.
Other embodiment of present disclosure can provide non-transitory computer-readable medium and/or storage medium, and/or non-transient state machine readable media and/or storage medium, it stores the machine code and/or computer program with at least one code segment that can be performed by machine and/or computer, thus make this machine and/or computer perform the step be included in the calculation element that can be coupled communicatedly with the external device (ED) for Video processing.These steps can comprise average vertical speed and/or the Mean Horizontal Velocity of the pixel subset determined in picture frame.These steps can comprise: based on the motion vector calculated of each pixel in pixel subset, average vertical speed and Mean Horizontal Velocity, determine the depth value of each pixel in pixel subset.Pixel subset comprises the pixel with predetermined motion vector.
Therefore, present disclosure can realize with the combination of hardware or hardware and software.Present disclosure can realize in the mode of centralization at least one computer system, or can realize in a distributed fashion, and in distributed mode, different elements can be expanded in the middle of the computer system of several interconnection.The computer system or the miscellaneous equipment that are suitable for any type realizing method described herein may be applicable.The combination of hardware and software can be the general-purpose computing system with computer program, and this computer program can computer for controlling system when being loaded and perform, and makes computer system realize method described herein.Present disclosure can realize with the hardware comprising a part for the integrated circuit also performing other function.
Present disclosure also can be embedded in computer program, and this computer program comprises all features of the execution mode realizing method described herein, and can realize these methods when being loaded in computer systems, which.Computer program in the present context performs the one group of instruction represented with any language, code or mark of specific function after referring to one or two directly or in following of the system that is intended to make to have information processing capability: a) be converted to another kind of language, code or mark; B) reproduce with different material forms.
Although describe present disclosure for various embodiment, one skilled in the art will appreciate that when not departing from the scope of present disclosure, can various amendment be carried out, and each equivalent can be substituted.In addition, when not departing from the scope of present disclosure, multiple amendment can be carried out with the instruction making specific situation or material adapt to present disclosure.Therefore, present disclosure should be not limited to disclosed specific embodiment, and present disclosure will comprise all embodiments fallen in the scope of claims.
In addition, the application also can be configured as follows:
(1) for a method for Video processing, described method comprises:
In calculation element:
The depth value of each pixel in the described pixel subset in described picture frame is determined based on the average level pixel speed of the motion vector calculated of each pixel in the pixel subset in picture frame, the average vertical pixel speed of described picture frame and described picture frame
Wherein, described average vertical pixel speed and described average level pixel speed is determined based on the pixel in described picture frame with predetermined motion vector.
(2) for a method for Video processing, described method comprises:
In calculation element:
Determine average vertical speed and/or the Mean Horizontal Velocity of the pixel subset in picture frame; And
The depth value of the described each pixel in described pixel subset is determined based on the motion vector calculated of each pixel in described pixel subset, described average vertical speed and described Mean Horizontal Velocity,
Wherein, described pixel subset comprises the pixel with predetermined motion vector.
(3) method Gen Ju (2), the described depth value determined comprised based on the described each pixel in described picture frame produces the stereo pairs of described picture frame.
(4) method Gen Ju (3), wherein, the stereo pairs of described generation comprises:
Left image, wherein, the described each pixel in described picture frame offsets particular value left; And
Right image, wherein, the described each pixel in described picture frame offsets described particular value to the right,
Wherein, the described depth value determined that is associated with the described each pixel in described picture frame of described particular value is proportional.
(5) method Gen Ju (2), wherein, described pixel subset comprises a line in described picture frame or multirow pixel.
(6) method Gen Ju (2), wherein, the motion vector calculated described in the described each pixel in described pixel subset comprises the speed of described each pixel in X-direction and/or Y-direction.
(7) method Gen Ju (2), wherein, described picture frame is two-dimentional 2D picture frame.
(8) method Gen Ju (2), wherein, described picture frame is the two-dimentional 2D picture frame of the pixel resolution with minification.
(9) method Gen Ju (2), comprises and receives described picture frame from one or two external device (ED) and/or local storage.
(10) method Gen Ju (2), wherein, it is one or more that described calculation element comprises in television set, smart phone, flat computer, laptop computer and/or desktop PC.
(11) method Gen Ju (9), wherein, it is one or more that described external device (ED) comprises in video player, Set Top Box, personal video recorder, video camera and/or Cloud Server.
(12) for a calculation element for Video processing, described calculation element comprises:
One or more processor and/or circuit, can operate and be used for:
Determine average vertical speed and/or the Mean Horizontal Velocity of the pixel subset in picture frame; And
The depth value of the described each pixel in described pixel subset is determined based on the motion vector calculated of each pixel in described pixel subset, described average vertical speed and described Mean Horizontal Velocity,
Wherein, described pixel subset comprises the pixel with predetermined motion vector.
(13) calculation element Gen Ju (12), wherein, described one or more processor and/or circuit can operate the stereo pairs for producing described picture frame based on the described depth value determined of the described each pixel in described picture frame.
(14) calculation element Gen Ju (13), wherein, described one or more processor and/or circuit can operate for producing described stereo pairs based on left image and right image,
Wherein, produce described left image by the described each pixel in described picture frame is offset particular value left,
Wherein, produce described right image by the described each pixel in described picture frame is offset described particular value to the right, and
Wherein, the described depth value determined of the described each pixel in described particular value and described picture frame is proportional.
(15) calculation element Gen Ju (12), wherein, described one or more processor and/or circuit can operate for compute depth figure.
(16) calculation element Gen Ju (12), wherein, described pixel subset comprises a line in described picture frame or multirow pixel.
(17) calculation element Gen Ju (12), wherein, the motion vector calculated described in the described each pixel in described pixel subset comprises the speed of described each pixel in X-direction and/or Y-direction.
(18) calculation element Gen Ju (12), wherein, described one or more processor and/or circuit can operate for receiving described picture frame from one or two in external device (ED) and/or local storage.
(19) calculation element Gen Ju (18), wherein, it is one or more that described external device (ED) comprises in video player, Set Top Box, personal video recorder, video camera and/or Cloud Server.
(20) calculation element Gen Ju (12), wherein, it is one or more that described calculation element comprises in television set, smart phone, flat computer, laptop computer and/or desktop PC.
Claims (10)
1., for a method for Video processing, described method comprises:
In calculation element:
The depth value of each pixel in the described pixel subset in described picture frame is determined based on the average level pixel speed of the motion vector calculated of each pixel in the pixel subset in picture frame, the average vertical pixel speed of described picture frame and described picture frame
Wherein, described average vertical pixel speed and described average level pixel speed is determined based on the pixel in described picture frame with predetermined motion vector.
2., for a method for Video processing, described method comprises:
In calculation element:
Determine average vertical speed and/or the Mean Horizontal Velocity of the pixel subset in picture frame; And
The depth value of the described each pixel in described pixel subset is determined based on the motion vector calculated of each pixel in described pixel subset, described average vertical speed and described Mean Horizontal Velocity,
Wherein, described pixel subset comprises the pixel with predetermined motion vector.
3. method according to claim 2, the described depth value determined comprised based on the described each pixel in described picture frame produces the stereo pairs of described picture frame.
4. method according to claim 3, wherein, the stereo pairs of described generation comprises:
Left image, wherein, the described each pixel in described picture frame offsets particular value left; And
Right image, wherein, the described each pixel in described picture frame offsets described particular value to the right,
Wherein, the described depth value determined that is associated with the described each pixel in described picture frame of described particular value is proportional.
5. method according to claim 2, wherein, described pixel subset comprises a line in described picture frame or multirow pixel.
6. method according to claim 2, wherein, the motion vector calculated described in the described each pixel in described pixel subset comprises the speed of described each pixel in X-direction and/or Y-direction.
7. method according to claim 2, wherein, described picture frame is two-dimentional 2D picture frame.
8. method according to claim 2, wherein, described picture frame is the two-dimentional 2D picture frame of the pixel resolution with minification.
9. method according to claim 2, comprises and receives described picture frame from one or two external device (ED) and/or local storage.
10., for a calculation element for Video processing, described calculation element comprises:
One or more processor and/or circuit, can operate and be used for:
Determine average vertical speed and/or the Mean Horizontal Velocity of the pixel subset in picture frame; And
The depth value of the described each pixel in described pixel subset is determined based on the motion vector calculated of each pixel in described pixel subset, described average vertical speed and described Mean Horizontal Velocity,
Wherein, described pixel subset comprises the pixel with predetermined motion vector.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US13/911,389 US9483836B2 (en) | 2011-02-28 | 2013-06-06 | Method and apparatus for real-time conversion of 2-dimensional content to 3-dimensional content |
| US13/911,389 | 2013-06-06 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104243950A true CN104243950A (en) | 2014-12-24 |
| CN104243950B CN104243950B (en) | 2016-08-24 |
Family
ID=52144931
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410240725.8A Expired - Fee Related CN104243950B (en) | 2013-06-06 | 2014-06-03 | For the method and apparatus that 2 dimension contents are converted to 3-dimensional content in real time |
Country Status (2)
| Country | Link |
|---|---|
| JP (1) | JP5858254B2 (en) |
| CN (1) | CN104243950B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104615421A (en) * | 2014-12-30 | 2015-05-13 | 广州酷狗计算机科技有限公司 | Virtual gift display method and device |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111647807B (en) * | 2020-05-18 | 2022-03-04 | 樟树市兴隆高新材料有限公司 | High-alloy die steel and preparation process thereof |
| CN115786816B (en) * | 2022-12-14 | 2024-03-01 | 河钢股份有限公司 | Controllable eutectic carbide high-strength high-toughness cold-working die steel and preparation method thereof |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001359119A (en) * | 2000-06-15 | 2001-12-26 | Toshiba Corp | Stereoscopic video image generating method |
| US20110096832A1 (en) * | 2009-10-23 | 2011-04-28 | Qualcomm Incorporated | Depth map generation techniques for conversion of 2d video data to 3d video data |
| CN102724531A (en) * | 2012-06-05 | 2012-10-10 | 上海易维视科技有限公司 | Method and system for converting two-dimensional video into three-dimensional video |
| WO2013023325A1 (en) * | 2011-08-18 | 2013-02-21 | 北京世纪高蓝科技有限公司 | Method for converting 2d into 3d based on image motion information |
| CN103250184A (en) * | 2010-11-23 | 2013-08-14 | 高通股份有限公司 | Depth estimation based on global motion |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000261828A (en) * | 1999-03-04 | 2000-09-22 | Toshiba Corp | Stereoscopic video image generating method |
| JP3938122B2 (en) * | 2002-09-20 | 2007-06-27 | 日本電信電話株式会社 | Pseudo three-dimensional image generation apparatus, generation method, program therefor, and recording medium |
| JP2006014121A (en) * | 2004-06-29 | 2006-01-12 | Sanyo Electric Co Ltd | Image encoding method, image encoding apparatus, and imaging apparatus |
| CN102326394B (en) * | 2009-09-08 | 2014-06-11 | 株式会社东芝 | Image processing method and apparatus |
| KR20120049997A (en) * | 2010-11-10 | 2012-05-18 | 삼성전자주식회사 | Image process device, display apparatus and methods thereof |
-
2014
- 2014-06-02 JP JP2014113801A patent/JP5858254B2/en active Active
- 2014-06-03 CN CN201410240725.8A patent/CN104243950B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001359119A (en) * | 2000-06-15 | 2001-12-26 | Toshiba Corp | Stereoscopic video image generating method |
| US20110096832A1 (en) * | 2009-10-23 | 2011-04-28 | Qualcomm Incorporated | Depth map generation techniques for conversion of 2d video data to 3d video data |
| CN103250184A (en) * | 2010-11-23 | 2013-08-14 | 高通股份有限公司 | Depth estimation based on global motion |
| WO2013023325A1 (en) * | 2011-08-18 | 2013-02-21 | 北京世纪高蓝科技有限公司 | Method for converting 2d into 3d based on image motion information |
| CN102724531A (en) * | 2012-06-05 | 2012-10-10 | 上海易维视科技有限公司 | Method and system for converting two-dimensional video into three-dimensional video |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104615421A (en) * | 2014-12-30 | 2015-05-13 | 广州酷狗计算机科技有限公司 | Virtual gift display method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104243950B (en) | 2016-08-24 |
| JP2014239436A (en) | 2014-12-18 |
| JP5858254B2 (en) | 2016-02-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10440398B2 (en) | Probabilistic model to compress images for three-dimensional video | |
| WO2019149206A1 (en) | Depth estimation method and apparatus, electronic device, program, and medium | |
| JP6370708B2 (en) | Generation of a depth map for an input image using an exemplary approximate depth map associated with an exemplary similar image | |
| US9483836B2 (en) | Method and apparatus for real-time conversion of 2-dimensional content to 3-dimensional content | |
| CN102741879B (en) | Method for generating depth maps from monocular images and systems using the same | |
| US8441521B2 (en) | Method and apparatus for determining view of stereoscopic image for stereo synchronization | |
| WO2020143191A1 (en) | Image frame prediction method, image frame prediction apparatus and head display apparatus | |
| EP2214137B1 (en) | A method and apparatus for frame interpolation | |
| US20100073364A1 (en) | Conversion method and apparatus with depth map generation | |
| EP2594078A1 (en) | Vision-based quality metric for three dimensional video | |
| CN110869980A (en) | System and method for distribution and presentation of content as a spherical video and 3D portfolio | |
| US9661307B1 (en) | Depth map generation using motion cues for conversion of monoscopic visual content to stereoscopic 3D | |
| CN105204347A (en) | Method, device and system for smart home interaction based on augmented reality technologies | |
| US11494975B2 (en) | Method for analyzing three-dimensional model and device for analyzing three-dimensional model | |
| US9154772B2 (en) | Method and apparatus for converting 2D content into 3D content | |
| Yang et al. | Dynamic 3D scene depth reconstruction via optical flow field rectification | |
| CN104243950A (en) | Method and apparatus for real-time conversion of 2-dimensional content to 3-dimensional content | |
| US11825066B2 (en) | Video reproduction apparatus, reproduction method, and program | |
| WO2023147780A1 (en) | Video frame coding mode screening method and apparatus, and electronic device | |
| US20130057647A1 (en) | Apparatus and method for converting 2d content into 3d content | |
| JP6025740B2 (en) | Image processing apparatus using energy value, image processing method thereof, and display method | |
| US20240196065A1 (en) | Information processing apparatus and information processing method | |
| CN116665004B (en) | Augmented reality image processing method, system, equipment and storage medium | |
| US20120044244A1 (en) | Method for Image Processing and an Apparatus Thereof | |
| US20240185511A1 (en) | Information processing apparatus and information processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20160824 |