WO2014111968A1 - 映像生成装置、映像生成プログラム、及び映像生成方法 - Google Patents
映像生成装置、映像生成プログラム、及び映像生成方法 Download PDFInfo
- Publication number
- WO2014111968A1 WO2014111968A1 PCT/JP2013/000138 JP2013000138W WO2014111968A1 WO 2014111968 A1 WO2014111968 A1 WO 2014111968A1 JP 2013000138 W JP2013000138 W JP 2013000138W WO 2014111968 A1 WO2014111968 A1 WO 2014111968A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- line
- information
- sight
- stroke
- video generation
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims description 79
- 238000000605 extraction Methods 0.000 claims description 18
- 230000006870 function Effects 0.000 claims description 7
- 239000000284 extract Substances 0.000 claims description 3
- 238000007654 immersion Methods 0.000 abstract 1
- 238000004519 manufacturing process Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 102100035964 Gastrokine-2 Human genes 0.000 description 1
- 101001075215 Homo sapiens Gastrokine-2 Proteins 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 208000002173 dizziness Diseases 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 230000008929 regeneration Effects 0.000 description 1
- 238000011069 regeneration method Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
- G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
- G11B27/036—Insert-editing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/32—Digital ink
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
- G11B27/34—Indicating arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47205—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
Definitions
- the present invention relates to a video generation apparatus, a video generation program, and a video generation method for generating a making video that reproduces a state of drawing using an omnidirectional image editing apparatus.
- omnidirectional image editing apparatus when using the omnidirectional image editing apparatus (hereinafter simply referred to as “omnidirectional image editing apparatus”) described in the earlier application (see Patent Document 1) by the present applicant, the creator performs the steps of changing the line of sight and drawing. Repeat to complete the omnidirectional image. The line of sight is not moved during the stroke, and the stroke is continuously performed to some extent with one line of sight. Note that the “omnidirectional image” refers to an image that looks like a background covering the omnidirectional of the worker.
- a method for automatically generating a making picture by a conventional painting there are a method for recording a work screen, a method for recording stroke information, and the like.
- the method of recording the work screen reproduces the operation of the graphical user interface (GUI) used for the work and the enlarged display and rotation display of the work image that accompanies the work.
- Video that clearly presents the work procedure can be generated.
- a method of increasing the reproduction speed is often handled. For example, the reproduction time can be reduced to 10 minutes by setting the state of work for 60 minutes to 6 times speed.
- the method of recording stroke information can generate a video showing a stroke while always displaying the entire image of the work, and it is also possible to display no unnecessary information other than the stroke related to the completed work image. It is.
- a method of omitting the time other than the stroke is often handled. For example, the reproduction time can be shortened by not reproducing the time for carrying the pen from the end point of the line to the next start point, the time when no operation is performed, and the like.
- an omnidirectional image as a two-dimensional image that can be displayed on a general flat display (for example, a liquid crystal display), there is a “look-around image”.
- a look-around image is a natural image that is generally familiar, unlike a fish-eye lens image or a panoramic image, and is always represented as being in a certain direction. That is, it is necessary to move the line of sight in order to represent all directions in the omnidirectional image.
- Such a look image can effectively maintain an immersive feeling if the line-of-sight movement is smooth. This is similar to the point that camera work (for example, pan / tilt / zoom) is desired to be smooth in a camera used in actual shooting and a virtual camera used in computer graphics.
- camera work for example, pan / tilt / zoom
- the conventional method does not move the line of sight during the stroke. For this reason, for example, when the number of strokes with respect to one line of sight is enormous, the time during which no look is performed is long. On the other hand, when the number of strokes for one line of sight is small or does not exist, the look is excessively performed and the viewer is dizzy.
- the gaze movement in drawing work is not intended to be smooth, but is intended to be performed for a stroke with a new gaze, so that it is distorted as a result of reproducing this as it is Most cases.
- the present invention has been made in view of the technical problems as described above, and in the reproduction of omnidirectional image drawing, an automatic video that presents the state of handwriting in an easy-to-understand manner while effectively maintaining the immersive feeling of looking around.
- the purpose is to realize a typical generation.
- the video generation device records a line-of-sight information and a stroke information group by executing a storage unit storing a video generation program and the video generation program stored in the storage unit. From the reproduction sequence, the stroke information group is associated with the effective line-of-sight information, the number of frames between the line-of-sight information is constant, and when moving the line of sight from the first line of sight to the third line of sight, A central control unit for controlling to reproduce a stroke information group corresponding to the second line of sight is provided in a range from the first and second central frames to the second and third central frames. It is characterized by that.
- the video generation device extracts the effective line-of-sight information possessed by the reproduction sequence by executing the video generation program by storing the video generation program and the storage unit that stores the video generation program.
- a frame relating to line-of-sight information is selected, stroke information relating to the selected frame is extracted, and if the stroke information includes a drawing attribute, an omnidirectional image is updated based on the drawing attribute and the line-of-sight information.
- new line-of-sight information is generated by interpolation based on the extracted line-of-sight information, neighboring line-of-sight information, and the position of the extracted frame.
- the omnidirectional image is drawn on the screen image based on the obtained line-of-sight information, and the screen image is the image of the selected frame. And write it, and a central control unit for controlling so as to comprising the.
- the video generation program according to the third aspect of the present invention is such that the computer executes the video generation program stored in the storage unit, thereby enabling effective line-of-sight information from a reproduction sequence in which the line-of-sight information and the stroke information group are recorded.
- the stroke information group is associated with each other, the number of frames between the line-of-sight information is constant, and the line of sight moves from the first line of sight to the third line of sight, the first and second center In a range from the frame to the second and third central frames, it functions as a central control unit that controls to reproduce the stroke information group corresponding to the second line of sight.
- a video generation program including: a procedure for executing a video generation program stored in a storage unit; a procedure for extracting effective line-of-sight information included in a reproduction sequence; A procedure for selecting a frame relating to line-of-sight information; a procedure for extracting stroke information relating to the selected frame; and, if the stroke information includes a drawing attribute, an omni-direction based on the drawing attribute and the line-of-sight information
- new line-of-sight information is generated by interpolation based on the extracted line-of-sight information, neighboring line-of-sight information, and the position of the extracted frame.
- an omnidirectional image is drawn on the screen image based on the line-of-sight information generated by the interpolation, and the screen is displayed.
- the central control unit executes the video generation program stored in the storage unit, so that it is effective from the reproduction sequence in which the line-of-sight information and the stroke information group are recorded.
- the stroke information group is associated with the line-of-sight information, the number of frames between the line-of-sight information is constant, and the line of sight moves from the first line of sight to the third line of sight, the first and second A stroke information group corresponding to the second line of sight is reproduced in a range from a central frame to the second and third central frames.
- the central control unit executes a procedure for executing the video generation program stored in the storage unit, a procedure for extracting effective line-of-sight information included in the reproduction sequence, and the extraction Selecting a frame related to the selected line-of-sight information, extracting a stroke information related to the selected frame, and, if the stroke information includes a drawing attribute, based on the drawing attribute and the line-of-sight information
- the procedure for updating the omnidirectional image and the extraction of all the stroke information related to the selected frame are completed, new line-of-sight information is obtained based on the extracted line-of-sight information and the adjacent line-of-sight information and the position of the extracted frame.
- Interpolation is generated, and an omnidirectional image is drawn on the screen image based on the line-of-sight information generated by the interpolation.
- a video generation apparatus and a video generation program for realizing automatic generation of a video that presents the state of stroke in an easy-to-understand manner while effectively maintaining an immersive feeling by looking around And a video generation method.
- FIG. 1 It is a figure which shows the structure of the video production
- (A) And (b) is a figure which shows the kind and structure of the information currently recorded on the reproduction sequence. It is a flowchart which shows the flow of the whole process of the video generation program which concerns on one Embodiment of this invention. It is a figure which shows an example of a reproduction sequence. It is a figure which shows the concept which divides a reproduction sequence into a group. It is a figure which shows the concept of the correlation of a reproduction sequence. It is a figure which shows the example which has arrange
- (A) And (b) is a figure which shows the example of a video production
- FIG. 1 shows and describes a configuration of a video generation apparatus according to an embodiment of the present invention.
- the video generation device 1 includes a personal computer 10, a display device 20, a video parameter input device 30, and the like.
- the personal computer 10 includes a central control unit 11 including a multitask processor, a main memory 12 including a RAM (Random Access Memory) as a temporary storage device, an image control unit 13 such as a graphic card, and an input / output control unit 14.
- the image control unit 13 also includes a video memory 13a. Similar to the main memory 12 of the personal computer 10 main body, the video memory 13a is a place for temporarily storing data, and the memory attached to the graphic card is also referred to as VRAM (Video RAM). When a 3D graphic image is displayed on the screen, the amount of data required at that time increases. The data that has been processed by the image controller 13 is stored in the video memory 13a and used as needed. As the capacity of the video memory 13a is increased, even a fine 3D graphic image can be displayed smoothly and without a defect. In recent years, the speed of VRAM has been increased, and a memory standard dedicated to high-speed processing called GDDR (Graphics DDR) has appeared, and high-speed transfer of enormous data in three-dimensional graphics drawing has been realized.
- GDDR Graphics DDR
- the display device 20 is a device that can display an image, represented by a liquid crystal display or the like.
- the video parameter input device 30 is a device that allows the user to arbitrarily specify adjustments related to the look video output data 41 (for example, adjustment of line-of-sight information, playback speed, image quality, etc.), that is, representative of a mouse, a touch panel, and a pen tablet.
- Program data 50, orientation pixel association data 42, and sequence input data 40 are input to the personal computer 10 via the media read / write interface 16, and look-out video output data 41 is output from the personal computer 10 to the outside via the media read / write interface 16. Is done.
- the program data 50 is software capable of operating the present invention. This corresponds to data of a video generation program, which will be described later.
- the azimuth image correlation data 42 is a table or function that correlates the azimuth and the pixel position. When a polygon model is used, the polygon model data corresponds to this and becomes a three-dimensional object.
- the azimuth image association data 42 may accompany the program data 50, or may read externally defined data.
- the sequence input data 40 is data handled by software, is data in which drawing operations in the omnidirectional image editing apparatus are sequentially recorded, and is used as a reproduction sequence described later.
- the input data is temporarily stored in the main memory 12.
- the look video output data 41 is an image group handled by software, and is output as a continuous image, that is, video data.
- the orientation image association data 42 and the program data 50 may be read from an external storage medium (not shown), received from an external computer (not shown) via a communication network by a communication control unit (not shown), and the input / output control unit 14 Via the built-in nonvolatile storage medium 15.
- the look video output data 41 may be written to an external storage medium (not shown) or transmitted to an external computer (not shown) via a communication network.
- FIG. 2 shows a memory map of the main memory 12 used for video generation.
- Each image has color information with opacity (a: alpha, r: red, g: green, b: blue) as a two-dimensional array.
- “Alpha” indicates opacity information.
- PNG with alpha 32-bit PNG; 32-bit Portable Network-Graphics
- An alpha value of zero means completely transparent, 255 means completely opaque.
- the omnidirectional image related information is information for representing a background (that is, an omnidirectional image) covering the omnidirectional of the creator in the omnidirectional image editing apparatus and the viewer in the present invention.
- This figure is an example of using a polygon model.
- Polygon vertices have three-dimensional spatial coordinates (x: horizontal, y: vertical, z: depth) and two-dimensional texture coordinates (u: horizontal, v: vertical). Since the polygon surface represents a triangle, it has three references to polygon vertices and references to texture images for the number of layers. Since the polygon model represents a three-dimensional shape, it has polygon vertices and polygon surfaces as arrays.
- the reference to the texture image may be one relative reference within the layer.
- the screen image is an image for one frame that is projected as an omnidirectional image on a two-dimensional coordinate plane by coordinate transformation and presented to the creator by the display device 20 or output as the look-up video output data 41.
- the process of updating the omnidirectional image is mainly projected on the screen image and presented to the viewer.
- Drawing is an operation of drawing figures, lines, etc. on a two-dimensional (planar) image mainly using a pointing device in an omnidirectional image editing apparatus.
- a drawing image is a two-dimensional (planar) image that is a target on which the creator actually draws.
- the drawing image is in a storage area different from the screen image, has the same coordinate system as the screen image, and is completely transparent before the drawing starts.
- the opacity information and the color information are updated at the location where the drawing is performed. This is equivalent to the creator drawing on a transparent layer overlying the screen image.
- Examples of the drawing method include freehand, straight line, curved line, rectangle, circle, image pasting, and the like.
- the drawing attribute may be a width or color when the method is a line, a filled pattern when the method is a figure, and the like. In the present invention, the drawing is automatically performed by a reproduction sequence described later.
- the drawing image is also a temporary image until the contents of the drawing are reflected in the omnidirectional image, and may be used as necessary. For example, if it is not necessary to return to the line-of-sight change mode (correct the line of sight) again before drawing ends in drawing, and the drawing rule is to immediately reflect the drawing contents in the omnidirectional image, do not use the drawing image. Is also possible.
- the reproduction sequence is data in which drawing operations in the omnidirectional image editing apparatus are sequentially recorded as information, and is used for updating the omnidirectional image in the present invention.
- FIG. 3 shows details and types of information recorded in the reproduction sequence.
- ⁇ There are at least two types of information as elements of the reproduction sequence, mainly gaze information and stroke information.
- Gaze information mainly has a gaze angle (yaw angle, pitch angle, roll angle) and viewing angle.
- the line-of-sight information may include a shift amount (horizontal shift, vertical shift).
- Shift is a method of translating images.For example, in the expression of perspective using an omnidirectional image editing device, the viewing angle of the ground from a distance to the horizon, the upper part of a nearby high building, etc. It can be expressed in detail without enlarging.
- the stroke information has a drawing attribute and / or a drawing end flag.
- the drawing attributes possessed by the stroke information are, for example, a case where the drawing method is a straight line, and start and end coordinates (x: horizontal pixel, y: vertical pixel), color information (a: alpha, r: red, g: Green, b: blue), line width.
- start and end coordinates x: horizontal pixel, y: vertical pixel
- color information a: alpha, r: red, g: Green, b: blue
- line width line width
- a curve or a freehand it can be approximated by using a plurality of short straight lines.
- the drawing end flag included in the stroke information is a flag for determining whether to perform a drawing end process (that is, a process of drawing a drawing image drawn so far in an omnidirectional image and clearing the drawing image). .
- the drawing is performed in a state of being directed in some direction in principle. That is, the stroke information included in the reproduction sequence is related to any line-of-sight information.
- the drawing image is used as an example, and after the stroke information is extracted, the drawing image is updated and the omnidirectional image is updated. Similarly, the process of drawing on the screen image is similarly divided.
- the figure does not include the concept of layers, but when using layers, pay attention to the order in which omnidirectional images and drawing images are drawn on screen images.
- step S1 it is first determined whether or not extraction of all valid line-of-sight information included in the reproduction sequence has been completed.
- step S2 since extraction has not been completed yet, the process branches to No, and the next effective line-of-sight information is extracted (step S2).
- step S3 it is determined whether or not all the frames related to the extracted line-of-sight information have been selected.
- step S3 since selection has not been completed yet, the process branches to No and the next frame is selected (step S4).
- step S5 it is determined whether or not extraction of all stroke information related to the selected frame has been completed. If the extraction has not been completed yet, the process branches to No, and the next stroke information is extracted (step S6). However, if there is no stroke information related to the selected frame, extraction is completed as it is, step S5 is branched to Yes, and the process proceeds to step S12.
- step S7 it is determined whether or not the stroke information includes a drawing attribute. If it is included, the process branches to Yes, the drawing image is updated based on the drawing attribute (step S8), and the process proceeds to step S9. On the other hand, if not included, the process proceeds to step S9 as it is.
- step S9 it is determined whether or not the stroke information includes a drawing end flag. If yes, the process branches to Yes, the drawing image is drawn on the omnidirectional image based on the extracted line-of-sight information (step S10), the drawing image is cleared (step S11), and the process goes to step S5. Return.
- step S5 it is determined again whether or not all the stroke information related to the selected frame has been extracted. If the extraction has been completed, the process branches to Yes, and new line-of-sight information is generated by interpolation based on the extracted line-of-sight information, neighboring line-of-sight information, and the extracted frame position (step S12).
- An omnidirectional image is drawn on the screen image based on the line-of-sight information generated by interpolation (step S13), and a drawing image is drawn on the screen image based on the line-of-sight information generated by interpolation and the extracted line-of-sight information (step S14).
- the screen image is written out as an image of the selected frame (step S15), and the process returns to step S3.
- “exporting as an image of a selected frame” is a process of generating an image to be presented to the viewer.
- step S3 it is determined again whether or not selection of all the frames related to the extracted line-of-sight information has been completed. If it is determined that the selection has been completed, the process branches to Yes and returns to step S1. In step S1, it is determined again whether or not all effective line-of-sight information included in the reproduction sequence has been extracted (step S1). If extraction has not been completed, the processes of steps S1 to S15 as described above are repeated. . Then, when the extraction is completed, this process is terminated.
- FIG. 5 shows an example of the reproduction sequence.
- This figure is an example of a reproduction sequence with a total of 42 elements with numbers 0-41.
- the line-of-sight information is represented by “ ⁇ ”
- the stroke information having the drawing attribute is represented by “ ⁇ ”
- the stroke information having the drawing end flag is represented by “ ⁇ ”.
- Fig. 6 shows the concept of dividing the above-mentioned reproduction sequence into groups.
- one group is represented by one line. Stroke information with a drawing end flag is the end of the group. If there is a group for which no line-of-sight information exists, the group may be combined with the immediately preceding group.
- Fig. 7 shows the concept of replay sequence association.
- the last line-of-sight information in the group is regarded as valid line-of-sight information. Gaze information that is not considered valid is discarded. As a result, the line-of-sight information of each group can be used correctly, and at the same time, the line-of-sight information having no associated stroke information can be eliminated.
- the line-of-sight information group is represented as V
- the line-of-sight information with the subscript j is represented as V [j]. That is, the subscript of the line-of-sight information group V is also a group number.
- the stroke information group is represented as S
- the stroke information with the suffix i is represented as S [i].
- a stroke head position list related to the line-of-sight information V [j] is expressed as p
- a head position of stroke information related to the line-of-sight information V [j] of the subscript j is expressed as p [j]. That is, the head stroke information related to the line-of-sight information V [j] of the subscript j is represented by S [p [j]].
- the stroke information group related to the line-of-sight information V [j] includes stroke information whose subscript is greater than or equal to p [j] and less than p [j + 1] (that is, S [p [j] +0], S [p [j ] +1], S [p [j] +2],..., S [p [j + 1] -1]), the total number of stroke information related to the line-of-sight information V [j] of the subscript j is p [ j + 1] ⁇ p [j].
- the above is an example of associating a reproduction sequence.
- other methods may be used for association.
- a method suitable for the drawing rule in the omnidirectional image editing apparatus that is the basis of the reproduction sequence is desired.
- FIG. 8 shows an example in which the reproduction sequence and the reproduction frame are arranged in time series.
- the example in the figure represents the relationship between the above-described line-of-sight information and stroke information, and the playback frames for video generation in time series.
- the line-of-sight information used for drawing is a total of four V [0] to V [3] based on the above example.
- the time between each line-of-sight information is made equal.
- a group of information related to one line-of-sight information is defined as a group. For example, the group number associated with V [0] is 0.
- the boundary between the groups is assumed to be near the center of two adjacent line-of-sight information.
- the number of playback frames is 10 frames per line-of-sight information (that is, per group), for a total of 40 frames. That is, The frame number related to V [0] is 0 to 9 The frame numbers associated with V [1] are 10-19. The frame numbers related to V [2] are 20 to 29. The frame number associated with V [3] is 30 to 39. And The time between each frame is set at an equal interval, and the time of the line-of-sight information is set near the center of the frame group related thereto.
- the time of each frame (that is, frame numbers 10 to 19) is adjusted so that the time of V [1] is the center time of frame number 10 and frame number 20 (that is, frame number 15).
- the time of frame number 10 is the central time of V [0] and V [1].
- the fixed number of frames may actually allow an error of several frames (about 1 to 2 frames) (if you want to generate a total of 50 frames of video in 4 groups, the frames of each group The number will be 12 or 13.)
- the stroke information group related to V [0] is a total of five stroke information groups S [0] to S [4].
- the value of p [4] is obtained by adding 1 to the last subscript (31) of the stroke information, that is, used for obtaining the total number of stroke information in the terminal data V [3].
- the time between each stroke information is set at equal intervals, and the time of the line-of-sight information is set near the center of the stroke information group related thereto.
- the time of each stroke information (S [0] to S [4]) is adjusted so that the time of V [0] is the center time of S [0] and S [5].
- stroke information (group) in the range after the time of the immediately preceding frame and below the time of the current frame, that is, S [8 ] To S [9] are used.
- the line-of-sight information when writing a stroke is V [1] because it is the line-of-sight information of the group with which frame number 13 is related (ie, group number 1).
- CatmullRom (V [ ⁇ 1], V [0], V [1], V [2], (10 ⁇ 5) ⁇ 10) is calculated for each of the yaw angle, pitch angle, roll angle, and viewing angle. And interpolate. However, since V [ ⁇ 1] does not exist, V [0] which is the leading end of the data is used instead, and CatmullRom (V [0], V [0], V [1], V [2], (10 Calculate -5) ⁇ 10). By using the curve interpolation method, the line-of-sight movement becomes curvilinear and smooth, and a sudden change of direction of the line of sight can be prevented.
- the frame image is generated using the omnidirectional image, the drawing image, and the line of sight prepared as described above.
- FIG. 9 shows a video generation example (drawing, video generation).
- FIG. 9 is an example of image generation when a plurality of rectangles are drawn in various directions.
- the creator draws one rectangle on the front, draws three rectangles facing downward, draws two rectangles facing the front right, and draws one rectangle on the back. Shall.
- the drawing method is all straight, and four pieces of stroke information having a drawing attribute are used for each rectangle. Just before the movement of the line of sight, one piece of stroke information having a drawing end flag is used.
- the generated image moves smoothly while passing through the line of sight at the time of drawing (except for the line of sight that was discarded, and so on).
- the stroke of the group including the line of sight information is completed about half.
- the group is switched when the frame passes through the vicinity of the center of the line of sight at the time of drawing and the next line of sight (for example, frame number 10), and the stroke of the next group starts.
- the present invention includes the following.
- the line-of-sight information and the stroke are obtained by executing the main memory 12 as a storage unit that stores a video generation program (program data 50) and the video generation program stored in the storage unit.
- the stroke information group is associated with the effective line-of-sight information, the number of frames between the line-of-sight information is constant, and the line of sight passes from the first line of sight to the third line of sight.
- a video generation device characterized by comprising the control unit 11 is provided.
- the main memory 12 as a storage unit for storing the video generation program (program data 50), and the effective line-of-sight information possessed by the reproduction sequence can be obtained by executing the video generation program. Extracting, selecting a frame relating to the extracted line-of-sight information, extracting stroke information relating to the selected frame, and if the stroke information includes a drawing attribute, the drawing attribute and the line-of-sight information are When the omnidirectional image is updated and all the stroke information related to the selected frame is extracted, new line-of-sight information is interpolated based on the extracted line-of-sight information, neighboring line-of-sight information, and the position of the extracted frame.
- the line-of-sight information and the stroke information group are recorded by executing the video generation program (program data 50) stored in the main memory 12 as the storage unit.
- the stroke information group is associated with the effective line-of-sight information, the number of frames between the line-of-sight information is constant, and when moving the line of sight from the first line of sight to the third line of sight, A range from the first and second central frames to the second and third central frames is made to function as a central controller 11 that controls to reproduce the stroke information group corresponding to the second line of sight.
- a video generation program characterized by the above is provided.
- the computer 10 extracts the procedure for executing the video generation program (program data 50) stored in the main memory 12 as the storage unit and the effective line-of-sight information possessed by the reproduction sequence.
- the extracted line-of-sight information, the adjacent line-of-sight information, and the position of the extracted frame are used.
- New line-of-sight information is generated by interpolation, and an omnidirectional image is generated based on the line-of-sight information generated by the interpolation.
- the image generation program for causing to function as a central control unit 11 for controlling to execute the procedure to write the screen image as the image of the selected frame, it is provided.
- the central control unit 11 executes the video generation program (program data 50) stored in the main memory 12 as the storage unit, whereby the line-of-sight information and the stroke information group are recorded.
- the stroke information group is associated with the effective line-of-sight information from the reproduced sequence, the number of frames between the line-of-sight information is fixed, and the line of sight moves from the first line of sight to the third line of sight via the second line of sight
- a video generation method comprising: reproducing a stroke information group corresponding to the second line of sight in a range from the first and second central frames to the second and third central frames. Is provided.
- the central control unit 11 executes a video generation program (program data 50) stored in the main memory 12 serving as a storage unit, and effective line-of-sight information included in the reproduction sequence.
- a procedure for selecting a frame related to the extracted line-of-sight information, a procedure for extracting stroke information related to the selected frame, and, if the stroke information includes a drawing attribute, the drawing The procedure for updating the omnidirectional image based on the attribute and the line-of-sight information, and when the extraction of all the stroke information related to the selected frame is completed, the extracted line-of-sight information, the adjacent line-of-sight information, and the position of the extracted frame New gaze information is generated by interpolation based on the omnidirectional image based on the generated gaze information
- the image generation method characterized by performing the procedure to write the screen image as the image of the selected frame, is provided.
- the present invention generates an image in which the state in which the omnidirectional image is updated (that is, the state of the stroke) can be observed with a high probability, so that the state of the stroke can be presented in an easily understandable manner.
- the video is automatically generated using the data storing the operations performed by the creator, it is possible to reduce the burden of work related to the video generation.
- the stroke playback speed is not always constant. However, it is considered that the change in the stroke reproduction speed does not affect the immersive feeling as much as the change in the movement speed of the line of sight.
- the present invention allows a smooth line of sight movement by allowing a change in the stroke reproduction speed, and can be expected to effectively maintain the immersive feeling by looking around.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Databases & Information Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Processing Or Creating Images (AREA)
- User Interface Of Digital Computer (AREA)
- Architecture (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
Abstract
Description
すなわち、
V[0]に関連するフレーム番号を0~9
V[1]に関連するフレーム番号を10~19
V[2]に関連するフレーム番号を20~29
V[3]に関連するフレーム番号を30~39
とする。
各フレーム間の時間は等間隔になるようにし、さらに視線情報の時刻が、それに関連するフレーム群の中央付近となるようにする。
V[0]に関連するストローク情報群をS[0]~S[4]の合計5個
V[1]に関連するストローク情報群をS[5]~S[17]の合計13個
V[2]に関連するストローク情報群をS[18]~S[26]の合計9個
V[3]に関連するストローク情報群をS[27]~S[31]の合計5個
とする。
このとき、ストローク先頭位置pの内容は、p[0]=0、p[1]=5、p[2]=18、p[3]=27、p[4]=32となる。なお、p[4]の値はストローク情報の最後の添字(31)に1を加えたものであり、すなわち、終端のデータV[3]においてストローク情報総数を求めるために用いる。
本発明の第1の態様によれば、映像生成プログラム(プログラムデータ50)を記憶する記憶部たるメインメモリ12と、前記記憶部に記憶された映像生成プログラムを実行することで、視線情報およびストローク情報群が記録された再現シーケンスから、有効な視線情報に関してストローク情報群を対応付け、視線情報間のフレーム数を一定とし、第1の視線から第2の視線を経て第3の視線へと視線を移動するときには、前記第1及び第2の中央のフレームから前記第2及び第3の中央のフレームまでの範囲には、前記第2の視線に対応するストローク情報群を再生するよう制御する中央制御部11と、を備えたことを特徴とする映像生成装置が提供される。
10 パーソナルコンピュータ
11 中央制御部
12 メインメモリ
13 画像制御部
13aビデオメモリ
14 入出力制御部
15 内蔵不揮発性記憶媒体
16 メディア読み書きインターフェイス
20 表示装置
30 映像パラメータ入力装置
40 シーケンス入力データ
41 見回し映像出力データ
42 方位画素対応付けデータ
50 プログラムデータ
Claims (15)
- 映像生成プログラムを記憶する記憶部と、
前記記憶部に記憶された映像生成プログラムを実行することで、視線情報およびストローク情報群が記録された再現シーケンスから、有効な視線情報に関してストローク情報群を対応付け、視線情報間のフレーム数を一定とし、第1の視線から第2の視線を経て第3の視線へと視線を移動するときには、前記第1及び第2の中央のフレームから前記第2及び第3の中央のフレームまでの範囲には、前記第2の視線に対応するストローク情報群を再生するよう制御する中央制御部と、を備えたこと
を特徴とする映像生成装置。 - 映像生成プログラムを記憶する記憶部と、
前記映像生成プログラムを実行することで、
再現シーケンスが持つ有効な視線情報を抽出し、
前記抽出された視線情報に関するフレームを選択し、
前記選択されたフレームに関するストローク情報を抽出し、
前記ストローク情報がドローイング属性を含んでいる場合には、該ドローイング属性および前記視線情報を基に全方位画像を更新し、
前記選択されたフレームに関する全てのストローク情報を抽出完了すると、前記抽出された視線情報および近傍の視線情報ならびに抽出されたフレームの位置を基に新たな視線情報を補間生成し、この補間生成された視線情報を基に全方位画像をスクリーン画像に描画し、該スクリーン画像を選択フレームの画像として書き出す、
ように制御する中央制御部と、を備えたこと
を特徴とする映像生成装置。 - 前記再現シーケンスは、視線情報とストローク情報を含み、
前記視線情報は視線角度や視野角の情報を含み、
前記ストローク情報はドローイング属性を含むこと
を特徴とする請求項2に記載の映像生成装置。 - 前記ドローイング属性は、始点座標、終点座標、色情報、線の幅の情報の少なくともいずれかを含むこと
を特徴とする請求項3に記載の映像生成装置。 - 前記ストローク情報はドローイング終了フラグを含み、
前記中央制御部は、前記ストローク情報がドローイング属性を含んでいる場合には、該ドローイング属性を基にドローイング画像を更新し、前記ストローク情報がドローイング終了フラグを含んでいる場合には、前記視線情報を基に前記ドローイング画像を全方位画像に描画し、前記補間生成された視線情報および抽出された視線情報を基にドローイング画像をスクリーン画像に描画すること
を特徴とする請求項2乃至4のいずれかに記載の映像生成装置。 - コンピュータを、
記憶部に記憶された映像生成プログラムを実行することで、視線情報およびストローク情報群が記録された再現シーケンスから、有効な視線情報に関してストローク情報群を対応付け、視線情報間のフレーム数を一定とし、第1の視線から第2の視線を経て第3の視線へと視線を移動するときには、前記第1及び第2の中央のフレームから前記第2及び第3の中央のフレームまでの範囲には、前記第2の視線に対応するストローク情報群を再生するよう制御する中央制御部として機能させること
を特徴とする映像生成プログラム。 - コンピュータを、
記憶部に記憶された映像生成プログラムを実行する手順と、
再現シーケンスが持つ有効な視線情報を抽出する手順と、
前記抽出された視線情報に関するフレームを選択する手順と、
前記選択されたフレームに関するストローク情報を抽出する手順と、
前記ストローク情報がドローイング属性を含んでいる場合には、該ドローイング属性および前記視線情報を基に全方位画像を更新する手順と、
前記選択されたフレームに関する全てのストローク情報を抽出完了すると、前記抽出された視線情報および近傍の視線情報ならびに抽出されたフレームの位置を基に新たな視線情報を補間生成し、この補間生成された視線情報を基に全方位画像をスクリーン画像に描画し、該スクリーン画像を選択フレームの画像として書き出す手順と、
を実行するように制御する中央制御部として機能させること
を特徴とする映像生成プログラム。 - 前記再現シーケンスは、視線情報とストローク情報を含み、
前記視線情報は視線角度や視野角の情報を含み、
前記ストローク情報はドローイング属性を含むこと
を特徴とする請求項7に記載の映像生成プログラム。 - 前記ドローイング属性は、始点座標、終点座標、色情報、線の幅の情報の少なくともいずれかを含むこと
を特徴とする請求項8に記載の映像生成プログラム。 - 前記ストローク情報はドローイング終了フラグを含み、
前記中央制御部は、前記ストローク情報がドローイング属性を含んでいる場合には、該ドローイング属性を基にドローイング画像を更新し、前記ストローク情報がドローイング終了フラグを含んでいる場合には、前記視線情報を基に前記ドローイング画像を全方位画像に描画し、前記補間生成された視線情報および抽出された視線情報を基にドローイング画像をスクリーン画像に描画すること
を特徴とする請求項7乃至9のいずれかに記載の映像生成プログラム。 - 中央制御部が、記憶部に記憶された映像生成プログラムを実行することで、視線情報およびストローク情報群が記録された再現シーケンスから、有効な視線情報に関してストローク情報群を対応付け、視線情報間のフレーム数を一定とし、第1の視線から第2の視線を経て第3の視線へと視線を移動するときには、前記第1及び第2の中央のフレームから前記第2及び第3の中央のフレームまでの範囲には、前記第2の視線に対応するストローク情報群を再生すること
を特徴とする映像生成方法。 - 中央制御部が、
記憶部に記憶された映像生成プログラムを実行する手順と、
再現シーケンスが持つ有効な視線情報を抽出する手順と、
前記抽出された視線情報に関するフレームを選択する手順と、
前記選択されたフレームに関するストローク情報を抽出する手順と、
前記ストローク情報がドローイング属性を含んでいる場合には、該ドローイング属性および前記視線情報を基に全方位画像を更新する手順と、
前記選択されたフレームに関する全てのストローク情報を抽出完了すると、前記抽出された視線情報および近傍の視線情報ならびに抽出されたフレームの位置を基に新たな視線情報を補間生成し、この補間生成された視線情報を基に全方位画像をスクリーン画像に描画し、該スクリーン画像を選択フレームの画像として書き出す手順と、
を実行すること
を特徴とする映像生成方法。 - 前記再現シーケンスは、視線情報とストローク情報を含み、
前記視線情報は視線角度や視野角の情報を含み、
前記ストローク情報はドローイング属性を含むこと
を特徴とする請求項12に記載の映像生成方法。 - 前記ドローイング属性は、始点座標、終点座標、色情報、線の幅の情報の少なくともいずれかを含むこと
を特徴とする請求項13に記載の映像生成方法。 - 前記ストローク情報はドローイング終了フラグを含み、
前記中央制御部は、前記ストローク情報がドローイング属性を含んでいる場合には、該ドローイング属性を基にドローイング画像を更新し、前記ストローク情報がドローイング終了フラグを含んでいる場合には、前記視線情報を基に前記ドローイング画像を全方位画像に描画し、前記補間生成された視線情報および抽出された視線情報を基にドローイング画像をスクリーン画像に描画すること
を特徴とする請求項12乃至14のいずれかに記載の映像生成方法。
Priority Applications (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2014557173A JP5980960B2 (ja) | 2013-01-16 | 2013-01-16 | 映像生成装置、映像生成プログラム、及び映像生成方法 |
US14/761,537 US9824721B2 (en) | 2013-01-16 | 2013-01-16 | Video generation device, video generation program, and video generation method |
PCT/JP2013/000138 WO2014111968A1 (ja) | 2013-01-16 | 2013-01-16 | 映像生成装置、映像生成プログラム、及び映像生成方法 |
KR1020157018950A KR102045413B1 (ko) | 2013-01-16 | 2013-01-16 | 영상 생성 장치, 영상 생성 프로그램이 저장된 매체, 및 영상 생성 방법 |
US15/816,184 US10249340B2 (en) | 2013-01-16 | 2017-11-17 | Video generation device, video generation program, and video generation method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2013/000138 WO2014111968A1 (ja) | 2013-01-16 | 2013-01-16 | 映像生成装置、映像生成プログラム、及び映像生成方法 |
Related Child Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/761,537 A-371-Of-International US9824721B2 (en) | 2013-01-16 | 2013-01-16 | Video generation device, video generation program, and video generation method |
US15/816,184 Division US10249340B2 (en) | 2013-01-16 | 2017-11-17 | Video generation device, video generation program, and video generation method |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2014111968A1 true WO2014111968A1 (ja) | 2014-07-24 |
Family
ID=51209103
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2013/000138 WO2014111968A1 (ja) | 2013-01-16 | 2013-01-16 | 映像生成装置、映像生成プログラム、及び映像生成方法 |
Country Status (4)
Country | Link |
---|---|
US (2) | US9824721B2 (ja) |
JP (1) | JP5980960B2 (ja) |
KR (1) | KR102045413B1 (ja) |
WO (1) | WO2014111968A1 (ja) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10104288B2 (en) * | 2017-02-08 | 2018-10-16 | Aspeed Technology Inc. | Method and apparatus for generating panoramic image with stitching process |
US10810700B2 (en) | 2019-03-05 | 2020-10-20 | Aspeed Technology Inc. | Method of adjusting texture coordinates based on control regions in a panoramic image |
CN111739382A (zh) * | 2020-06-17 | 2020-10-02 | 卢轩 | 一种利用ai辅助教学的书法教学装置 |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2012147303A1 (ja) * | 2011-04-25 | 2012-11-01 | Hayashi Mitsuo | 全方位画像編集プログラム及び全方位画像編集装置 |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0723689B1 (en) * | 1993-10-15 | 1998-02-04 | EVANS & SUTHERLAND COMPUTER CORPORATION | Direct rendering of textured height fields |
JP2001008232A (ja) * | 1999-06-25 | 2001-01-12 | Matsushita Electric Ind Co Ltd | 全方位映像出力方法と装置 |
US20020051081A1 (en) * | 2000-06-30 | 2002-05-02 | Osamu Hori | Special reproduction control information describing method, special reproduction control information creating apparatus and method therefor, and video reproduction apparatus and method therefor |
JP4297010B2 (ja) * | 2004-08-13 | 2009-07-15 | ソニー株式会社 | 情報処理装置および情報処理方法、並びに、プログラム |
JP4752921B2 (ja) * | 2009-01-28 | 2011-08-17 | ソニー株式会社 | 情報処理装置、アニメーション付加方法、及びプログラム |
US9202309B2 (en) * | 2011-09-01 | 2015-12-01 | Adobe Systems Incorporated | Methods and apparatus for digital stereo drawing |
-
2013
- 2013-01-16 WO PCT/JP2013/000138 patent/WO2014111968A1/ja active Application Filing
- 2013-01-16 JP JP2014557173A patent/JP5980960B2/ja active Active
- 2013-01-16 US US14/761,537 patent/US9824721B2/en active Active
- 2013-01-16 KR KR1020157018950A patent/KR102045413B1/ko active IP Right Grant
-
2017
- 2017-11-17 US US15/816,184 patent/US10249340B2/en active Active
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2012147303A1 (ja) * | 2011-04-25 | 2012-11-01 | Hayashi Mitsuo | 全方位画像編集プログラム及び全方位画像編集装置 |
Also Published As
Publication number | Publication date |
---|---|
US10249340B2 (en) | 2019-04-02 |
JP5980960B2 (ja) | 2016-08-31 |
US9824721B2 (en) | 2017-11-21 |
KR20150107747A (ko) | 2015-09-23 |
KR102045413B1 (ko) | 2019-11-15 |
JPWO2014111968A1 (ja) | 2017-01-19 |
US20180075880A1 (en) | 2018-03-15 |
US20150364157A1 (en) | 2015-12-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5896424B2 (ja) | 全方位画像編集プログラム及び全方位画像編集装置 | |
US9766793B2 (en) | Information processing device, information processing method and program | |
JP4982862B2 (ja) | プログラム、情報記憶媒体及び画像生成システム | |
KR102422929B1 (ko) | 디스플레이장치, 서버 및 그 제어방법 | |
US11651556B2 (en) | Virtual exhibition space providing method for efficient data management | |
US10249340B2 (en) | Video generation device, video generation program, and video generation method | |
WO2012032996A1 (ja) | 情報処理装置、情報処理方法、およびプログラム | |
EP3616402A1 (en) | Methods, systems, and media for generating and rendering immersive video content | |
KR102237519B1 (ko) | 2.5 차원화를 이용한 가상 전시 공간 제공 방법 | |
JPH0997344A (ja) | テクスチャ生成方法およびテクスチャ生成システム | |
JP2008287588A (ja) | 画像処理装置および方法 | |
WO2017002483A1 (ja) | プログラム、情報処理装置、深度定義方法及び記録媒体 | |
CN106169187A (zh) | 用于对视频中的物体设界的方法和设备 | |
GB2549723A (en) | A system and method for video editing in a virtual reality enviroment | |
CN108510433B (zh) | 空间展示方法、装置及终端 | |
JP5857606B2 (ja) | 奥行き製作支援装置、奥行き製作支援方法、およびプログラム | |
US20040155886A1 (en) | Image simulation processing method, processing apparatus, and program | |
JP2006293470A (ja) | 描画装置、方法および描画プログラム | |
JP5994320B2 (ja) | 奥行き制作支援装置、奥行き制作支援方法、及びプログラム | |
JP5523295B2 (ja) | 動画生成装置 | |
JP2002260003A (ja) | 映像表示装置 | |
JP5817300B2 (ja) | 奥行き製作支援装置、奥行き製作支援方法、およびプログラム | |
US20240112394A1 (en) | AI Methods for Transforming a Text Prompt into an Immersive Volumetric Photo or Video | |
WO2015087368A1 (ja) | 奥行き情報編集装置、奥行き情報編集方法、奥行き情報編集プログラム、及び奥行き情報編集プログラムを記録したコンピュータ読み取り可能な記録媒体 | |
JP2007025861A (ja) | バーチャルリアリティシステム及び方法、並びに、補間画像生成装置及び方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 13871855 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 20157018950 Country of ref document: KR Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 2014557173 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 14761537 Country of ref document: US |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 13871855 Country of ref document: EP Kind code of ref document: A1 |