WO2011061915A1 - Image display device and image display method - Google Patents

Image display device and image display method Download PDF

Info

Publication number
WO2011061915A1
WO2011061915A1 PCT/JP2010/006679 JP2010006679W WO2011061915A1 WO 2011061915 A1 WO2011061915 A1 WO 2011061915A1 JP 2010006679 W JP2010006679 W JP 2010006679W WO 2011061915 A1 WO2011061915 A1 WO 2011061915A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
image display
character area
character
scene change
Prior art date
Application number
PCT/JP2010/006679
Other languages
French (fr)
Japanese (ja)
Inventor
康治 井村
Original Assignee
パナソニック株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by パナソニック株式会社 filed Critical パナソニック株式会社
Publication of WO2011061915A1 publication Critical patent/WO2011061915A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/44—Receiver circuitry for the reception of television signals according to analogue transmission standards
    • H04N5/445—Receiver circuitry for the reception of television signals according to analogue transmission standards for displaying additional information
    • H04N5/44504—Circuit details of the additional information generator, e.g. details of the character or graphics signal generator, overlay mixing circuits
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440245—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440263—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/4508—Management of client data or end-user data
    • H04N21/4516—Management of client data or end-user data involving client characteristics, e.g. Set-Top-Box type, software version or amount of memory available
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
    • H04N21/41407—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop

Definitions

  • the present invention relates to an image display apparatus and an image display method related to display of a moving image in a mobile device, and more particularly to an image display apparatus and an image display method for improving the visibility of telops displayed in a television program or the like.
  • Mobile devices are made small to prioritize portability, and the display for display is also about 3 to 4 inches. Even if the content becomes high quality, if the display is small, the visibility of the small object being displayed is not very good, and in particular, the characters and telops etc. displayed in the television program are small and unreadable and difficult to read There is a problem such as
  • Patent Document 1 discloses a technique for detecting telop information.
  • edge detection processing is performed on an input image to specify a telop area.
  • Patent Document 2 the visibility is improved by changing and highlighting the color of the character portion of the telop area or enlarging the telop area.
  • the color and brightness of the background part of the character that is the background is time because the original image is used as the background so that only the part of the character is extracted and the character color is changed. Changes with the passage of time. As described above, there is a problem that it becomes difficult to read the character if there is movement in the background of the character that is stationary.
  • the present invention is an image display device and an image display method that solve the above-mentioned problems and improve the visibility of small characters even when viewing a moving image or a TV program with a mobile device.
  • the moving picture display device of the present invention performs an emphasizing process on a character area extraction unit that detects and cuts out an area including characters from a predetermined frame image in a moving image and the cut out character area
  • An emphasizing processing unit a memory for holding an emphasizing character area subjected to the emphasizing processing; and a synthesizing processing unit for synthesizing the emphasizing character area held in the memory with respect to frame images after the predetermined frame image Have.
  • the image display apparatus may further include a scene change detection unit that detects a scene switch, and the character area extraction unit may set a frame image in which the scene switch is detected as the predetermined frame image. It is.
  • the character region extraction unit sets, as the predetermined frame image, a frame image in which switching of a scene is detected each time a scene change is detected by the scene change detection unit. It is.
  • the enhancement processing in the above-described image display device is an image enlargement processing.
  • the portable terminal device of the present invention is equipped with the above-mentioned image display device.
  • the method further includes a scene change detection step of detecting a scene switch, and the character area extraction step includes, as the predetermined frame, a frame in which a scene change is detected in the scene change detection step.
  • Image display method In the image display method described above, the method further includes a scene change detection step of detecting a scene switch, and the character area extraction step includes, as the predetermined frame, a frame in which a scene change is detected in the scene change detection step.
  • the image display method repeats the scene change detection step, the character area extraction step, the enhancement processing step, the storage step, and the synthesis processing step.
  • the enhancement processing in the image display method according to any one of the above is image enlargement processing.
  • the moving image display apparatus and the image display method of the present invention when an area including telops and characters is detected, an emphasizing process for improving visibility is performed, and then stored in a memory and stored in a frame after detection. Since the highlighted character area is composited and displayed, the background portion of the character stands still, and the visibility of the character is improved.
  • the moving image display apparatus and method of the present invention since the scene change is detected and the character area is reextracted in the detected frame, the above-mentioned problems do not occur.
  • the mobile terminal device of the present invention is equipped with the moving image display device of the present invention, the visibility of characters is improved even with a device having a small display such as a mobile device.
  • FIG. 1 is a block diagram of a first embodiment of a video display device according to the present invention.
  • FIG. 5 is a block diagram of a second embodiment of a video display device according to the present invention. It is a block diagram of a portable terminal unit carrying a video display device according to the present invention. It is a 1st processing flow figure of the animation display method by the present invention. It is a 2nd processing flow of the moving image display method by this invention.
  • 100 is a character area extraction unit
  • 101 is an emphasis processing unit
  • 102 is a memory
  • 103 is a combining processing unit
  • 104 is a display output buffer.
  • the input image is input to the character area extraction unit 100.
  • the character area extraction unit 100 detects an area including characters from the input image, and outputs the detection result and the image data of the cut-out character area when it can be detected.
  • the detection result “1” means that a character area has been detected, and “0” means that a character area has not been detected.
  • the emphasizing processing unit 101 performs emphasizing processing to improve the visibility of characters in the input character area only when the detection result is “1” and the detection result in the previous frame is “0”. And stored in the memory 102 (this is called a highlighted character area). When the detection result is “0”, the emphasizing process is not performed, and the contents of the memory 102 are cleared. When the detection result is “1” and the detection result in the previous frame is also “1”, the emphasizing process is not performed and the memory 102 is not accessed. By performing such control, the highlighted character area is stored in the memory 102 only when the character area is detected for the first time.
  • the combining processing unit 103 combines the highlighted character area stored in the memory 102 and outputs the combined result to the display output buffer 104.
  • the enhanced character area is combined with the input image, so that the background portion of the character is also stationary, and the visibility of the character is improved.
  • Second Embodiment A second best mode for carrying out the moving picture display device of the present invention will be described with reference to FIG. In FIG. 2, the same processing blocks as in FIG. 1 are given the same numbers, and the description will be omitted.
  • reference numeral 201 denotes a scene change detection unit that detects scene switching.
  • Reference numeral 202 has almost the same function as the above-described emphasis processing unit 101, but differs in that emphasis processing is performed in consideration of the detection result of the scene change detection unit 201.
  • the scene change detection unit 201 detects switching of a scene.
  • Scene change detection compares the values of pixels at the same position in the input image and the previous input image, counts the number of pixels for which the absolute difference value is greater than or equal to a threshold, and the count value is different from the threshold If it is too large, it is determined that the scene has been switched, and the determination result at this time is represented as "1". When it is determined that switching has not been performed, the determination result is represented as “0”.
  • various methods can be considered for the method of scene change detection, the present invention is not limited to the above detection method.
  • the enhancement processing unit 202 clears the contents of the memory 102 without performing the enhancement process regardless of the detection result of the scene change detection unit 201.
  • enhancement processing is performed and stored in the memory 102 regardless of the scene change detection result.
  • reference numeral 301 denotes a recording medium
  • 302 denotes a separation unit
  • 303 denotes an image decoding unit
  • 304 denotes an audio decoding unit
  • 305 denotes a moving image display device
  • 306 denotes a speaker
  • 307 denotes a display.
  • the content such as a TV program stored in the recording medium 301 is read out.
  • the separation unit 302 separates the read content into encoded image data and encoded acoustic data, and outputs the separated data.
  • the image decoding unit 303 decodes the encoded image data into viewable image data.
  • There are various possible encoding methods for the image coded data but a technique (generally called MPEG-4) etc. defined in ISO / IEC 14496 Part 2 can be mentioned as an example.
  • the sound decoding unit 304 decodes the coded sound data into sound data that can be viewed.
  • a technique (generally called AAC) defined in ISO / IEC 14496 Part. 3 can be cited as an example.
  • the image decoded by the image decoding unit 303 is subjected to emphasizing processing of the character area in the moving image display device 305, and is synthesized and output to the input image.
  • the combined image is displayed on the display 307.
  • the sound data decoded by the sound decoding unit 304 is ringed by the speaker 306.
  • An area including characters such as telops is detected and cut out in the character area extraction step (ST01) for the input image.
  • the character area detection determination step (ST02) it is determined whether a character area has been detected in the input image.
  • the previous frame character area detection determination step (ST03) it is determined whether the character area has been detected in the input image in the previous frame. If the character area can be detected in the current input image and the character area can not be detected in the previous input image, emphasizing processing is performed on the extracted character area in the emphasizing processing step (ST04).
  • the emphasizing process is an enlargement process.
  • the emphasized character area is stored in the memory, and in the synthesizing step, the input image is synthesized and output.
  • the emphasis character area stored in the memory and the input image are synthesized and output in the synthesizing process step (ST06).
  • Table 1 shows the relationship between the character area detection result and the operation in the current input image and the previous input image.
  • the case where the character area can be detected is referred to as a detection result 1
  • the case where the character area can not be detected is referred to as a detection result 0.
  • This process is repeated frame by frame.
  • FIG. 5 in the scene change detection step (ST08), the current input image and the previous input image are compared in pixel units, and the number of pixels where the absolute value of the pixel value difference exceeds a certain threshold is counted. Do. If this count value is larger than another threshold value, it is determined that a scene change has been detected in the current input image. If a character area is detected in the current input image and a scene change is also detected, the character area is subjected to emphasis processing, stored in a memory, and then combined with the input image and output.
  • This process is repeated frame by frame.
  • the present invention is applicable to small-sized and highly portable devices, and in particular, mobile phone terminals, PDAs (Personal Digital Assistants), portable TVs, etc. can be considered.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Computer Graphics (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

In an image display device which is provided with a small display, the visibility of a small character such as a telop in a TV program is increased. The image display device has a character region extraction unit for detecting a region including a character from a predetermined frame image in a moving image and cutting out the region, an emphasis processing unit for performing emphasis processing with respect to the character region which has been cut out, a memory for storing the emphasized character region which has been subjected to the emphasis processing, and a synthesis processing unit for synthesizing the emphasized character region which has been stored in the memory with respect to frame images subsequent to the predetermined frame image.

Description

画像表示装置及び画像表示方法Image display apparatus and image display method
　本発明は、モバイル機器における動画像の表示に関する画像表示装置及び画像表示方法であり、特に、テレビ番組等で表示されるテロップの視認性を改善する画像表示装置及び画像表示方法に関するものである。 The present invention relates to an image display apparatus and an image display method related to display of a moving image in a mobile device, and more particularly to an image display apparatus and an image display method for improving the visibility of telops displayed in a television program or the like.
　携帯端末やPDAなどのモバイル機器において搭載されるプロセッサ等の処理性能の向上に伴い、従来のメールやインターネット上のホームページ等の閲覧といった用途から、動画の視聴へと広がってきた。2006年4月からは、地上デジタル放送の伝送帯域の一部を使い、携帯端末等での視聴を目的としたワンセグ放送が開始された。また、今後はアナログ放送が停止以降の帯域を利用した新たなサービスが検討されており、ワンセグよりも高画質なコンテンツの視聴の可能性が出てきた。一方、家庭内の機器と連係し、録画したテレビ番組を記録媒体等に適切な手段で書き込んだり、インターネットからダウンロードしたコンテンツを記録媒体に書き込んだりして、モバイル機器を使って出先で視聴するといった機能も実現されてきている。 With the improvement of processing performance of processors installed in mobile devices such as portable terminals and PDAs, applications such as browsing of conventional e-mails and homepages on the Internet have expanded to viewing of moving images. From April 2006, One-Seg broadcasting for viewing on portable terminals etc. was started using a part of the transmission band of terrestrial digital broadcasting. In addition, new services using bandwidth after the stop of analog broadcasting are being considered in the future, and the possibility of viewing and listening to content with higher image quality than 1 seg has come out. On the other hand, in cooperation with home devices, write recorded TV programs on recording media etc. by appropriate means, write contents downloaded from the Internet on recording media, and view on the go using mobile devices Functions have also been realized.
　このように放送、家電、インターネットなど様々な手段で高画質なコンテンツを入手・視聴できる環境が整いつつある。 As described above, there is an environment in which high-quality content can be obtained and viewed by various means such as broadcasting, home appliances, and the Internet.
日本国特開2007-074091号公報Japanese Patent Application Laid-Open No. 2007-074091 日本国特開2005-084157号公報Japanese Patent Application Laid-Open No. 2005-084157
　モバイル機器は携帯性を優先するために小型に作られており、表示のためのディスプレイも3～4インチ程度となっている。コンテンツが高画質になっても、ディスプレイが小さいと、表示されている小さい対象物の視認性はあまり良くなく、特に、テレビ番組中に表示される文字やテロップなどが小さくて読めない・読みづらいといった課題がある。 Mobile devices are made small to prioritize portability, and the display for display is also about 3 to 4 inches. Even if the content becomes high quality, if the display is small, the visibility of the small object being displayed is not very good, and in particular, the characters and telops etc. displayed in the television program are small and unreadable and difficult to read There is a problem such as
　このような課題に対応するため、画面内のテロップ情報を検出し、強調表示を行う技術の検討が行われている。テロップ情報の検出技術に関しては、特許文献１に開示されている。特許文献１では、入力された画像に対してエッジ検出処理を施し、テロップ領域を特定するものである。 In order to cope with such a subject, examination of the technique which detects telop information on a screen and performs highlighting is performed. Patent Document 1 discloses a technique for detecting telop information. In Patent Document 1, edge detection processing is performed on an input image to specify a telop area.
　また、特許文献２では、テロップ領域の文字部分の色を変えて強調表示したり、テロップ領域を拡大表示したりして視認性の向上をはかっている。文字の部分のみを抽出して文字色を変化させ、それ以外の部分は元の画像が下地となるようにしてあるため、動画の場合、下地である文字の背景部分の色や明るさが時間の経過とともに変化してしまう。このように、静止している文字の背景に動きがあると、その文字を読みにくくなってしまうという課題がある。 Further, in Patent Document 2, the visibility is improved by changing and highlighting the color of the character portion of the telop area or enlarging the telop area. In the case of a moving image, the color and brightness of the background part of the character that is the background is time because the original image is used as the background so that only the part of the character is extracted and the character color is changed. Changes with the passage of time. As described above, there is a problem that it becomes difficult to read the character if there is movement in the background of the character that is stationary.
　本発明は、上記の課題を解決し、モバイル機器で動画やTV番組等の視聴を行う場合でも、小さい文字の視認性を改善する画像表示装置及び画像表示方法である。 The present invention is an image display device and an image display method that solve the above-mentioned problems and improve the visibility of small characters even when viewing a moving image or a TV program with a mobile device.
　上記の課題を鑑みて、本発明の動画表示装置は、動画像内の所定フレーム画像から文字を含む領域を検出して切り出す文字領域抽出部と、前記切り出した文字領域に対して強調処理を施す強調処理部と、前記強調処理を施した強調文字領域を保持するメモリと、前記所定フレーム画像以降のフレーム画像に対して、前記メモリに保持された前記強調文字領域を合成する合成処理部とを有している。 In view of the above problems, the moving picture display device of the present invention performs an emphasizing process on a character area extraction unit that detects and cuts out an area including characters from a predetermined frame image in a moving image and the cut out character area An emphasizing processing unit; a memory for holding an emphasizing character area subjected to the emphasizing processing; and a synthesizing processing unit for synthesizing the emphasizing character area held in the memory with respect to frame images after the predetermined frame image Have.
　また、前記の画像表示装置は、シーンの切替わりを検出するシーンチェンジ検出部を有し、前記文字領域抽出部は、シーンの切替わりを検出したフレーム画像を前記所定フレーム画像とする画像表示装置である。 The image display apparatus may further include a scene change detection unit that detects a scene switch, and the character area extraction unit may set a frame image in which the scene switch is detected as the predetermined frame image. It is.
　また、前記記載の画像表示装置は、前記シーンチェンジ検出部でシーンチェンジを検出するたびに、前記文字領域抽出部は、シーンの切替わりを検出したフレーム画像を前記所定フレーム画像とする画像表示装置である。 Further, in the image display device described above, the character region extraction unit sets, as the predetermined frame image, a frame image in which switching of a scene is detected each time a scene change is detected by the scene change detection unit. It is.
　また、前述の画像表示装置における前記強調処理は、画像拡大処理である。 Further, the enhancement processing in the above-described image display device is an image enlargement processing.
　また、本発明の携帯端末装置は、前述の画像表示装置を搭載している。 Moreover, the portable terminal device of the present invention is equipped with the above-mentioned image display device.
　また、本発明の動画表示方法は、動画像内の所定フレーム画像から文字を含む文字領域を検出して切り出す文字領域抽出ステップと、前記文字領域に対して強調処理を施す強調処理ステップと、前記強調処理を施した強調文字領域を記憶する記憶ステップと、前記所定フレーム画像以降のフレームに対して前記強調文字領域を合成する合成処理ステップと、を有する。 In the moving image display method according to the present invention, a character area extraction step of detecting and cutting out a character area including characters from a predetermined frame image in a moving image, an emphasizing processing step of emphasizing the character area, and And storing the emphasized character area subjected to the emphasizing process, and synthesizing processing step of synthesizing the emphasized character area with respect to the frames after the predetermined frame image.
　また、前記記載の画像表示方法であって、シーンの切替わりを検出するシーンチェンジ検出ステップを有し、前記文字領域抽出ステップは、前記シーンチェンジ検出ステップでシーンチェンジを検出したフレームを前記所定フレームとする画像表示方法である。 In the image display method described above, the method further includes a scene change detection step of detecting a scene switch, and the character area extraction step includes, as the predetermined frame, a frame in which a scene change is detected in the scene change detection step. Image display method.
　また、前記画像表示方法であって、前記シーンチェンジ検出ステップと前記文字領域抽出ステップと前記強調処理ステップと前記記憶ステップと前記合成処理ステップと、を繰り返す画像表示方法である。 In the image display method, the image display method repeats the scene change detection step, the character area extraction step, the enhancement processing step, the storage step, and the synthesis processing step.
　また、前記いずれか１つに記載の画像表示方法における前記強調処理は、画像拡大処理である。 Further, the enhancement processing in the image display method according to any one of the above is image enlargement processing.
　本発明の動画表示装置および画像表示方法によれば、テロップや文字を含む領域を検出したら、視認性を改善するための強調処理を施してからメモリの保持しておき、検出以降のフレームに保持した強調文字領域を合成して表示するため、文字の背景部分が静止し、文字の視認性が向上する。 According to the moving image display apparatus and the image display method of the present invention, when an area including telops and characters is detected, an emphasizing process for improving visibility is performed, and then stored in a memory and stored in a frame after detection. Since the highlighted character area is composited and displayed, the background portion of the character stands still, and the visibility of the character is improved.
　検出した文字領域は同じものであっても、シーンチェンジの前後では、画面の明るさや配色等が大幅に変わり、シーンチェンジ前に検出した強調文字領域をそのまま合成するとシーンチェンジ後の画像の明るさや配色にそぐわない可能性がある。しかしながら、本発明の動画像表示装置および方法によれば、シーンチェンジを検出し、検出後のフレームにおいて文字領域を再度抽出するため、前述のような問題は発生しない。 Even if the detected character area is the same, the brightness and color arrangement etc. of the screen will change significantly before and after the scene change, and if the highlighted character area detected before the scene change is synthesized as it is, the brightness and brightness of the image after the scene change It may not match the color scheme. However, according to the moving image display apparatus and method of the present invention, since the scene change is detected and the character area is reextracted in the detected frame, the above-mentioned problems do not occur.
　本発明の携帯端末装置は、本発明の動画表示装置を搭載しているため、モバイル機器のようにディスプレイの小さい機器であっても、文字の視認性が良好になる。 Since the mobile terminal device of the present invention is equipped with the moving image display device of the present invention, the visibility of characters is improved even with a device having a small display such as a mobile device.
本発明による動画表示装置の第１の形態のブロック図である。FIG. 1 is a block diagram of a first embodiment of a video display device according to the present invention. 本発明による動画表示装置の第２の形態のブロック図である。FIG. 5 is a block diagram of a second embodiment of a video display device according to the present invention. 本発明による動画表示装置を搭載した携帯端末装置のブロック図である。It is a block diagram of a portable terminal unit carrying a video display device according to the present invention. 本発明による動画表示方法の第１の処理フロー図である。It is a 1st processing flow figure of the animation display method by the present invention. 本発明による動画表示方法の第２の処理フローである。It is a 2nd processing flow of the moving image display method by this invention.
　本発明を実施するための最良の形態を以下に述べる。 The best mode for carrying out the present invention will be described below.
　（第１実施形態）
　本発明の動画表示装置を実施するための第１の最良の形態を図１で説明する。図１において、１００は文字領域抽出部、１０１は強調処理部、１０２はメモリ、１０３は合成処理部、１０４は表示出力用バッファである。 First Embodiment
A first best mode for carrying out the moving picture display device of the present invention will be described with reference to FIG. In FIG. 1, 100 is a character area extraction unit, 101 is an emphasis processing unit, 102 is a memory, 103 is a combining processing unit, and 104 is a display output buffer.
　入力画像は、文字領域抽出部１００に入力される。文字領域抽出部１００は、入力された画像から文字を含む領域の検出を行い、検出結果と検出できた場合は切りだした文字領域の画像データを出力する。ここでは、検出結果は、「１」は文字領域を検出した事を意味し、「０」は文字領域を検出しなかった事を意味する。 The input image is input to the character area extraction unit 100. The character area extraction unit 100 detects an area including characters from the input image, and outputs the detection result and the image data of the cut-out character area when it can be detected. Here, as the detection result, “1” means that a character area has been detected, and “0” means that a character area has not been detected.
　強調処理部１０１は、前記検出結果が「１」であり、前フレームにおける検出結果が「０」の場合の時のみ、入力された文字領域に対して文字の視認性を向上させるための強調処理を施し、メモリ１０２に格納する（これを強調文字領域と呼ぶ）。検出結果が「０」の時は、強調処理は行わず、メモリ１０２の内容をクリアする。検出結果が「１」であり、前フレームにおける検出結果も「１」である場合、強調処理は行わず、メモリ１０２に対してもアクセスを行わない。このように制御する事で、初めて文字領域を検出した時のみ、強調文字領域がメモリ１０２に格納される。 The emphasizing processing unit 101 performs emphasizing processing to improve the visibility of characters in the input character area only when the detection result is “1” and the detection result in the previous frame is “0”. And stored in the memory 102 (this is called a highlighted character area). When the detection result is “0”, the emphasizing process is not performed, and the contents of the memory 102 are cleared. When the detection result is “1” and the detection result in the previous frame is also “1”, the emphasizing process is not performed and the memory 102 is not accessed. By performing such control, the highlighted character area is stored in the memory 102 only when the character area is detected for the first time.
　合成処理部１０３は、メモリ１０２に格納された強調文字領域を合成して表示出力用バッファ１０４に出力する。 The combining processing unit 103 combines the highlighted character area stored in the memory 102 and outputs the combined result to the display output buffer 104.
　このように、文字領域を検出した後は、強調文字領域を入力画像に合成するため、文字の背景部分も静止した画像となり、文字の視認性が向上する。 As described above, after the character area is detected, the enhanced character area is combined with the input image, so that the background portion of the character is also stationary, and the visibility of the character is improved.
（第２実施形態）
　本発明の動画表示装置を実施するための第２の最良の形態を図２で説明する。図２において、図１と同じ処理ブロックには同一の番号を付与するものとし、説明も割愛する。 Second Embodiment
A second best mode for carrying out the moving picture display device of the present invention will be described with reference to FIG. In FIG. 2, the same processing blocks as in FIG. 1 are given the same numbers, and the description will be omitted.
　図２において、２０１はシーンの切り替えを検出するシーンチェンジ検出部である。２０２は前述の強調処理部１０１とほぼ同等の機能を有するが、シーンチェンジ検出部２０１の検出結果も踏まえた上で強調処理を実施する点が異なっている。 In FIG. 2, reference numeral 201 denotes a scene change detection unit that detects scene switching. Reference numeral 202 has almost the same function as the above-described emphasis processing unit 101, but differs in that emphasis processing is performed in consideration of the detection result of the scene change detection unit 201.
　シーンチェンジ検出部２０１は、シーンの切り替えを検出する。シーンチェンジ検出は、入力画像と前の入力画像における同じ位置の画素の値を比較し、差分絶対値がある閾値以上となっている画素の数を計数し、計数値が前記とは異なる閾値よりも大きかった場合、シーンが切替わったと判定し、この時の判定結果は「１」として表す。切替わっていないと判定した場合、判定結果は「０」として表す。シーンチェンジ検出の方法は様々な手法が考えられるが、本発明は、前記の検出方法に限定されるものではない。 The scene change detection unit 201 detects switching of a scene. Scene change detection compares the values of pixels at the same position in the input image and the previous input image, counts the number of pixels for which the absolute difference value is greater than or equal to a threshold, and the count value is different from the threshold If it is too large, it is determined that the scene has been switched, and the determination result at this time is represented as "1". When it is determined that switching has not been performed, the determination result is represented as “0”. Although various methods can be considered for the method of scene change detection, the present invention is not limited to the above detection method.
　強調処理部２０２は、文字領域抽出部１００の検出結果は「０」の時は、シーンチェンジ検出部２０１の検出結果に関わらず、強調処理は行わず、メモリ１０２の内容をクリアする。 When the detection result of the character area extraction unit 100 is “0”, the enhancement processing unit 202 clears the contents of the memory 102 without performing the enhancement process regardless of the detection result of the scene change detection unit 201.
　現在のフレームにおける検出結果が「１」であり、前フレーム検出結果が「０」の場合、シーンチェンジの検出結果に関わらず、強調処理を実施し、メモリ１０２に格納する。 If the detection result in the current frame is “1” and the previous frame detection result is “0”, enhancement processing is performed and stored in the memory 102 regardless of the scene change detection result.
　現在のフレームのおける検出結果が「１」であり、前フレーム検出結果が「１」であり、かつ、シーンチェンジ検出結果が「０」の場合も、強調処理を実施し、メモリ１０２に格納する。シーンチェンジが検出されなかった場合は、強調処理は実施せず、メモリ１０２にもアクセスしない。このように、文字領域を検出した後は、強調処理を施した文字領域を入力画像に合成するため、文字の背景部分も静止した画像となり、文字の視認性が向上するとともに、シーンの切替わりの前後で、文字領域の明るさや配色に対する違和感をなくすことができる。 Even when the detection result in the current frame is “1”, the previous frame detection result is “1”, and the scene change detection result is “0”, enhancement processing is performed and stored in the memory 102. . If a scene change is not detected, the emphasizing process is not performed and the memory 102 is not accessed. As described above, after the text area is detected, the text area subjected to the emphasizing process is combined with the input image, so that the background part of the text is also a static image, and the visibility of the text is improved. Before and after, the sense of incongruity with the brightness and color arrangement of the character area can be eliminated.
（第３実施形態）
　本発明の携帯端末装置の最良な実施の形態について、図３を使って説明を行う。図３において３０１は記録媒体、３０２は分離部、３０３は画像復号部、３０４は音響復号部、３０５は動画表示装置、３０６はスピーカ、３０７はディスプレイである。 Third Embodiment
The preferred embodiment of the mobile terminal device of the present invention will be described with reference to FIG. In FIG. 3, reference numeral 301 denotes a recording medium, 302 denotes a separation unit, 303 denotes an image decoding unit, 304 denotes an audio decoding unit, 305 denotes a moving image display device, 306 denotes a speaker, and 307 denotes a display.
　携帯端末装置において、記録媒体３０１に格納されているTV番組等のコンテンツを読み出す。分離部３０２は、読み出したコンテンツから符号化画像データと符号化音響データに分離して出力する。画像復号部３０３は、符号化画像データを視聴可能な画像データに復号する。画像符号化データの符号化方式は、さまざまなものが考えられるが、ISO/IEC(アイエスオー・アイイーシー) 14496 Part.2で規定されている技術(通称MPEG-4)等が例として挙げられる。音響復号部３０４は、符号化音響データを視聴可能な音響データに復号する。符号化音響データも同様にISO/IEC 14496 Part.3で規定されている技術(通称 AAC)が例としてあげられる。 In the portable terminal device, the content such as a TV program stored in the recording medium 301 is read out. The separation unit 302 separates the read content into encoded image data and encoded acoustic data, and outputs the separated data. The image decoding unit 303 decodes the encoded image data into viewable image data. There are various possible encoding methods for the image coded data, but a technique (generally called MPEG-4) etc. defined in ISO / IEC 14496 Part 2 can be mentioned as an example. . The sound decoding unit 304 decodes the coded sound data into sound data that can be viewed. Similarly, for encoded audio data, a technique (generally called AAC) defined in ISO / IEC 14496 Part. 3 can be cited as an example.
　画像復号部３０３で復号された画像は、動画表示装置３０５において文字領域の強調処理を施し、入力画像に合成して出力を行う。合成した画像はディスプレイ３０７に表示される。音響復号部３０４で復号された音響データは、スピーカ３０６にて鳴動する。 The image decoded by the image decoding unit 303 is subjected to emphasizing processing of the character area in the moving image display device 305, and is synthesized and output to the input image. The combined image is displayed on the display 307. The sound data decoded by the sound decoding unit 304 is ringed by the speaker 306.
　このような構成にすることで、小型の携帯端末装置において、文字領域を強調した画像の視聴が可能となり、文字などの視認性が向上する。 With such a configuration, in a small-sized mobile terminal device, it becomes possible to view an image in which a character area is emphasized, and the visibility of characters and the like is improved.
（第４実施形態）
　本発明の動画表示方法の最良な実施の形態について、図４を使って説明を行う。入力画像に対して、文字領域抽出ステップ（ＳＴ０１）でテロップなどの文字を含む領域を検出して切りだしを行う。文字領域検出判定ステップ（ＳＴ０２）において、入力画像において文字領域を検出できたかどうかの判定を行う。また、前フレーム文字領域検出判定ステップ（ＳＴ０３）において、前フレームにおける入力画像において文字領域を検出できたかどうかの判定を行う。現在の入力画像において文字領域が検出できた場合であり、かつ、前の入力画像において文字領域が検出できていない場合、切り出した文字領域に対して強調処理ステップ（ＳＴ０４）で強調処理を施す。ここでは、強調処理は拡大処理であるものとする。メモリ格納ステップ（ＳＴ０５）において強調文字領域をメモリに格納し、さらに合成処理ステップにおいて入力画像に合成して出力を行う。 Fourth Embodiment
A preferred embodiment of the moving picture display method of the present invention will be described with reference to FIG. An area including characters such as telops is detected and cut out in the character area extraction step (ST01) for the input image. In the character area detection determination step (ST02), it is determined whether a character area has been detected in the input image. In the previous frame character area detection determination step (ST03), it is determined whether the character area has been detected in the input image in the previous frame. If the character area can be detected in the current input image and the character area can not be detected in the previous input image, emphasizing processing is performed on the extracted character area in the emphasizing processing step (ST04). Here, it is assumed that the emphasizing process is an enlargement process. In the memory storing step (ST05), the emphasized character area is stored in the memory, and in the synthesizing step, the input image is synthesized and output.
　一方、前の入力画像において、文字領域が検出できなかった場合、合成処理ステップ（ＳＴ０６）においてメモリに格納されている強調文字領域と入力画像を合成して出力を行う。 On the other hand, when the character area is not detected in the previous input image, the emphasis character area stored in the memory and the input image are synthesized and output in the synthesizing process step (ST06).
　文字領域を検出できなかった場合は、メモリクリアステップ（ＳＴ０７）にてメモリ内容をクリアして合成処理も行わず入力画像をそのまま出力する。現入力画像と前の入力画像における文字領域検出結果と動作の関係を表１に示す。表１において、文字領域が検出できた場合を検出結果１とし、検出できなかった場合を検出結果０とする。 If the character area can not be detected, the memory contents are cleared in the memory clear step (ST07) and the input image is output as it is without performing the synthesizing process. Table 1 shows the relationship between the character area detection result and the operation in the current input image and the previous input image. In Table 1, the case where the character area can be detected is referred to as a detection result 1, and the case where the character area can not be detected is referred to as a detection result 0.
　この処理をフレーム単位で繰り返す。 This process is repeated frame by frame.
Figure JPOXMLDOC01-appb-T000001
Figure JPOXMLDOC01-appb-T000001
（第５実施形態）
　本発明の動画表示方法の最良な実施の形態について、図５を使って説明を行う。図４と同じステップについては、説明を割愛する。図５において、シーンチェンジ検出ステップ（ＳＴ０８）では、現在の入力画像と前の入力画像とを画素単位で比較し、画素の値の差分の絶対値がある閾値を超えている画素の数を計数する。この計数値がもうひとつの閾値よりも大きい場合、現在の入力画像においてシーンチェンジを検出したものと判定する。現在の入力画像において文字領域を検出し、シーンチェンジも検出した場合は、文字領域に強調処理を施し、メモリに格納したのちに入力画像に合成して出力する。シーンチェンジを検出しなかったが前入力画像で文字領域を検出しなかった場合も同様の動作をする。シーンチェンジを検出したが、前入力画像でも文字領域を検出していた場合は、メモリに格納されている強調文字領域を現入力画像に合成して出力を行う。現入力画像と前入力画像における文字領域の検出結果、シーンチェンジ検出結果と動作についての関係を表２に示す。 Fifth Embodiment
A preferred embodiment of the moving picture display method of the present invention will be described with reference to FIG. The same steps as FIG. 4 will not be described. In FIG. 5, in the scene change detection step (ST08), the current input image and the previous input image are compared in pixel units, and the number of pixels where the absolute value of the pixel value difference exceeds a certain threshold is counted. Do. If this count value is larger than another threshold value, it is determined that a scene change has been detected in the current input image. If a character area is detected in the current input image and a scene change is also detected, the character area is subjected to emphasis processing, stored in a memory, and then combined with the input image and output. The same operation is performed when the scene change is not detected but the character area is not detected in the previous input image. When a scene change is detected, but a character area is detected even in the previous input image, an enhanced character area stored in the memory is combined with the current input image and output. Table 2 shows the relationship between the detection result of the character area in the current input image and the previous input image, the scene change detection result, and the operation.
　この処理をフレーム単位で繰り返す。 This process is repeated frame by frame.
Figure JPOXMLDOC01-appb-T000002
Figure JPOXMLDOC01-appb-T000002
　本発明を詳細にまた特定の実施態様を参照して説明したが、本発明の精神と範囲を逸脱することなく様々な変更や修正を加えることができることは当業者にとって明らかである。 Although the invention has been described in detail and with reference to specific embodiments, it will be apparent to those skilled in the art that various changes and modifications can be made without departing from the spirit and scope of the invention.
　本出願は、２００９年１１月１７日出願の日本特許出願（特願２００９－２６１５３８）、に基づくものであり、その内容はここに参照として取り込まれる。 This application is based on Japanese Patent Application (No. 2009-261538) filed on Nov. 17, 2009, the contents of which are incorporated herein by reference.
本発明は、小型で携帯性の高い機器に適用が可能であり、特に携帯電話端末、PDA(パーソナルデジタルアシスタント)、ポータブルテレビなどが考えられる。 The present invention is applicable to small-sized and highly portable devices, and in particular, mobile phone terminals, PDAs (Personal Digital Assistants), portable TVs, etc. can be considered.
　１００　　文字領域抽出部
　１０１　　強調処理部
　１０２　　メモリ
　１０３　　合成処理部 
　１０４　　表示出力用バッファ
　２０１　　シーンチェンジ検出部
　２０２　　強調処理部
　３０１　　記録媒体
　３０２　　分離部
　３０３　　画像復号部
　３０４　　音響復号部
　３０５　　動画表示装置
　３０６　　スピーカ
　３０７　　ディスプレイ 100 character area extraction unit 101 emphasis processing unit 102 memory 103 synthesis processing unit
104 display output buffer 201 scene change detection unit 202 enhancement processing unit 301 recording medium 302 separation unit 303 image decoding unit 304 acoustic decoding unit 305 moving image display device 306 speaker 307 display

Claims (9)

  1. 動画像内の所定フレーム画像から文字を含む領域を検出して切り出す文字領域抽出部と、前記切り出した文字領域に対して強調処理を施す強調処理部と、
    前記強調処理を施した強調文字領域を保持するメモリと、
    前記所定フレーム画像以降のフレーム画像に対して、前記メモリに保持された前記強調文字領域を合成する合成処理部と、
    を有する画像表示装置。     A character region extraction unit that detects and cuts out a region including characters from a predetermined frame image in a moving image;
    A memory for holding an emphasized character area subjected to the emphasizing process;
    A combining processing unit that combines the highlighted character area held in the memory with frame images after the predetermined frame image;
    An image display device having
  2. 請求項１記載の画像表示装置であって、
    シーンの切替わりを検出するシーンチェンジ検出部を有し、
    前記文字領域抽出部は、シーンの切替わりを検出したフレーム画像を前記所定フレーム画像とする画像表示装置。     The image display apparatus according to claim 1,
    Has a scene change detection unit that detects scene switching,
    The character region extraction unit is an image display device in which a frame image in which switching of a scene is detected is set as the predetermined frame image.
  3. 請求項２記載の画像表示装置であって、
    前記シーンチェンジ検出部でシーンチェンジを検出するたびに、
    前記文字領域抽出部は、シーンの切替わりを検出したフレーム画像を前記所定フレーム画像とする画像表示装置。     The image display apparatus according to claim 2,
    Every time a scene change is detected by the scene change detection unit,
    The character region extraction unit is an image display device in which a frame image in which switching of a scene is detected is set as the predetermined frame image.
  4. 請求項１から請求項３のいずれか１項に記載の画像表示装置であって、
    前記強調処理は、画像拡大処理である画像表示装置。     The image display device according to any one of claims 1 to 3, wherein
    The image display apparatus, wherein the enhancement processing is image enlargement processing.
  5. 請求項１から請求項４のいずれか１項に記載の画像表示装置を搭載する携帯端末装置。 A portable terminal device equipped with the image display device according to any one of claims 1 to 4.
  6. 動画像内の所定フレーム画像から文字を含む文字領域を検出して切り出す文字領域抽出ステップと、
    前記文字領域に対して強調処理を施す強調処理ステップと、
    前記強調処理を施した強調文字領域を記憶する記憶ステップと、
    前記所定フレーム画像以降のフレームに対して前記強調文字領域を合成する合成処理ステップと、
    を有する画像表示方法。     A character area extraction step of detecting and cutting out a character area including characters from a predetermined frame image in a moving image;
    An emphasizing process step of emphasizing the character area;
    A storage step of storing an emphasized character area subjected to the emphasizing process;
    Combining processing step of combining the highlighted character area with the frames after the predetermined frame image;
    A method of displaying an image.
  7. 請求項６記載の画像表示方法であって、
    シーンの切替わりを検出するシーンチェンジ検出ステップを有し、
    前記文字領域抽出ステップは、前記シーンチェンジ検出ステップでシーンチェンジを検出したフレームを前記所定フレームとする画像表示方法。     The image display method according to claim 6, wherein
    It has a scene change detection step that detects scene switching,
    The character area extraction step is an image display method in which a frame in which a scene change is detected in the scene change detection step is set as the predetermined frame.
  8. 請求項７記載の画像表示方法であって、
    前記シーンチェンジ検出ステップと前記文字領域抽出ステップと前記強調処理ステップと前記記憶ステップと前記合成処理ステップと、を繰り返す画像表示方法。     8. The image display method according to claim 7, wherein
    An image display method repeating the scene change detection step, the character area extraction step, the enhancement processing step, the storage step, and the combination processing step.
  9. 前記請求項６から請求項８のいずれか１項に記載の画像表示方法であって、
    前記強調処理は画像拡大処理である画像表示方法。     The image display method according to any one of claims 6 to 8.
    The image display method, wherein the enhancement processing is image enlargement processing.
PCT/JP2010/006679 2009-11-17 2010-11-12 Image display device and image display method WO2011061915A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2009261538A JP2013030821A (en) 2009-11-17 2009-11-17 Image display apparatus and image display method
JP2009-261538 2009-11-17

Publications (1)

Publication Number Publication Date
WO2011061915A1 true WO2011061915A1 (en) 2011-05-26

Family

ID=44059402

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2010/006679 WO2011061915A1 (en) 2009-11-17 2010-11-12 Image display device and image display method

Country Status (2)

Country Link
JP (1) JP2013030821A (en)
WO (1) WO2011061915A1 (en)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009065432A (en) * 2007-09-06 2009-03-26 Mitsubishi Electric Corp Image reproducing device

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009065432A (en) * 2007-09-06 2009-03-26 Mitsubishi Electric Corp Image reproducing device

Also Published As

Publication number Publication date
JP2013030821A (en) 2013-02-07

Similar Documents

Publication Publication Date Title
KR101527672B1 (en) System and method for video caption re-overlaying for video adaptation and retargeting
KR102226563B1 (en) Content adaptive telecine and interlace reverser
US8218622B2 (en) System and method for processing videos and images to a determined quality level
JP4996725B2 (en) Video processing device
JP2010041337A (en) Image processing unit and image processing method
WO2011070793A1 (en) Image display device and image display method
JP4575803B2 (en) Compression encoding apparatus and compression encoding program
WO2010073192A1 (en) Image scaling curve generation
JP2023521553A (en) Patch-based video coding for machines
JP2014077993A (en) Display device
JP2003338991A (en) Image display device and its method
EP2330818A1 (en) Frame rate converting apparatus and corresponding point estimating apparatus, corresponding point estimating method and corresponding point estimating program
EP2999215A1 (en) Image processing device and image processing method
JP6045405B2 (en) Video processing apparatus, display apparatus, television receiver, and video processing method
JP2007259193A (en) Video output apparatus, and method
WO2011061915A1 (en) Image display device and image display method
EP1848203B2 (en) Method and system for video image aspect ratio conversion
US7215375B2 (en) Method for line average differences based de-interlacing
JP2011055485A (en) Video processing apparatus
JP4991884B2 (en) Image processing apparatus and image processing method
WO2014001095A1 (en) Method for audiovisual content dubbing
JPWO2005101819A1 (en) Display device
JP2012004890A (en) Video signal output device, and video signal output method
JP5557311B2 (en) Video display device, display control method, and program
JP4530671B2 (en) Image reproducing apparatus and image reproducing method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10831314

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 10831314

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP