WO2015045679A1 - Information device and control program - Google Patents

Information device and control program Download PDF

Info

Publication number
WO2015045679A1
WO2015045679A1 PCT/JP2014/071715 JP2014071715W WO2015045679A1 WO 2015045679 A1 WO2015045679 A1 WO 2015045679A1 JP 2014071715 W JP2014071715 W JP 2014071715W WO 2015045679 A1 WO2015045679 A1 WO 2015045679A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
rotation
unit
target image
user
Prior art date
Application number
PCT/JP2014/071715
Other languages
French (fr)
Japanese (ja)
Inventor
健文 大塚
Original Assignee
シャープ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by シャープ株式会社 filed Critical シャープ株式会社
Publication of WO2015045679A1 publication Critical patent/WO2015045679A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/40—Processing or translation of natural language
    • G06F40/58—Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G06V10/243—Aligning, centring, orientation detection or correction of the image by compensating for image skew or non-uniform image deformations

Definitions

  • the present invention relates to information equipment.
  • OCR Optical Character Recognition
  • Word Lens says that the result of translating characters obtained by OCR processing is displayed in the preview image of the camera application by performing OCR processing on the image generated by the camera of the smartphone shooting the subject. It has a function.
  • Patent Document 1 discloses a method for improving the accuracy of OCR processing by automatically correcting the rotation angle (tilt angle) of an image.
  • the above-mentioned Word Lens does not have a function for correcting the rotation angle of the image taken by the camera. Therefore, it is difficult for the user to shoot an upright image when there is no horizontally stable scaffold during shooting, or when the position where the subject can be shot is limited. There was a problem that OCR processing and translation processing by Word Lens were not performed properly.
  • the present invention has been made to solve the above problems, and an object thereof is to provide an information device capable of improving the accuracy of OCR processing and translation processing.
  • an information device that acquires a first target image including a character to be translated and translates a character included in the first target image.
  • An image rotating means for generating a second target image obtained by rotating the first target image in accordance with a user input operation, and the second target image as a target for character recognition processing.
  • a character recognition unit for recognizing the included character, a translation result for the character that has been successfully recognized by the character recognition unit, as a translation target, and a display result in which the translation result is superimposed on the translation target in the second target image
  • Translation processing means for generating an image.
  • the information device it is possible to provide an information device that can improve the accuracy of the OCR process and the translation process.
  • FIG. 1 It is a block diagram which shows the structure of the smart phone which concerns on Embodiment 1 of this invention. It is a figure which shows the flow of the translation process of the smart phone which concerns on Embodiment 1 of this invention.
  • (A)-(d) is a figure which illustrates the case where the OCR process and translation are performed with respect to the character contained in the image in the smart phone which concerns on Embodiment 1 of this invention.
  • FIG. 2 shows another example of the flow of the translation process of the smart phone which concerns on Embodiment 2 of this invention.
  • (A)-(b) is a figure which illustrates the case where the OCR process and translation are performed with respect to the character contained in an image in the smart phone which concerns on Embodiment 2 of this invention.
  • Embodiment 1 Embodiment 1 of the present invention will be described with reference to FIGS. 1 to 3 as follows.
  • FIG. 1 is a functional block diagram schematically showing the configuration of the smartphone 1 (information device) of the present embodiment.
  • the smartphone 1 includes a touch panel 10, a main control unit 20, a photographing unit 30, and a storage unit 40.
  • the touch panel 10 is, for example, a capacitive touch panel, and includes a display unit 11 and an input unit 12 (input surface).
  • the display unit 11 is, for example, a liquid crystal display.
  • the input unit 12 is a user interface that accepts an input operation by the user, and is arranged at a position overlapping the display screen of the display unit 11. Therefore, the input unit 12 can detect an input operation on the display screen of the display unit 11.
  • the input unit 12 mainly receives an input signal (for example, a signal indicating a change in capacitance and a change in capacitance) indicating that a touch input by a finger (or a touch pen or the like) has been performed. This is given to the input position detection unit 22 provided in the control unit 20.
  • an input signal for example, a signal indicating a change in capacitance and a change in capacitance
  • the photographing unit 30 is, for example, a main camera provided on the surface opposite to the surface on which the display unit 11 is disposed (that is, the back surface of the smartphone 1).
  • the photographing unit 30 gives an image generated by photographing a subject to the image acquisition unit 21 included in the main control unit 20.
  • the storage unit 40 is a storage device that stores various data used in the smartphone 1.
  • the main control unit 20 comprehensively controls operations of various hardware (for example, the touch panel 10 and the photographing unit 30) included in the smartphone 1 and functions of the smartphone 1.
  • the main control unit 20 includes an image acquisition unit 21, an input position detection unit 22, an angle correction unit 23 (image rotation unit and rotation button display unit), an OCR processing unit 24 (character recognition unit), and a translation process described below. It functions as a section 25 (translation processing means).
  • the image acquisition unit 21 displays the image from the photographing unit 30 on the display unit 11. Note that the image acquisition unit 21 can also cause the display unit 11 to display an image currently captured by the imaging unit 30 as a preview image of the camera application (see FIG. 3A described later).
  • the input position detection unit 22 detects an input position that is a position where an input from the user to the input unit 12 is given based on an input signal from the input unit 12.
  • Various buttons are displayed on the display unit 11 as an input operation target by the user, and the user responds to the button via the input position detection unit 22 by tapping each button.
  • the attached operation can be performed by the smartphone 1. Further, the user can cause the smartphone 1 to perform various predetermined operations via the input position detection unit 22 not only by tapping a button but also by a drag operation or the like on the input unit 12.
  • the angle correction unit 23 (image rotation means) has a function of rotating (rotating) the image displayed on the display unit 11 and adjusting (correcting) the rotation angle of the image.
  • the angle correction unit 23 (rotation button display means) causes the display unit 11 to display a “left rotation” or “right rotation” button (rotation button). When the user taps the rotation button displayed on the display unit 11, processing in the angle correction unit 23 is performed.
  • the angle correction unit 23 is illustrated as having a function as a rotation button display unit in addition to a function as an image rotation unit.
  • the rotation button display means may be provided separately.
  • the angle correction unit 23 may be provided with only a function as an image rotation unit, and a functional block having a function as a rotation button display unit may be separately provided in the main control unit 20.
  • the angle correction unit 23 rotates the image counterclockwise by 1 °. Further, each time the user taps the “rotate right” button once, the angle correction unit 23 rotates the image by 1 ° in the clockwise direction (see FIG. 3C described later).
  • the rotation angle of the image every time the rotation button is tapped is not limited to 1 °, and may be another appropriate value (for example, 5 °) according to the specification of the smartphone 1.
  • the rotation center position (rotation center position) of the image is, for example, the center position of the display unit 11. It should be noted that the center position of the image rotation may be a predetermined position other than the center position of the display unit 11, and the coordinates indicating the center position of the image rotation may be given to the angle correction unit 23 in advance. .
  • the angle correction unit 23 causes the display unit 11 to display the rotated image.
  • the angle correction unit 23 gives the rotated image to the OCR processing unit 24.
  • the angle correction unit 23 does not rotate the image, the image acquired by the image acquisition unit 21 from the photographing unit 30 is directly supplied from the angle correction unit 23 to the OCR processing unit 24.
  • the OCR processing unit 24 performs OCR processing (character recognition processing) on the image displayed on the display unit 11. That is, the OCR processing unit 24 converts characters included in the image into character codes that can be processed by a computer. Then, the OCR processing unit 24 gives the character code obtained as a result of the OCR process and the coordinates of the character in the image corresponding to the character code to the translation processing unit 25.
  • OCR processing character recognition processing
  • the image to be subjected to the OCR processing may be an image after the rotation in the angle correction unit 23 (second target image) or in the angle correction unit 23. It may be an image before the rotation (first target image).
  • the translation processing unit 25 has a function of translating a character code of a source language (for example, English) into a predetermined target language (for example, Japanese).
  • the translation processing unit 25 performs the above-described translation by referring to dictionary data (bilingual dictionary data in which the meaning of each word in the source language is indicated by the target language) recorded in the storage unit 40 in advance. .
  • the translation processing unit 25 translates the character code given from the OCR processing unit 24 and generates a translation result. Then, the translation processing unit 25 refers to the character coordinates in the image corresponding to the translated character code, generates a display image in which the translation result is superimposed on the character to be translated in the image, and A display image is displayed on the display unit 11.
  • the target image on which the translation result is superimposed may be either the first target image or the second target image described above (see (b) and (d) of FIG. 3 described later).
  • FIG. 2 is a flowchart illustrating the flow of translation processing in the smartphone 1.
  • the user activates the photographing unit 30 by performing an input operation on the smartphone 1, and starts photographing the subject by the photographing unit 30 (S1).
  • the image acquisition unit 21 acquires an image generated by the shooting unit 30 shooting a subject (S2), and causes the display unit 11 to display the image.
  • the OCR processing unit 24 performs an OCR process on the image displayed on the display unit 11 (S3), and converts characters included in the image into a character code that can be processed by a computer.
  • the translation processing unit 25 translates the character code and generates a translation result (S4).
  • the translation processing unit 25 causes the display unit 11 to display the translation result on the character to be translated in the image. That is, the translation processing unit 25 displays the generated display image on the display unit 11 (S5).
  • the angle correction unit 23 checks whether or not the user has tapped the rotation button (“left rotation” or “right rotation” button) displayed on the display unit 11 (process S6).
  • the angle correction unit 23 moves in the direction of the rotation button (the “left rotation” or “right rotation” button) tapped by the user. Accordingly, each time the user taps the rotation button once, the image is rotated by a predetermined angle (for example, 1 °) in a predetermined direction (counterclockwise or clockwise) (S7). Then, the process returns to the process S3, and the processes S3 to S6 are repeated.
  • FIGS. 3A to 3D show that the smartphone 1 performs OCR processing and translation on the English character “Chapter 1 This is a pen. How are you?” Included in the image of the subject. It is a figure which illustrates the case where it was performed.
  • FIG. 3A is a diagram illustrating a point in time when the image of the subject currently captured by the image capturing unit 30 is displayed on the display unit 11 as a preview image of the camera application.
  • FIG. 3B is a diagram illustrating a case where OCR processing and translation are performed on an image generated when the photographing unit 30 photographs a subject.
  • FIG. 6B since the image to be subjected to the OCR process is tilted, the accuracy of the OCR process in the OCR processing unit 24 is reduced, and all the characters included in the image are correctly recognized. Not.
  • the OCR processing unit 24 succeeds only in the OCR processing of “is” which is a part of English characters.
  • the translation processing unit 25 performs the translation process only on the English character “is” that has been successfully subjected to the OCR processing in the OCR processing unit 24, and generates the Japanese character “is” as the translation result.
  • the Japanese character “Is” as a result of translation by the translation processing unit 25 is superimposed on the English character “is” and displayed on the display unit 11.
  • FIG. 3C is a diagram illustrating a point in time when the user adjusts the rotation angle of the image by tapping the “right rotation” button displayed on the display unit 11. Each time the user taps the “rotate right” button once, the image rotates 1 ° clockwise. When the user performs an operation of rotating the image an appropriate number of times, the user can manually adjust the rotation angle of the image so that the image is generally upright.
  • FIG. 3D is a figure which illustrates the case where an OCR process and translation are performed to the image in which the rotation angle was adjusted.
  • the accuracy of the OCR processing does not decrease, and all characters included in the image are correctly recognized. Yes.
  • the Japanese character “Chapter 1 This is a pen?” Is superimposed on the above-mentioned English characters as a translation result and displayed on the display unit 11. It is displayed.
  • the rotation angle of the image can be adjusted by an angle (for example, 1 °). Therefore, according to the smartphone 1, the convenience of the operation by the user for adjusting the rotation angle of the image is improved.
  • FIG. 4 is a functional block diagram schematically showing the configuration of the smartphone 2 (information device) of the present embodiment.
  • the smartphone 2 is obtained by replacing the main control unit 20 included in the smartphone 1 of the first embodiment with the main control unit 26.
  • the main control unit 26 of the present embodiment is the same as the main control unit 20 of the first embodiment except that (i) a rotation center setting unit 27 (rotation center calculation means) is further added, and (ii) the angle correction unit 23 is It is replaced by the angle correction unit 28 (image rotation means).
  • the rotation center setting unit 27 has a function of setting a rotation center position prior to image rotation processing in an angle correction unit 28 described later.
  • the rotation center setting unit 27 acquires two input positions from the input position detection unit 22 and detects that the user is placing two fingers on the input unit 12.
  • the rotation center setting unit 27 sets the midpoints of the two input positions corresponding to the positions of the two fingers at the time when the user places the two fingers on the input unit 12 as the rotation center position ( (Rotation center position). Also.
  • the rotation center setting unit 27 sets a predetermined position such as the center position of the display unit 11 (that is, the image rotation center position similar to the angle correction unit 23 of the first embodiment) as the image rotation center position. May be.
  • the rotation center setting unit 27 gives coordinates indicating the rotation center position of the image to the angle correction unit 28. Prior to the processing of the rotation center setting unit 27, the rotation center setting unit 27 determines which of the midpoints of the two input positions or the center position of the display unit 11 is set as the rotation center position of the image. The user can arbitrarily set.
  • the angle correction unit 28 of the present embodiment has a function of rotating the image displayed on the display unit 11 and adjusting the rotation angle of the image, like the angle correction unit 23 of the first embodiment.
  • the angle correction unit 28 of the present embodiment is different from the angle correction unit 23 of the first embodiment in the interface for accepting the user's input operation.
  • the angle correction unit 28 performs processing by rotating the two fingers on the input unit 12.
  • the angle correction unit 28 refers to the temporal change in the two input positions acquired from the input position detection unit 22 and detects that the two input positions are rotating, so that the user holds two fingers. Recognize that an operation of rotating counterclockwise or clockwise is performed.
  • the angle correction unit 28 rotates the image counterclockwise.
  • the angle correction unit 28 rotates the image in the clockwise direction (see FIG. 7A described later).
  • the rotation center position of the image is a position set in the rotation center setting unit 27.
  • the rotation angle of the image in the angle correction unit 28 is determined as an amount proportional to the angle at which the user has rotated the two fingers (that is, the rotation angle of the two input positions). For example, when the user rotates two fingers by 60 °, the angle correction unit 28 rotates the image by 30 °.
  • the proportionality constant between the rotation angle of the two fingers of the user and the rotation angle of the image in the angle correction unit 28 may be set as appropriate according to the specifications of the smartphone 2.
  • the angle correction unit 28 displays the rotated image on the display unit 11 and gives it to the OCR processing unit 24 as with the angle correction unit 23 of the first embodiment.
  • a rotation button of “left rotation” or “right rotation” may be displayed on the display unit 11, and the angle correction unit 28 of the present embodiment may be further provided with the same function as the angle correction unit 23 of the first embodiment. Good.
  • the smartphone 2 when configured to set a predetermined position such as the center position of the display unit 11 as the center position of image rotation, the rotation center setting unit 27 is excluded and the angle correction unit 28 receives the image.
  • the coordinates of the center position of the rotation may be given in advance.
  • FIG. 5 is a flowchart illustrating the flow of translation processing in the smartphone 2.
  • Processes S11 to S18 in FIG. 5 exemplify the process flow when the center position of the display unit 11 is set as the center position of image rotation. Note that the processes S11 to S15 in FIG. 5 are the same as the processes S1 to S5 in FIG. Therefore, only the processes S16 to S18 in FIG. 5 and the processes before and after that will be described.
  • the rotation center setting unit 27 acquires the input position from the input position detection unit 22, and determines whether or not the two input positions are on the input unit 12 (process S16). When the rotation center setting unit 27 determines that the two input positions are on the input unit 12 (YES in S16), the rotation center setting unit 27 further refers to temporal changes in the two input positions. Then, it is determined whether or not the two input positions are rotating (processing S17).
  • the rotation center setting unit 27 sets the center position of the display unit 11 as the rotation center position of the image.
  • the angle correction unit 28 rotates the image by an angle proportional to the rotation angle of the two input positions according to the rotation direction of the two input positions, with the center position of the display unit 11 as the center position of the image rotation. (Process S18). Then, the process returns to the process S13, and the processes S13 to S16 are repeated.
  • rotation center setting unit 27 determines that the two input positions are not on input unit 12 (NO in S16)
  • rotation center setting unit 27 determines that the two input positions are not rotating. If it has been (NO in S17), the image is not rotated, and the translation result displayed in the process S15 is continuously displayed on the display unit 11.
  • FIG. 6 is a flowchart illustrating the flow of translation processing in the smartphone 2.
  • Processes S21 to S28 in FIG. 6 exemplify a process flow in the case where the midpoint between the two input positions is set as the rotation center position of the image. Note that the processes S21 to S27 in FIG. 6 are the same as the processes S11 to S17 in FIG. Therefore, only the process S28 in FIG. 6 and the processes before and after that will be described.
  • the rotation center setting unit 27 sets the middle point of the two input positions as the rotation center position of the image.
  • the angle correction unit 28 rotates the image by an angle proportional to the rotation angle of the two input positions according to the rotation direction of the two input positions, with the middle point of the two input positions as the center position of the rotation of the images. (Step S28). Then, the process returns to the process S23, and the processes S23 to S26 are repeated.
  • FIGS. 7A and 7B show a case where the OCR process and the translation are performed on the English characters included in the image in which the subject is photographed in the smartphone 2 as in FIG. 3 described above. It is a figure illustrated.
  • FIG. 7A is a diagram corresponding to FIG. 3C, and the user performs an operation of rotating two fingers clockwise on the input unit 12 to change the rotation angle of the image. It is a figure which illustrates the time of adjusting.
  • the clockwise rotation angle of the image is determined in proportion to the angle at which the user has rotated two fingers in the clockwise direction.
  • FIG. 7 is a diagram similar to (d) in FIG. 3, and is a diagram illustrating a case where OCR processing and translation are performed on an image whose rotation angle has been adjusted.
  • FIG. 7B illustrates a case where the center position of the display unit 11 is set as the center position of image rotation, as in the first embodiment. Also in FIG. 7B, as in FIG. 3D, all the English characters included in the image are subjected to OCR processing, and the translation result is superimposed on the English characters. 11 is displayed.
  • the user can adjust the rotation angle of the image manually while checking the translation result displayed on the display unit 11 to obtain an appropriate translation result. Can be obtained.
  • the smartphone 2 since the user can easily adjust the rotation angle of the image per operation, the convenience of the user's operation is improved.
  • center position of the display unit 11 is set as the center position of the image rotation, no matter which position of the input unit 12 the user rotates two fingers, Since the translation result can be displayed at the position, the visibility of the translation result is improved.
  • the user can display the translation result at a desired position on the display unit 11.
  • the degree of freedom of screen layout is improved.
  • the smartphones 1 and 2 including the touch panel 10 are illustrated as an example of the information device, but the present invention is another information device that operates using the touch panel 10 as an input device (for example, it can also be applied to a PC). Therefore, the touch panel 10 may be an external input device externally attached to the information device.
  • the imaging unit 30 does not need to be built in the information device, and may be externally attached to the information device.
  • the control blocks (especially the main control units 20 and 26) of the smartphones 1 and 2 may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or a CPU (Central Processing Unit) And may be realized by software.
  • a logic circuit hardware
  • IC chip integrated circuit
  • CPU Central Processing Unit
  • the smartphones 1 and 2 include a CPU that executes instructions of a program that is software that realizes each function, and a ROM (Read Only Memory) in which the program and various data are recorded so as to be readable by the computer (or CPU).
  • a storage device (these are referred to as “recording media”), a RAM (Random Access Memory) for expanding the program, and the like are provided.
  • the objective of this invention is achieved when a computer (or CPU) reads the said program from the said recording medium and runs it.
  • a “non-temporary tangible medium” such as a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used.
  • the program may be supplied to the computer via an arbitrary transmission medium (such as a communication network or a broadcast wave) that can transmit the program.
  • a transmission medium such as a communication network or a broadcast wave
  • the present invention can also be realized in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.
  • An information device (smart phone 1) according to aspect 1 of the present invention is an information device that acquires a first target image including characters to be translated and translates characters included in the first target image.
  • Image rotating means (angle correction unit 23) for generating a second target image obtained by rotating the first target image in response to an input operation, and the second target image as a target for character recognition processing.
  • a character recognition unit (OCR processing unit 24) for recognizing a character included in the character, and a translation result is generated with the character recognized by the character recognition unit as a translation target, and the translation result is translated into the second target image.
  • Translation processing means (translation processing unit 25) for generating a display image superimposed on the object.
  • the user does not perform high-accuracy character recognition processing (i.e., an image that is not upright and inclined) due to reasons such as limited shooting conditions of the subject.
  • high-accuracy character recognition processing i.e., an image that is not upright and inclined
  • the second target image ie, the character recognition process and the subsequent translation process are appropriately performed by manually adjusting the rotation angle of the target image by its own input operation (that is, A generally upright image) can be obtained.
  • the user can perform the above-described adjustment operation while confirming the display image generated in the translation processing means with a display device (for example, a touch panel of a smartphone), which is suitable for character recognition processing and subsequent translation processing.
  • a display device for example, a touch panel of a smartphone
  • a second target image can be obtained.
  • the information apparatus which concerns on aspect 2 of this invention is the rotation which is the object which performs the input operation for the touch panel (10) which receives a user's input operation in the said aspect 1, and a user to rotate the said 1st target image.
  • Rotation button display means (angle correction unit 23) for displaying buttons on the touch panel, and the image rotation means rotates a predetermined position on the touch panel every time the user taps the rotation button once.
  • the second target image may be generated by rotating the first target image in a predetermined direction by a predetermined angle.
  • the user can tap the rotation button displayed on the touch panel and obtain the second target image while rotating the first target image by a predetermined angle (for example, 1 °).
  • a predetermined angle for example, 1 °.
  • the information device is the information apparatus according to aspect 2, in which the image rotation unit is configured to perform a predetermined operation on the touch panel when the user rotates two fingers on the touch panel in a predetermined direction.
  • the second target image may be generated by rotating the first target image in a predetermined direction by an angle proportional to the rotation angle of the two fingers with the position as the rotation center position.
  • the convenience of operation by the user can be improved.
  • a display image including the translation result can be displayed at a predetermined position on the touch panel (for example, a position near the center). The visibility of the translation result can be improved.
  • the information device is the information apparatus according to aspect 2, in which the two input positions respectively corresponding to the positions of the two fingers at the time when the user places the two fingers on the touch panel.
  • Rotation center calculation means (rotation center setting unit 27) for calculating a midpoint is further provided, and the image rotation means is configured such that when the user rotates the two fingers in a predetermined direction on the touch panel, The second target image may be generated by rotating the first target image in a predetermined direction by an angle proportional to the rotation angle of the two fingers with the point as the rotation center position.
  • the user can easily adjust the rotation angle of the image per operation, the convenience of the operation by the user can be improved. Further, by setting the middle point of the two input positions as the center position of the rotation of the image, the user can display a display image including the translation result at a desired position of the display unit 11. The degree of freedom of the screen layout can be improved.
  • the information device may be realized by a computer, and in this case, information for realizing the information device by the computer by causing the computer to operate as each unit included in the information device.
  • An apparatus control program and a computer-readable recording medium on which the apparatus control program is recorded also fall within the scope of the present invention.
  • the present invention can be used for information equipment.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Artificial Intelligence (AREA)
  • Multimedia (AREA)
  • Character Input (AREA)
  • Image Processing (AREA)
  • Position Input By Displaying (AREA)
  • Machine Translation (AREA)
  • Document Processing Apparatus (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

This smartphone (1) is provided with the following: an angle correction unit (23) that generates a second target image by rotating, in accordance with an input operation performed by a user, a first target image that contains text to be translated; an OCR unit (24) that recognizes the text in the second target image; and a translation unit (25) that generates a translation result from the text that was successfully recognized by the OCR unit (24) and generates a display image comprising said translation result superimposed onto the part of the second target image that was translated.

Description

情報機器および制御プログラムInformation equipment and control program
　本発明は、情報機器に関する。 The present invention relates to information equipment.
　現在、ＯＣＲ（Optical Character Recognition，光学文字認識）処理の技術は、イメージスキャナ、ＰＣ（Personal Computer）、およびスマートフォン等の様々な情報機器に適用されている。 Currently, OCR (Optical Character Recognition) technology is applied to various information devices such as image scanners, PCs (Personal Computers), and smartphones.
　特に、近年では、米国Quest Visual社のWord Lensというスマートフォン向けのアプリケーションに、ＯＣＲ処理の技術が適用されている。Word Lensは、スマートフォンのカメラが被写体を撮影することによって生成された画像に対してＯＣＲ処理を行うことにより、ＯＣＲ処理によって得られた文字を翻訳した結果を、カメラアプリケーションのプレビュー画像に表示させるという機能を有している。 In particular, in recent years, OCR processing technology has been applied to an application for a smartphone called Word Lens of Quest Visual. Word Lens says that the result of translating characters obtained by OCR processing is displayed in the preview image of the camera application by performing OCR processing on the image generated by the camera of the smartphone shooting the subject. It has a function.
　ところで、ＯＣＲ処理を行う対象となる画像が傾いている場合（すなわち、画像が正立していない場合）には、ＯＣＲ処理の精度が低下し、画像に含まれている文字が、正しく認識されないという問題がある。特許文献１には、画像の回転角度（傾き角）を自動的に補正することによって、ＯＣＲ処理の精度を向上する方法が開示されている。 By the way, when the image to be subjected to the OCR process is tilted (that is, when the image is not upright), the accuracy of the OCR process is reduced, and the characters included in the image are not recognized correctly. There is a problem. Patent Document 1 discloses a method for improving the accuracy of OCR processing by automatically correcting the rotation angle (tilt angle) of an image.
日本国公開特許公報「特開２００５－１９６５６３号公報（２００５年７月２１日公開）」Japanese Patent Publication “JP 2005-196563 A (published July 21, 2005)”
　特許文献１の方法では、画像の回転角度の算出に先立ち、画像内の文字を含む矩形を自動的に抽出している。ゆえに、画像の回転角度がある程度以上の値である場合には、矩形の抽出が適切に行われず、ＯＣＲ処理の精度を向上するための画像の回転角度の補正が自動的に行うことができないという問題があった。 In the method of Patent Document 1, a rectangle including characters in an image is automatically extracted prior to calculation of the rotation angle of the image. Therefore, when the rotation angle of the image is a certain value or more, the rectangle is not properly extracted, and the correction of the rotation angle of the image for improving the accuracy of the OCR processing cannot be automatically performed. There was a problem.
　また、上述のWord Lensは、カメラによって撮影された画像の回転角度を補正する機能を有していない。それゆえ、撮影時に水平に安定した足場がない場合、または、被写体を撮影することが可能な位置が限られている場合等には、ユーザが正立した画像を撮影することが困難であるため、Word LensによるＯＣＲ処理および翻訳処理が適切に行われないという問題があった。 Also, the above-mentioned Word Lens does not have a function for correcting the rotation angle of the image taken by the camera. Therefore, it is difficult for the user to shoot an upright image when there is no horizontally stable scaffold during shooting, or when the position where the subject can be shot is limited. There was a problem that OCR processing and translation processing by Word Lens were not performed properly.
　本発明は、上記の問題を解決するためになされたものであり、その目的は、ＯＣＲ処理および翻訳処理の精度を向上させることが可能な情報機器を提供することにある。 The present invention has been made to solve the above problems, and an object thereof is to provide an information device capable of improving the accuracy of OCR processing and translation processing.
　上記の課題を解決するために、本発明の一態様に係る情報機器は、翻訳対象となる文字を含む第１対象画像を取得し、当該第１対象画像に含まれる文字を翻訳する情報機器であって、ユーザの入力操作に応じて上記第１対象画像を回転させた第２対象画像を生成する画像回転手段と、上記第２対象画像を文字認識処理の対象として、上記第２対象画像に含まれる文字を認識する文字認識手段と、上記文字認識手段において認識に成功した文字を翻訳対象として翻訳結果を生成し、上記第２対象画像に上記翻訳結果を上記翻訳対象に重畳させた表示用画像を生成する翻訳処理手段と、を備えている。 In order to solve the above-described problem, an information device according to an aspect of the present invention is an information device that acquires a first target image including a character to be translated and translates a character included in the first target image. An image rotating means for generating a second target image obtained by rotating the first target image in accordance with a user input operation, and the second target image as a target for character recognition processing. A character recognition unit for recognizing the included character, a translation result for the character that has been successfully recognized by the character recognition unit, as a translation target, and a display result in which the translation result is superimposed on the translation target in the second target image Translation processing means for generating an image.
　本発明の一態様に係る情報機器によれば、ＯＣＲ処理および翻訳処理の精度を向上させることが可能な情報機器を提供できるという効果を奏する。 According to the information device according to one aspect of the present invention, it is possible to provide an information device that can improve the accuracy of the OCR process and the translation process.
本発明の実施形態１に係るスマートフォンの構成を示すブロック図である。It is a block diagram which shows the structure of the smart phone which concerns on Embodiment 1 of this invention. 本発明の実施形態１に係るスマートフォンの翻訳処理の流れを示す図である。It is a figure which shows the flow of the translation process of the smart phone which concerns on Embodiment 1 of this invention. （ａ）～（ｄ）は、本発明の実施形態１に係るスマートフォンにおいて、画像に含まれる文字に対して、ＯＣＲ処理および翻訳が行われた場合を例示する図である。(A)-(d) is a figure which illustrates the case where the OCR process and translation are performed with respect to the character contained in the image in the smart phone which concerns on Embodiment 1 of this invention. 本発明の実施形態２に係るスマートフォンの構成を示すブロック図である。It is a block diagram which shows the structure of the smart phone which concerns on Embodiment 2 of this invention. 本発明の実施形態２に係るスマートフォンの翻訳処理の流れを示す図である。It is a figure which shows the flow of the translation process of the smart phone which concerns on Embodiment 2 of this invention. 本発明の実施形態２に係るスマートフォンの翻訳処理の流れの別の一例を示す図である。It is a figure which shows another example of the flow of the translation process of the smart phone which concerns on Embodiment 2 of this invention. （ａ）～（ｂ）は、本発明の実施形態２に係るスマートフォンにおいて、画像に含まれる文字に対して、ＯＣＲ処理および翻訳が行われた場合を例示する図である。(A)-(b) is a figure which illustrates the case where the OCR process and translation are performed with respect to the character contained in an image in the smart phone which concerns on Embodiment 2 of this invention.
　〔実施形態１〕
　本発明の実施形態１について図１～図３に基づいて説明すれば、以下の通りである。 Embodiment 1
Embodiment 1 of the present invention will be described with reference to FIGS. 1 to 3 as follows.
　（スマートフォン１の構成）
　図１は、本実施形態のスマートフォン１（情報機器）の構成を概略的に示す機能ブロック図である。スマートフォン１は、タッチパネル１０、主制御部２０、撮影部３０、および記憶部４０を備える。 (Configuration of smartphone 1)
FIG. 1 is a functional block diagram schematically showing the configuration of the smartphone 1 (information device) of the present embodiment. The smartphone 1 includes a touch panel 10, a main control unit 20, a photographing unit 30, and a storage unit 40.
　タッチパネル１０は、例えば、静電容量方式のタッチパネルであり、表示部１１および入力部１２（入力面）を備えている。表示部１１は、例えば液晶ディスプレイである。入力部１２は、ユーザによる入力操作を受け付けるユーザインターフェースであり、表示部１１の表示画面と重畳する位置に配置されている。そのため、入力部１２は、表示部１１の表示画面に対する入力操作を検出することができる。 The touch panel 10 is, for example, a capacitive touch panel, and includes a display unit 11 and an input unit 12 (input surface). The display unit 11 is, for example, a liquid crystal display. The input unit 12 is a user interface that accepts an input operation by the user, and is arranged at a position overlapping the display screen of the display unit 11. Therefore, the input unit 12 can detect an input operation on the display screen of the display unit 11.
　入力部１２は、指（またはタッチペン等）によるタッチ入力が行われたことを示す入力信号（例えば、静電容量の変化が生じた位置、および静電容量の変化量を示す信号）を、主制御部２０が備える入力位置検出部２２へ与える。 The input unit 12 mainly receives an input signal (for example, a signal indicating a change in capacitance and a change in capacitance) indicating that a touch input by a finger (or a touch pen or the like) has been performed. This is given to the input position detection unit 22 provided in the control unit 20.
　撮影部３０は、例えば、表示部１１が配設されている面とは反対の面（すなわち、スマートフォン１の背面）に設けられたメインカメラである。撮影部３０は、被写体を撮影することによって生成した画像を、主制御部２０が備える画像取得部２１へ与える。記憶部４０は、スマートフォン１において使用される各種データを格納する記憶装置である。 The photographing unit 30 is, for example, a main camera provided on the surface opposite to the surface on which the display unit 11 is disposed (that is, the back surface of the smartphone 1). The photographing unit 30 gives an image generated by photographing a subject to the image acquisition unit 21 included in the main control unit 20. The storage unit 40 is a storage device that stores various data used in the smartphone 1.
　主制御部２０は、スマートフォン１が備える各種ハードウェア（例えば、タッチパネル１０および撮影部３０）の動作、およびスマートフォン１の機能を統括的に制御する。また、主制御部２０は、以下に述べる画像取得部２１、入力位置検出部２２、角度補正部２３（画像回転手段、回転ボタン表示手段）、ＯＣＲ処理部２４（文字認識手段）、および翻訳処理部２５（翻訳処理手段）として機能する。 The main control unit 20 comprehensively controls operations of various hardware (for example, the touch panel 10 and the photographing unit 30) included in the smartphone 1 and functions of the smartphone 1. The main control unit 20 includes an image acquisition unit 21, an input position detection unit 22, an angle correction unit 23 (image rotation unit and rotation button display unit), an OCR processing unit 24 (character recognition unit), and a translation process described below. It functions as a section 25 (translation processing means).
　画像取得部２１は、撮影部３０からの画像を表示部１１に表示させる。なお、画像取得部２１は、撮影部３０が現在撮影している画像を、カメラアプリケーションのプレビュー画像として、表示部１１に表示させることもできる（後述の図３の（ａ）を参照）。 The image acquisition unit 21 displays the image from the photographing unit 30 on the display unit 11. Note that the image acquisition unit 21 can also cause the display unit 11 to display an image currently captured by the imaging unit 30 as a preview image of the camera application (see FIG. 3A described later).
　入力位置検出部２２は、入力部１２からの入力信号に基づき、入力部１２に対するユーザからの入力が与えられた位置である入力位置を検出する。表示部１１には、ユーザによる入力操作の対象として各種のボタン（またはアイコン）が表示されており、ユーザは、各ボタンをタップすることにより、入力位置検出部２２を介して、当該ボタンに対応付けられた動作をスマートフォン１に行わせることができる。また、ユーザは、ボタンのタップに限らず、入力部１２におけるドラッグ操作等によっても、入力位置検出部２２を介して、所定の各種の動作をスマートフォン１に行わせることができる。 The input position detection unit 22 detects an input position that is a position where an input from the user to the input unit 12 is given based on an input signal from the input unit 12. Various buttons (or icons) are displayed on the display unit 11 as an input operation target by the user, and the user responds to the button via the input position detection unit 22 by tapping each button. The attached operation can be performed by the smartphone 1. Further, the user can cause the smartphone 1 to perform various predetermined operations via the input position detection unit 22 not only by tapping a button but also by a drag operation or the like on the input unit 12.
　角度補正部２３（画像回転手段）は、表示部１１に表示されている画像を回転させ、画像の回転角度を調整（補正）する機能を有している。角度補正部２３（回転ボタン表示手段）は、「左回転」または「右回転」のボタン（回転ボタン）を表示部１１に表示させる。ユーザが、表示部１１に表示されている回転ボタンをタップすることによって、角度補正部２３における処理が行われる。 The angle correction unit 23 (image rotation means) has a function of rotating (rotating) the image displayed on the display unit 11 and adjusting (correcting) the rotation angle of the image. The angle correction unit 23 (rotation button display means) causes the display unit 11 to display a “left rotation” or “right rotation” button (rotation button). When the user taps the rotation button displayed on the display unit 11, processing in the angle correction unit 23 is performed.
　なお、本実施形態では、角度補正部２３が、画像回転手段としての機能に加えて、回転ボタン表示手段としての機能を備えるように構成している場合を例示しているが、画像回転手段と回転ボタン表示手段とは、個別に設けられてもよい。例えば、角度補正部２３に画像回転手段としての機能のみを備えさせ、回転ボタン表示手段としての機能を有する機能ブロックを、主制御部２０内に別途に設けてもよい。 In the present embodiment, the angle correction unit 23 is illustrated as having a function as a rotation button display unit in addition to a function as an image rotation unit. The rotation button display means may be provided separately. For example, the angle correction unit 23 may be provided with only a function as an image rotation unit, and a functional block having a function as a rotation button display unit may be separately provided in the main control unit 20.
　ユーザが「左回転」のボタンを１回タップするごとに、角度補正部２３は、画像を反時計方向に１°回転させる。また、ユーザが「右回転」のボタンを１回タップするごとに、角度補正部２３は、画像を時計方向に１°回転させる（後述の図３の（ｃ）を参照）。 Each time the user taps the “rotate left” button once, the angle correction unit 23 rotates the image counterclockwise by 1 °. Further, each time the user taps the “rotate right” button once, the angle correction unit 23 rotates the image by 1 ° in the clockwise direction (see FIG. 3C described later).
　なお、回転ボタンを１回タップするごとの画像の回転角度は、１°に限定されず、スマートフォン１の仕様に応じた他の適当な値（例えば５°）でもよい。また、画像の回転の中心位置（回転中心位置）は、例えば、表示部１１の中心位置である。なお、画像の回転の中心位置は、表示部１１の中心位置以外の所定の位置であってもよく、角度補正部２３に、画像の回転の中心位置を示す座標があらかじめ与えられていればよい。 In addition, the rotation angle of the image every time the rotation button is tapped is not limited to 1 °, and may be another appropriate value (for example, 5 °) according to the specification of the smartphone 1. Further, the rotation center position (rotation center position) of the image is, for example, the center position of the display unit 11. It should be noted that the center position of the image rotation may be a predetermined position other than the center position of the display unit 11, and the coordinates indicating the center position of the image rotation may be given to the angle correction unit 23 in advance. .
　そして、角度補正部２３は、回転後の画像を、表示部１１に表示させる。また、角度補正部２３は、回転後の画像を、ＯＣＲ処理部２４に与える。なお、角度補正部２３において、画像の回転が行われない場合には、画像取得部２１が撮影部３０から取得した画像が、そのまま角度補正部２３からＯＣＲ処理部２４に与えられる。 Then, the angle correction unit 23 causes the display unit 11 to display the rotated image. The angle correction unit 23 gives the rotated image to the OCR processing unit 24. When the angle correction unit 23 does not rotate the image, the image acquired by the image acquisition unit 21 from the photographing unit 30 is directly supplied from the angle correction unit 23 to the OCR processing unit 24.
　ＯＣＲ処理部２４は、表示部１１に表示されている画像に対して、ＯＣＲ処理（文字認識処理）を行う。すなわち、ＯＣＲ処理部２４は、画像に含まれる文字を、計算機による処理が可能な文字コードに変換する。そして、ＯＣＲ処理部２４は、ＯＣＲ処理の結果として得られた文字コードと当該文字コードに対応する画像中の文字の座標とを、翻訳処理部２５に与える。 The OCR processing unit 24 performs OCR processing (character recognition processing) on the image displayed on the display unit 11. That is, the OCR processing unit 24 converts characters included in the image into character codes that can be processed by a computer. Then, the OCR processing unit 24 gives the character code obtained as a result of the OCR process and the coordinates of the character in the image corresponding to the character code to the translation processing unit 25.
　なお、ＯＣＲ処理部２４において、ＯＣＲ処理の対象となる画像は、角度補正部２３における回転が行われた後の画像（第２対象画像）であってもよいし、または、角度補正部２３における回転が行われる前の画像（第１対象画像）であってもよい。 Note that, in the OCR processing unit 24, the image to be subjected to the OCR processing may be an image after the rotation in the angle correction unit 23 (second target image) or in the angle correction unit 23. It may be an image before the rotation (first target image).
　翻訳処理部２５は、ソース言語（例えば、英語）の文字コードを、所定のターゲット言語（例えば、日本語）に翻訳する機能を有する。翻訳処理部２５は、記憶部４０にあらかじめ記録されている辞書データ（ソース言語の各語の語義が、ターゲット言語により示された２言語辞書のデータ）を参照することによって、上述の翻訳を行う。 The translation processing unit 25 has a function of translating a character code of a source language (for example, English) into a predetermined target language (for example, Japanese). The translation processing unit 25 performs the above-described translation by referring to dictionary data (bilingual dictionary data in which the meaning of each word in the source language is indicated by the target language) recorded in the storage unit 40 in advance. .
　翻訳処理部２５は、ＯＣＲ処理部２４から与えられた文字コードに対して翻訳を行い、翻訳結果を生成する。そして、翻訳処理部２５は、翻訳が行われた文字コードに対応する画像中の文字の座標を参照し、画像内の翻訳対象の文字に翻訳結果を重畳させた表示用画像を生成し、当該表示用画像を表示部１１に表示させる。翻訳結果を重畳させる対象となる画像は、上述の第１対象画像または第２対象画像のいずれでもよい（後述の図３の（ｂ）および（ｄ）を参照）。 The translation processing unit 25 translates the character code given from the OCR processing unit 24 and generates a translation result. Then, the translation processing unit 25 refers to the character coordinates in the image corresponding to the translated character code, generates a display image in which the translation result is superimposed on the character to be translated in the image, and A display image is displayed on the display unit 11. The target image on which the translation result is superimposed may be either the first target image or the second target image described above (see (b) and (d) of FIG. 3 described later).
　（スマートフォン１における翻訳の処理の流れ）
　図２は、スマートフォン１における翻訳の処理の流れを例示するフローチャートである。ユーザは、スマートフォン１に入力操作を行うことにより、撮影部３０を起動させ、撮影部３０による被写体の撮影を開始する（Ｓ１）。画像取得部２１は、撮影部３０が被写体を撮影することによって生成した画像を取得し（Ｓ２）、当該画像を表示部１１に表示させる。 (Flow of translation processing on smartphone 1)
FIG. 2 is a flowchart illustrating the flow of translation processing in the smartphone 1. The user activates the photographing unit 30 by performing an input operation on the smartphone 1, and starts photographing the subject by the photographing unit 30 (S1). The image acquisition unit 21 acquires an image generated by the shooting unit 30 shooting a subject (S2), and causes the display unit 11 to display the image.
　ＯＣＲ処理部２４は、表示部１１に表示されている画像に対して、ＯＣＲ処理を行い（Ｓ３）、画像に含まれる文字を、計算機による処理が可能な文字コードに変換する。翻訳処理部２５は、文字コードに対して翻訳を行い、翻訳結果を生成する（Ｓ４）。 The OCR processing unit 24 performs an OCR process on the image displayed on the display unit 11 (S3), and converts characters included in the image into a character code that can be processed by a computer. The translation processing unit 25 translates the character code and generates a translation result (S4).
　そして、翻訳処理部２５は、画像内の翻訳対象の文字に翻訳結果を重畳させて、表示部１１に表示させる。すなわち、翻訳処理部２５は、生成した表示用画像を表示部１１に表示させる（Ｓ５）。角度補正部２３は、ユーザが表示部１１に表示されている回転ボタン（「左回転」または「右回転」のボタン）をタップしたか否かを確認する（処理Ｓ６）。 Then, the translation processing unit 25 causes the display unit 11 to display the translation result on the character to be translated in the image. That is, the translation processing unit 25 displays the generated display image on the display unit 11 (S5). The angle correction unit 23 checks whether or not the user has tapped the rotation button (“left rotation” or “right rotation” button) displayed on the display unit 11 (process S6).
　ユーザが表示部１１に表示されている回転ボタンをタップした場合（Ｓ６においてＹＥＳ）、角度補正部２３は、ユーザがタップした回転ボタン（「左回転」または「右回転」のボタン）の方向に応じて、ユーザが回転ボタンを１回タップするごとに、所定の方向（反時計方向または時計方向）に、所定の角度（例えば１°）だけ画像を回転させる（Ｓ７）。そして、処理Ｓ３に戻り、処理Ｓ３～Ｓ６が反復される。 When the user taps the rotation button displayed on the display unit 11 (YES in S6), the angle correction unit 23 moves in the direction of the rotation button (the “left rotation” or “right rotation” button) tapped by the user. Accordingly, each time the user taps the rotation button once, the image is rotated by a predetermined angle (for example, 1 °) in a predetermined direction (counterclockwise or clockwise) (S7). Then, the process returns to the process S3, and the processes S3 to S6 are repeated.
　他方、ユーザが表示部１１に表示されている回転ボタンをタップしない場合（Ｓ６においてＮＯ）には、画像の回転が行われず、表示部１１には、処理Ｓ５において表示された翻訳結果が、継続して表示されている。 On the other hand, when the user does not tap the rotation button displayed on the display unit 11 (NO in S6), the image is not rotated, and the translation result displayed in the process S5 is continuously displayed on the display unit 11. Is displayed.
　（スマートフォン１の効果）
　スマートフォン１の効果について、図３を用いて説明する。図３の（ａ）～（ｄ）は、スマートフォン１において、被写体が撮影された画像に含まれる英語の文字「Chapter 1 This is a pen. How are you?」に対して、ＯＣＲ処理および翻訳が行われた場合を例示する図である。図３の（ａ）は、撮影部３０が現在撮影している被写体の画像を、カメラアプリケーションのプレビュー画像として、表示部１１に表示されている時点を例示する図である。 (Effect of smartphone 1)
The effect of the smartphone 1 will be described with reference to FIG. FIGS. 3A to 3D show that the smartphone 1 performs OCR processing and translation on the English character “Chapter 1 This is a pen. How are you?” Included in the image of the subject. It is a figure which illustrates the case where it was performed. FIG. 3A is a diagram illustrating a point in time when the image of the subject currently captured by the image capturing unit 30 is displayed on the display unit 11 as a preview image of the camera application.
　図３の（ｂ）は、撮影部３０が被写体を撮影することによって生成された画像に、ＯＣＲ処理および翻訳が行われた場合を例示する図である。図６の（ｂ）では、ＯＣＲ処理を行う対象となる画像が傾いているために、ＯＣＲ処理部２４におけるＯＣＲ処理の精度が低下し、画像に含まれている文字の全ては、正しく認識されていない。 FIG. 3B is a diagram illustrating a case where OCR processing and translation are performed on an image generated when the photographing unit 30 photographs a subject. In FIG. 6B, since the image to be subjected to the OCR process is tilted, the accuracy of the OCR process in the OCR processing unit 24 is reduced, and all the characters included in the image are correctly recognized. Not.
　図６の（ｂ）の場合、ＯＣＲ処理部２４は、英語の文字の一部である「is」のＯＣＲ処理にのみ成功している。このため、翻訳処理部２５は、ＯＣＲ処理部２４におけるＯＣＲ処理が成功した英語の文字「is」に対してのみ翻訳処理を行い、日本語の文字「です」を翻訳結果として生成する。そして、翻訳処理部２５による翻訳結果としての日本語の文字「です」が、英語の文字「is」に重畳され、表示部１１に表示されている。 In the case of FIG. 6B, the OCR processing unit 24 succeeds only in the OCR processing of “is” which is a part of English characters. For this reason, the translation processing unit 25 performs the translation process only on the English character “is” that has been successfully subjected to the OCR processing in the OCR processing unit 24, and generates the Japanese character “is” as the translation result. The Japanese character “Is” as a result of translation by the translation processing unit 25 is superimposed on the English character “is” and displayed on the display unit 11.
　図３の（ｃ）は、ユーザが表示部１１に表示されている「右回転」のボタンをタップすることによって、画像の回転角度を調整している時点を例示する図である。ユーザが、「右回転」のボタンを１回タップするごとに、画像が時計方向に１°回転する。ユーザが画像を適当な回数だけ回転させる操作を行うことにより、ユーザは、画像が概ね正立するように、画像の回転角度を手動で調整することができる。 FIG. 3C is a diagram illustrating a point in time when the user adjusts the rotation angle of the image by tapping the “right rotation” button displayed on the display unit 11. Each time the user taps the “rotate right” button once, the image rotates 1 ° clockwise. When the user performs an operation of rotating the image an appropriate number of times, the user can manually adjust the rotation angle of the image so that the image is generally upright.
　図３の（ｄ）は、回転角度が調整された画像に、ＯＣＲ処理および翻訳が行われた場合を例示する図である。図３の（ｄ）では、ＯＣＲ処理を行う対象となる画像が概ね正立しているために、ＯＣＲ処理の精度が低下せず、画像に含まれている文字の全てが、正しく認識されている。このため、上述の英語の文字の全てがＯＣＲ処理され、日本語の文字「１章　これはペンです　ごきげんいかが？」が、翻訳結果として上述の英語の文字に重畳され、表示部１１に表示されている。 (D) of FIG. 3 is a figure which illustrates the case where an OCR process and translation are performed to the image in which the rotation angle was adjusted. In FIG. 3D, since the image to be subjected to the OCR processing is generally upright, the accuracy of the OCR processing does not decrease, and all characters included in the image are correctly recognized. Yes. For this reason, all of the above-mentioned English characters are subjected to OCR processing, and the Japanese character “Chapter 1 This is a pen?” Is superimposed on the above-mentioned English characters as a translation result and displayed on the display unit 11. It is displayed.
　従って、ユーザは、正立した画像を撮影することができなかった場合でも、表示部１１に表示された翻訳結果を確認しながら、適切な翻訳結果が得られるまで、回転ボタンをタップし所定の角度（例えば１°）ずつ画像の回転角度を調整することができる。それゆえ、スマートフォン１によれば、画像の回転角度を調整するためのユーザによる操作の利便性が向上する。 Therefore, even when the user cannot take an upright image, he / she taps the rotation button until an appropriate translation result is obtained while confirming the translation result displayed on the display unit 11. The rotation angle of the image can be adjusted by an angle (for example, 1 °). Therefore, according to the smartphone 1, the convenience of the operation by the user for adjusting the rotation angle of the image is improved.
　〔実施形態２〕
　本発明の他の実施形態について、図４～図７に基づいて説明すれば、以下のとおりである。なお、説明の便宜上、前記実施形態にて説明した部材と同じ機能を有する部材については、同じ符号を付記し、その説明を省略する。 [Embodiment 2]
The following will describe another embodiment of the present invention with reference to FIGS. For convenience of explanation, members having the same functions as those described in the embodiment are given the same reference numerals, and descriptions thereof are omitted.
　（スマートフォン２の構成）
　図４は、本実施形態のスマートフォン２（情報機器）の構成を概略的に示す機能ブロック図である。スマートフォン２は、実施形態１のスマートフォン１が備える主制御部２０を、主制御部２６によって置き換えたものである。本実施形態の主制御部２６は、実施形態１の主制御部２０において、（ｉ）回転中心設定部２７（回転中心算出手段）をさらに付加し、かつ、（ｉｉ）角度補正部２３を、角度補正部２８（画像回転手段）によって置き換えたものである。 (Configuration of smartphone 2)
FIG. 4 is a functional block diagram schematically showing the configuration of the smartphone 2 (information device) of the present embodiment. The smartphone 2 is obtained by replacing the main control unit 20 included in the smartphone 1 of the first embodiment with the main control unit 26. The main control unit 26 of the present embodiment is the same as the main control unit 20 of the first embodiment except that (i) a rotation center setting unit 27 (rotation center calculation means) is further added, and (ii) the angle correction unit 23 is It is replaced by the angle correction unit 28 (image rotation means).
　回転中心設定部２７は、後述の角度補正部２８における画像の回転処理に先立ち、回転の中心位置を設定する機能を有している。回転中心設定部２７は、入力位置検出部２２から２つの入力位置を取得し、ユーザが、入力部１２上に２本の指を置いていることを検知する。 The rotation center setting unit 27 has a function of setting a rotation center position prior to image rotation processing in an angle correction unit 28 described later. The rotation center setting unit 27 acquires two input positions from the input position detection unit 22 and detects that the user is placing two fingers on the input unit 12.
　回転中心設定部２７は、ユーザが入力部１２上に２本の指を置いた時点における、２本の指の位置にそれぞれ対応する２つの入力位置の中点を、画像の回転の中心位置（回転中心位置）として設定してもよい。また。回転中心設定部２７は、表示部１１の中心位置等の所定の位置（すなわち、実施形態１の角度補正部２３と同様の画像の回転の中心位置）を、画像の回転の中心位置として設定してもよい。 The rotation center setting unit 27 sets the midpoints of the two input positions corresponding to the positions of the two fingers at the time when the user places the two fingers on the input unit 12 as the rotation center position ( (Rotation center position). Also. The rotation center setting unit 27 sets a predetermined position such as the center position of the display unit 11 (that is, the image rotation center position similar to the angle correction unit 23 of the first embodiment) as the image rotation center position. May be.
　回転中心設定部２７は、画像の回転の中心位置を示す座標を、角度補正部２８に与える。なお、回転中心設定部２７において、２つの入力位置の中点、または表示部１１の中心位置のいずれを、画像の回転の中心位置として設定するかについては、回転中心設定部２７の処理に先立ち、ユーザが任意に設定することができる。 The rotation center setting unit 27 gives coordinates indicating the rotation center position of the image to the angle correction unit 28. Prior to the processing of the rotation center setting unit 27, the rotation center setting unit 27 determines which of the midpoints of the two input positions or the center position of the display unit 11 is set as the rotation center position of the image. The user can arbitrarily set.
　本実施形態の角度補正部２８は、実施形態１の角度補正部２３と同様に、表示部１１に表示されている画像を回転させ、画像の回転角度を調整する機能を有している。但し、本実施形態の角度補正部２８は、ユーザの入力操作を受け付けるためのインターフェースが、実施形態１の角度補正部２３とは異なる。本実施形態では、ユーザが、入力部１２上に２本の指を置いた後に、入力部１２上において２本の指を回転させることによって、角度補正部２８における処理が行われる。 The angle correction unit 28 of the present embodiment has a function of rotating the image displayed on the display unit 11 and adjusting the rotation angle of the image, like the angle correction unit 23 of the first embodiment. However, the angle correction unit 28 of the present embodiment is different from the angle correction unit 23 of the first embodiment in the interface for accepting the user's input operation. In the present embodiment, after the user puts two fingers on the input unit 12, the angle correction unit 28 performs processing by rotating the two fingers on the input unit 12.
　角度補正部２８は、入力位置検出部２２から取得した２つの入力位置の時間的な変化を参照し、２つの入力位置が回転していることを検出することにより、ユーザが２本の指を反時計方向または時計方向に回転させる操作を行っていることを認識する。 The angle correction unit 28 refers to the temporal change in the two input positions acquired from the input position detection unit 22 and detects that the two input positions are rotating, so that the user holds two fingers. Recognize that an operation of rotating counterclockwise or clockwise is performed.
　ユーザが２本の指を反時計方向に回転させた場合、角度補正部２８は、画像を反時計方向に回転させる。また、ユーザが２本の指を時計方向に回転させた場合、角度補正部２８は、画像を時計方向に回転させる（後述の図７の（ａ）を参照）。 When the user rotates the two fingers counterclockwise, the angle correction unit 28 rotates the image counterclockwise. When the user rotates two fingers in the clockwise direction, the angle correction unit 28 rotates the image in the clockwise direction (see FIG. 7A described later).
　ここで、画像の回転の中心位置は、回転中心設定部２７において設定された位置である。なお、角度補正部２８における画像の回転角度は、ユーザが２本の指を回転させた角度（すなわち、２つの入力位置の回転角度）に比例した量として決定される。例えば、ユーザが２本の指を６０°回転させた場合には、角度補正部２８は、画像を３０°回転させる。なお、ユーザの２本の指の回転角度と、角度補正部２８における画像の回転角度との間の比例定数は、スマートフォン２の仕様に応じて、適宜設定されてよい。 Here, the rotation center position of the image is a position set in the rotation center setting unit 27. Note that the rotation angle of the image in the angle correction unit 28 is determined as an amount proportional to the angle at which the user has rotated the two fingers (that is, the rotation angle of the two input positions). For example, when the user rotates two fingers by 60 °, the angle correction unit 28 rotates the image by 30 °. The proportionality constant between the rotation angle of the two fingers of the user and the rotation angle of the image in the angle correction unit 28 may be set as appropriate according to the specifications of the smartphone 2.
　そして、角度補正部２８は、実施形態１の角度補正部２３と同様に、回転後の画像を、表示部１１に表示させ、また、ＯＣＲ処理部２４に与える。なお、「左回転」または「右回転」の回転ボタンを表示部１１に表示させ、本実施形態の角度補正部２８に、実施形態１の角度補正部２３と同様の機能をさらに備えさせてもよい。 Then, the angle correction unit 28 displays the rotated image on the display unit 11 and gives it to the OCR processing unit 24 as with the angle correction unit 23 of the first embodiment. Note that a rotation button of “left rotation” or “right rotation” may be displayed on the display unit 11, and the angle correction unit 28 of the present embodiment may be further provided with the same function as the angle correction unit 23 of the first embodiment. Good.
　また、表示部１１の中心位置等の所定の位置を、画像の回転の中心位置として設定するようにスマートフォン２を構成する場合には、回転中心設定部２７を除外し、角度補正部２８に画像の回転の中心位置の座標をあらかじめ与えればよい。 Further, when the smartphone 2 is configured to set a predetermined position such as the center position of the display unit 11 as the center position of image rotation, the rotation center setting unit 27 is excluded and the angle correction unit 28 receives the image. The coordinates of the center position of the rotation may be given in advance.
　（スマートフォン２における翻訳の処理の流れ：その１）
　図５は、スマートフォン２における翻訳の処理の流れを例示するフローチャートである。図５の処理Ｓ１１～Ｓ１８は、表示部１１の中心位置を、画像の回転の中心位置とする場合の処理の流れを例示している。なお、図５の処理Ｓ１１～Ｓ１５は、上述の図４の処理Ｓ１～Ｓ５と同様である。従って、図５の処理Ｓ１６～Ｓ１８ならびにその前後の処理についてのみ説明する。 (Flow of translation processing on smartphone 2: Part 1)
FIG. 5 is a flowchart illustrating the flow of translation processing in the smartphone 2. Processes S11 to S18 in FIG. 5 exemplify the process flow when the center position of the display unit 11 is set as the center position of image rotation. Note that the processes S11 to S15 in FIG. 5 are the same as the processes S1 to S5 in FIG. Therefore, only the processes S16 to S18 in FIG. 5 and the processes before and after that will be described.
　処理Ｓ１５の後に、回転中心設定部２７は、入力位置検出部２２から入力位置を取得し、２つの入力位置が入力部１２上にあるか否かを判定する（処理Ｓ１６）。そして、回転中心設定部２７が、２つの入力位置が入力部１２上にあると判定した場合（Ｓ１６においてＹＥＳ）、回転中心設定部２７は、さらに、２つの入力位置の時間的な変化を参照し、２つの入力位置が回転しているか否かを判定する（処理Ｓ１７）。 After the process S15, the rotation center setting unit 27 acquires the input position from the input position detection unit 22, and determines whether or not the two input positions are on the input unit 12 (process S16). When the rotation center setting unit 27 determines that the two input positions are on the input unit 12 (YES in S16), the rotation center setting unit 27 further refers to temporal changes in the two input positions. Then, it is determined whether or not the two input positions are rotating (processing S17).
　回転中心設定部２７は、２つの入力位置が回転していると判定した場合（Ｓ１７においてＹＥＳ）、表示部１１の中心位置を、画像の回転の中心位置として設定する。そして、角度補正部２８は、表示部１１の中心位置を画像の回転の中心位置として、２つの入力位置の回転方向に応じて、２つの入力位置の回転角度に比例する角度だけ画像を回転させる（処理Ｓ１８）。そして、処理Ｓ１３に戻り、処理Ｓ１３～Ｓ１６が反復される。 When it is determined that the two input positions are rotating (YES in S17), the rotation center setting unit 27 sets the center position of the display unit 11 as the rotation center position of the image. Then, the angle correction unit 28 rotates the image by an angle proportional to the rotation angle of the two input positions according to the rotation direction of the two input positions, with the center position of the display unit 11 as the center position of the image rotation. (Process S18). Then, the process returns to the process S13, and the processes S13 to S16 are repeated.
　他方、回転中心設定部２７が、２つの入力位置が入力部１２上にないと判定した場合（Ｓ１６においてＮＯ）、および、回転中心設定部２７が、２つの入力位置が回転していないと判定した場合（Ｓ１７においてＮＯ）には、画像の回転が行われず、表示部１１には、処理Ｓ１５において表示された翻訳結果が、継続して表示されている。 On the other hand, when rotation center setting unit 27 determines that the two input positions are not on input unit 12 (NO in S16), rotation center setting unit 27 determines that the two input positions are not rotating. If it has been (NO in S17), the image is not rotated, and the translation result displayed in the process S15 is continuously displayed on the display unit 11.
　（スマートフォン２における翻訳の処理の流れ：その２）
　図６は、スマートフォン２における翻訳の処理の流れを例示するフローチャートである。図６の処理Ｓ２１～Ｓ２８は、２つの入力位置の中点を、画像の回転の中心位置とする場合の処理の流れを例示している。なお、図６の処理Ｓ２１～Ｓ２７は、上述の図５の処理Ｓ１１～Ｓ１７と同様である。従って、図６の処理Ｓ２８ならびにその前後の処理についてのみ説明する。 (Flow of translation processing on smartphone 2: Part 2)
FIG. 6 is a flowchart illustrating the flow of translation processing in the smartphone 2. Processes S21 to S28 in FIG. 6 exemplify a process flow in the case where the midpoint between the two input positions is set as the rotation center position of the image. Note that the processes S21 to S27 in FIG. 6 are the same as the processes S11 to S17 in FIG. Therefore, only the process S28 in FIG. 6 and the processes before and after that will be described.
　回転中心設定部２７は、２つの入力位置が回転していると判定した場合（Ｓ２７においてＹＥＳ）、２つの入力位置の中点を、画像の回転の中心位置として設定する。そして、角度補正部２８は、２つの入力位置の中点を画像の回転の中心位置として、２つの入力位置の回転方向に応じて、２つの入力位置の回転角度に比例する角度だけ画像を回転させる（処理Ｓ２８）。そして、処理Ｓ２３に戻り、処理Ｓ２３～Ｓ２６が反復される。 When it is determined that the two input positions are rotating (YES in S27), the rotation center setting unit 27 sets the middle point of the two input positions as the rotation center position of the image. Then, the angle correction unit 28 rotates the image by an angle proportional to the rotation angle of the two input positions according to the rotation direction of the two input positions, with the middle point of the two input positions as the center position of the rotation of the images. (Step S28). Then, the process returns to the process S23, and the processes S23 to S26 are repeated.
　（スマートフォン２の効果）
　スマートフォン２の効果について、図７を用いて説明する。図７の（ａ）および（ｂ）は、スマートフォン２において、上述の図３と同様に、被写体が撮影された画像に含まれる英語の文字に対して、ＯＣＲ処理および翻訳が行われた場合を例示する図である。 (Effect of smartphone 2)
The effect of the smartphone 2 will be described with reference to FIG. FIGS. 7A and 7B show a case where the OCR process and the translation are performed on the English characters included in the image in which the subject is photographed in the smartphone 2 as in FIG. 3 described above. It is a figure illustrated.
　図７の（ａ）は、図３の（ｃ）に相当する図であり、ユーザは入力部１２上において、２本の指を時計方向に回転させる操作を行うことによって、画像の回転角度を調整している時点を例示する図である。ユーザが時計方向に２本の指を回転した角度に比例して、画像の時計方向の回転角度が決定される。ユーザが２本の指を適当な方向に、かつ適当な角度だけ回転させる操作を行うことにより、ユーザは画像が概ね正立するように、画像の回転角度を手動で調整することができる。 FIG. 7A is a diagram corresponding to FIG. 3C, and the user performs an operation of rotating two fingers clockwise on the input unit 12 to change the rotation angle of the image. It is a figure which illustrates the time of adjusting. The clockwise rotation angle of the image is determined in proportion to the angle at which the user has rotated two fingers in the clockwise direction. When the user performs an operation of rotating two fingers in an appropriate direction and by an appropriate angle, the user can manually adjust the rotation angle of the image so that the image is almost upright.
　図７の（ｂ）は、図３の（ｄ）と同様の図であり、回転角度が調整された画像に、ＯＣＲ処理および翻訳が行われた場合を例示する図である。なお、図７の（ｂ）は、実施形態１と同様に、表示部１１の中心位置が、画像の回転の中心位置として設定されている場合を例示している。図７の（ｂ）においても、図３の（ｄ）と同様に、画像に含まれている英語の文字の全てがＯＣＲ処理され、かつ、翻訳結果が当該英語の文字に重畳され、表示部１１に表示されている。 (B) in FIG. 7 is a diagram similar to (d) in FIG. 3, and is a diagram illustrating a case where OCR processing and translation are performed on an image whose rotation angle has been adjusted. FIG. 7B illustrates a case where the center position of the display unit 11 is set as the center position of image rotation, as in the first embodiment. Also in FIG. 7B, as in FIG. 3D, all the English characters included in the image are subjected to OCR processing, and the translation result is superimposed on the English characters. 11 is displayed.
　従って、ユーザは、正立した画像を撮影することができなかった場合でも、表示部１１に表示された翻訳結果を確認しながら、画像の回転角度を手動で調整することによって、適切な翻訳結果を得ることができる。スマートフォン２では、１回の操作当たりの画像の回転角度をユーザが容易に調整できるので、ユーザの操作の利便性が向上する。 Therefore, even when the user cannot take an upright image, the user can adjust the rotation angle of the image manually while checking the translation result displayed on the display unit 11 to obtain an appropriate translation result. Can be obtained. In the smartphone 2, since the user can easily adjust the rotation angle of the image per operation, the convenience of the user's operation is improved.
　また、画像の回転の中心位置として、表示部１１の中心位置が設定される場合には、ユーザが入力部１２のどの位置において２本の指を回転させても、表示部１１の中央付近の位置に翻訳結果を表示させることができるので、翻訳結果の視認性が向上する。 Further, when the center position of the display unit 11 is set as the center position of the image rotation, no matter which position of the input unit 12 the user rotates two fingers, Since the translation result can be displayed at the position, the visibility of the translation result is improved.
　また、画像の回転の中心位置として、２つの入力位置の中点が設定される場合には、ユーザが表示部１１の所望の位置に、翻訳結果を表示させることができるので、表示部１１の画面レイアウトの自由度が向上する。 Further, when the midpoint between the two input positions is set as the center position of the image rotation, the user can display the translation result at a desired position on the display unit 11. The degree of freedom of screen layout is improved.
　なお、上述の実施形態１および２では、情報機器の一例として、タッチパネル１０を内蔵したスマートフォン１および２を例示しているが、本発明は、タッチパネル１０を入力装置として動作する他の情報機器（例えばＰＣ）についても適用可能である。従って、タッチパネル１０は情報機器に外付けされた外部の入力装置であってもよい。撮影部３０もまた、情報機器に内蔵されている必要はなく、情報機器に外付けされていてもよい。 In the first and second embodiments described above, the smartphones 1 and 2 including the touch panel 10 are illustrated as an example of the information device, but the present invention is another information device that operates using the touch panel 10 as an input device ( For example, it can also be applied to a PC). Therefore, the touch panel 10 may be an external input device externally attached to the information device. The imaging unit 30 does not need to be built in the information device, and may be externally attached to the information device.
　〔実施形態３〕
　スマートフォン１および２の制御ブロック（特に主制御部２０および２６）は、集積回路（ＩＣチップ）等に形成された論理回路（ハードウェア）によって実現してもよいし、ＣＰＵ（Central Processing Unit）を用いてソフトウェアによって実現してもよい。 [Embodiment 3]
The control blocks (especially the main control units 20 and 26) of the smartphones 1 and 2 may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or a CPU (Central Processing Unit) And may be realized by software.
　後者の場合、スマートフォン１および２は、各機能を実現するソフトウェアであるプログラムの命令を実行するＣＰＵ、上記プログラムおよび各種データがコンピュータ（またはＣＰＵ）で読み取り可能に記録されたＲＯＭ（Read Only Memory）または記憶装置（これらを「記録媒体」と称する）、上記プログラムを展開するＲＡＭ（Random Access Memory）等を備える。そして、コンピュータ（またはＣＰＵ）が上記プログラムを上記記録媒体から読み取って実行することにより、本発明の目的が達成される。上記記録媒体としては、「一時的でない有形の媒体」、例えば、テープ、ディスク、カード、半導体メモリ、プログラマブルな論理回路等を用いることができる。また、上記プログラムは、該プログラムを伝送可能な任意の伝送媒体（通信ネットワークや放送波等）を介して上記コンピュータに供給されてもよい。なお、本発明は、上記プログラムが電子的な伝送によって具現化された、搬送波に埋め込まれたデータ信号の形態でも実現され得る。 In the latter case, the smartphones 1 and 2 include a CPU that executes instructions of a program that is software that realizes each function, and a ROM (Read Only Memory) in which the program and various data are recorded so as to be readable by the computer (or CPU). Alternatively, a storage device (these are referred to as “recording media”), a RAM (Random Access Memory) for expanding the program, and the like are provided. And the objective of this invention is achieved when a computer (or CPU) reads the said program from the said recording medium and runs it. As the recording medium, a “non-temporary tangible medium” such as a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. The program may be supplied to the computer via an arbitrary transmission medium (such as a communication network or a broadcast wave) that can transmit the program. The present invention can also be realized in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.
　〔まとめ〕
　本発明の態様１に係る情報機器（スマートフォン１）は、翻訳対象となる文字を含む第１対象画像を取得し、上記第１対象画像に含まれる文字を翻訳する情報機器であって、ユーザの入力操作に応じて上記第１対象画像を回転させた第２対象画像を生成する画像回転手段（角度補正部２３）と、上記第２対象画像を文字認識処理の対象として、上記第２対象画像に含まれる文字を認識する文字認識手段（ＯＣＲ処理部２４）と、上記文字認識手段において認識に成功した文字を翻訳対象として翻訳結果を生成し、上記第２対象画像に上記翻訳結果を上記翻訳対象に重畳させた表示用画像を生成する翻訳処理手段（翻訳処理部２５）と、を備えている。 [Summary]
An information device (smart phone 1) according to aspect 1 of the present invention is an information device that acquires a first target image including characters to be translated and translates characters included in the first target image. Image rotating means (angle correction unit 23) for generating a second target image obtained by rotating the first target image in response to an input operation, and the second target image as a target for character recognition processing. A character recognition unit (OCR processing unit 24) for recognizing a character included in the character, and a translation result is generated with the character recognized by the character recognition unit as a translation target, and the translation result is translated into the second target image. Translation processing means (translation processing unit 25) for generating a display image superimposed on the object.
　上記の構成によれば、ユーザは、被写体の撮影条件が限られている等の理由により、高精度な文字認識処理が行われない画像（すなわち、正立しておらず傾いている画像）が第１対象画像として得られた場合でも、自身の入力操作によって対象画像の回転角度を手動で調整することにより、文字認識処理、および後続の翻訳処理が適切に行われる第２対象画像（すなわち、概ね正立している画像）を得ることができる。 According to the above-described configuration, the user does not perform high-accuracy character recognition processing (i.e., an image that is not upright and inclined) due to reasons such as limited shooting conditions of the subject. Even when obtained as the first target image, the second target image (ie, the character recognition process and the subsequent translation process are appropriately performed by manually adjusting the rotation angle of the target image by its own input operation (that is, A generally upright image) can be obtained.
　従って、ユーザは、翻訳処理手段において生成された表示用画像を表示装置（例えばスマートフォンのタッチパネル）によって確認しながら上述の調整操作を行うことができ、文字認識処理、および後続の翻訳処理に好適な第２対象画像を得ることができる。 Therefore, the user can perform the above-described adjustment operation while confirming the display image generated in the translation processing means with a display device (for example, a touch panel of a smartphone), which is suitable for character recognition processing and subsequent translation processing. A second target image can be obtained.
　また、本発明の態様２に係る情報機器は、上記態様１において、ユーザの入力操作を受け付けるタッチパネル（１０）と、ユーザが上記第１対象画像を回転させるための入力操作を行う対象である回転ボタンを、上記タッチパネルに表示させる回転ボタン表示手段（角度補正部２３）をさらに備え、上記画像回転手段は、ユーザが上記回転ボタンを１回タップするごとに、上記タッチパネルの所定の位置を回転中心位置として、所定の角度だけ、上記第１対象画像を所定の方向に回転させて上記第２対象画像を生成してもよい。 Moreover, the information apparatus which concerns on aspect 2 of this invention is the rotation which is the object which performs the input operation for the touch panel (10) which receives a user's input operation in the said aspect 1, and a user to rotate the said 1st target image. Rotation button display means (angle correction unit 23) for displaying buttons on the touch panel, and the image rotation means rotates a predetermined position on the touch panel every time the user taps the rotation button once. As the position, the second target image may be generated by rotating the first target image in a predetermined direction by a predetermined angle.
　上記の構成によれば、ユーザは、タッチパネルに表示された回転ボタンをタップし、所定の角度（例えば１°）ずつ第１対象画像を回転させながら第２対象画像を得ることができるので、画像の回転角度を調整するためのユーザによる操作の利便性を向上させることができる。 According to the above configuration, the user can tap the rotation button displayed on the touch panel and obtain the second target image while rotating the first target image by a predetermined angle (for example, 1 °). The convenience of the operation by the user for adjusting the rotation angle can be improved.
　また、本発明の態様３に係る情報機器は、上記態様２において、上記画像回転手段は、ユーザが上記タッチパネル上において、２本の指を所定の方向に回転させたとき、上記タッチパネルの所定の位置を回転中心位置として、上記２本の指の回転角度に比例した角度だけ、上記第１対象画像を所定の方向に回転させて上記第２対象画像を生成してもよい。 The information device according to aspect 3 of the present invention is the information apparatus according to aspect 2, in which the image rotation unit is configured to perform a predetermined operation on the touch panel when the user rotates two fingers on the touch panel in a predetermined direction. The second target image may be generated by rotating the first target image in a predetermined direction by an angle proportional to the rotation angle of the two fingers with the position as the rotation center position.
　上記の構成によれば、ユーザが１回の操作当たりの画像の回転角度を容易に調整できるので、ユーザによる操作の利便性を向上させることができる。また、ユーザがタッチパネル上のどの位置において２本の指を回転させても、タッチパネルの所定の位置（例えば、中央付近の位置）に翻訳結果が含まれた表示用画像を表示させることができるので、翻訳結果の視認性を向上させることができる。 According to the above configuration, since the user can easily adjust the rotation angle of the image per operation, the convenience of operation by the user can be improved. In addition, even if the user rotates two fingers at any position on the touch panel, a display image including the translation result can be displayed at a predetermined position on the touch panel (for example, a position near the center). The visibility of the translation result can be improved.
　また、本発明の態様４に係る情報機器は、上記態様２において、ユーザが上記タッチパネル上に２本の指を置いた時点における、上記２本の指の位置にそれぞれ対応する２つの入力位置の中点を算出する回転中心算出手段（回転中心設定部２７）をさらに備え、上記画像回転手段は、ユーザが上記タッチパネル上において、上記２本の指を所定の方向に回転させたとき、上記中点を回転中心位置として、上記２本の指の回転角度に比例した角度だけ、上記第１対象画像を所定の方向に回転させて上記第２対象画像を生成してもよい。 The information device according to aspect 4 of the present invention is the information apparatus according to aspect 2, in which the two input positions respectively corresponding to the positions of the two fingers at the time when the user places the two fingers on the touch panel. Rotation center calculation means (rotation center setting unit 27) for calculating a midpoint is further provided, and the image rotation means is configured such that when the user rotates the two fingers in a predetermined direction on the touch panel, The second target image may be generated by rotating the first target image in a predetermined direction by an angle proportional to the rotation angle of the two fingers with the point as the rotation center position.
　上記の構成によれば、上述の態様３と同様に、ユーザが１回の操作当たりの画像の回転角度を容易に調整できるので、ユーザによる操作の利便性を向上させることができる。また、２つの入力位置の中点を画像の回転の中心位置とすることにより、ユーザが表示部１１の所望の位置に、翻訳結果が含まれた表示用画像を表示させることができるので、タッチパネルの画面レイアウトの自由度を向上させることができる。 According to the above configuration, similarly to the above-described aspect 3, since the user can easily adjust the rotation angle of the image per operation, the convenience of the operation by the user can be improved. Further, by setting the middle point of the two input positions as the center position of the rotation of the image, the user can display a display image including the translation result at a desired position of the display unit 11. The degree of freedom of the screen layout can be improved.
　また、本発明の各態様に係る情報機器は、コンピュータによって実現してもよく、この場合には、コンピュータを上記情報機器が備える各手段として動作させることにより上記情報機器をコンピュータにて実現させる情報機器の制御プログラム、およびそれを記録したコンピュータ読み取り可能な記録媒体も、本発明の範疇に入る。 In addition, the information device according to each aspect of the present invention may be realized by a computer, and in this case, information for realizing the information device by the computer by causing the computer to operate as each unit included in the information device. An apparatus control program and a computer-readable recording medium on which the apparatus control program is recorded also fall within the scope of the present invention.
　なお、本発明は上述した各実施形態に限定されるものではなく、請求項に示した範囲で種々の変更が可能であり、異なる実施形態にそれぞれ開示された技術的手段を適宜組み合わせて得られる実施形態も本発明の技術的範囲に含まれる。さらに、各実施形態にそれぞれ開示された技術的手段を組み合わせることにより、新しい技術的特徴を形成できる。 The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and can be obtained by appropriately combining technical means disclosed in different embodiments. Embodiments are also included in the technical scope of the present invention. Furthermore, a new technical feature can be formed by combining the technical means disclosed in each embodiment.
　本発明は、情報機器に利用することができる。 The present invention can be used for information equipment.
　１，２　スマートフォン（情報機器）、１０　タッチパネル
　２３　角度補正部（画像回転手段、回転ボタン表示手段）
　２４　ＯＣＲ処理部（文字認識手段）
　２５　翻訳処理部（翻訳処理手段）、２７　回転中心設定部（回転中心算出手段）
　２８　角度補正部（画像回転手段） 1, 2 Smartphone (information device), 10 Touch panel 23 Angle correction unit (image rotation means, rotation button display means)
24 OCR processing part (character recognition means)
25 translation processing unit (translation processing unit), 27 rotation center setting unit (rotation center calculation unit)
28 Angle correction unit (image rotation means)

Claims (5)

  1. 　翻訳対象となる文字を含む第１対象画像を取得し、上記第１対象画像に含まれる文字を翻訳する情報機器であって、
    　ユーザの入力操作に応じて上記第１対象画像を回転させた第２対象画像を生成する画像回転手段と、
    　上記第２対象画像を文字認識処理の対象として、上記第２対象画像に含まれる文字を認識する文字認識手段と、
    　上記文字認識手段において認識に成功した文字を翻訳対象として翻訳結果を生成し、上記第２対象画像に上記翻訳結果を上記翻訳対象に重畳させた表示用画像を生成する翻訳処理手段と、を備えていることを特徴とする情報機器。     An information device that acquires a first target image including characters to be translated and translates characters included in the first target image,
    Image rotating means for generating a second target image obtained by rotating the first target image according to a user input operation;
    Character recognition means for recognizing characters included in the second target image, with the second target image as a target of character recognition processing;
    A translation processing unit that generates a translation result for a character that has been successfully recognized by the character recognition unit, and that generates a display image in which the translation result is superimposed on the translation target on the second target image. Information equipment characterized by
  2. 　ユーザの入力操作を受け付けるタッチパネルと、
    　ユーザが上記第１対象画像を回転させるための入力操作を行う対象である回転ボタンを、上記タッチパネルに表示させる回転ボタン表示手段をさらに備え、
    　上記画像回転手段は、ユーザが上記回転ボタンを１回タップするごとに、上記タッチパネルの所定の位置を回転中心位置として、所定の角度だけ、上記第１対象画像を所定の方向に回転させて上記第２対象画像を生成することを特徴とする請求項１に記載の情報機器。     A touch panel that accepts user input operations;
    Rotation button display means for displaying on the touch panel a rotation button that is a target for the user to perform an input operation for rotating the first target image,
    The image rotation means rotates the first target image in a predetermined direction by a predetermined angle with a predetermined position of the touch panel as a rotation center position every time the user taps the rotation button once. The information device according to claim 1, wherein a second target image is generated.
  3. 　上記画像回転手段は、ユーザが上記タッチパネル上において、２本の指を所定の方向に回転させたとき、上記タッチパネルの所定の位置を回転中心位置として、上記２本の指の回転角度に比例した角度だけ、上記第１対象画像を所定の方向に回転させて上記第２対象画像を生成することを特徴とする請求項２に記載の情報機器。 The image rotation means is proportional to the rotation angle of the two fingers with the predetermined position of the touch panel as a rotation center position when the user rotates two fingers on the touch panel in a predetermined direction. The information device according to claim 2, wherein the second target image is generated by rotating the first target image in a predetermined direction by an angle.
  4. 　ユーザが上記タッチパネル上に２本の指を置いた時点における、上記２本の指の位置にそれぞれ対応する２つの入力位置の中点を算出する回転中心算出手段をさらに備え、
    　上記画像回転手段は、ユーザが上記タッチパネル上において、上記２本の指を所定の方向に回転させたとき、上記中点を回転中心位置として、上記２本の指の回転角度に比例した角度だけ、上記第１対象画像を所定の方向に回転させて上記第２対象画像を生成することを特徴とする請求項２に記載の情報機器。     A rotation center calculating means for calculating a midpoint between the two input positions respectively corresponding to the positions of the two fingers when the user places the two fingers on the touch panel;
    When the user rotates the two fingers in a predetermined direction on the touch panel, the image rotation means has an angle proportional to the rotation angle of the two fingers with the center point as the rotation center position. The information device according to claim 2, wherein the second target image is generated by rotating the first target image in a predetermined direction.
  5. 　請求項１から４のいずれか１項に記載の情報機器としてコンピュータを機能させるための制御プログラムであって、コンピュータを上記各手段として機能させるための制御プログラム。 A control program for causing a computer to function as the information device according to any one of claims 1 to 4, wherein the control program causes the computer to function as each of the above means.
PCT/JP2014/071715 2013-09-25 2014-08-20 Information device and control program WO2015045679A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2013198796A JP2015064781A (en) 2013-09-25 2013-09-25 Information equipment and control program
JP2013-198796 2013-09-25

Publications (1)

Publication Number Publication Date
WO2015045679A1 true WO2015045679A1 (en) 2015-04-02

Family

ID=52742833

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2014/071715 WO2015045679A1 (en) 2013-09-25 2014-08-20 Information device and control program

Country Status (2)

Country Link
JP (1) JP2015064781A (en)
WO (1) WO2015045679A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110991445A (en) * 2019-11-21 2020-04-10 北京达佳互联信息技术有限公司 Method, device, equipment and medium for identifying vertically arranged characters
CN111491057A (en) * 2019-01-25 2020-08-04 上海爱优威软件开发有限公司 Call auxiliary method and system

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7141182B2 (en) * 2017-12-28 2022-09-22 コニカミノルタ株式会社 Radiation image display device, image display method and control program

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006146454A (en) * 2004-11-18 2006-06-08 Sony Corp Information conversion device and method
JP2010286917A (en) * 2009-06-09 2010-12-24 Sharp Corp Image processor, image scanner, multifunction apparatus, image processing method, program, and recording medium

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006146454A (en) * 2004-11-18 2006-06-08 Sony Corp Information conversion device and method
JP2010286917A (en) * 2009-06-09 2010-12-24 Sharp Corp Image processor, image scanner, multifunction apparatus, image processing method, program, and recording medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
WINDOWS 8 ET AL.: "Windows no Joshiki ga Kawatta!", SUBETE WAKARU!, vol. 17, no. 18, 24 October 2012 (2012-10-24), pages 58 - 59 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111491057A (en) * 2019-01-25 2020-08-04 上海爱优威软件开发有限公司 Call auxiliary method and system
CN110991445A (en) * 2019-11-21 2020-04-10 北京达佳互联信息技术有限公司 Method, device, equipment and medium for identifying vertically arranged characters
CN110991445B (en) * 2019-11-21 2023-09-29 北京达佳互联信息技术有限公司 Vertical text recognition method, device, equipment and medium

Also Published As

Publication number Publication date
JP2015064781A (en) 2015-04-09

Similar Documents

Publication Publication Date Title
US10922862B2 (en) Presentation of content on headset display based on one or more condition(s)
US10317777B2 (en) Automatic zooming method and apparatus
US10552946B2 (en) Display control apparatus and method for controlling the same based on orientation
US20180213147A1 (en) Information processing apparatus having camera function and producing guide display to capture character recognizable image, control method thereof, and storage medium
KR20160027862A (en) Method for processing image data and electronic device supporting thereof
JP6206581B2 (en) Terminal device, display control method, and program
US10403238B2 (en) Presentation of representations of input with contours having a width based on the size of the input
US20140009385A1 (en) Method and system for rotating display image
CN113873148A (en) Video recording method, video recording device, electronic equipment and readable storage medium
US11770603B2 (en) Image display method having visual effect of increasing size of target image, mobile terminal, and computer-readable storage medium
JP6170241B2 (en) Character identification device and control program
WO2015045679A1 (en) Information device and control program
US20150347364A1 (en) Highlighting input area based on user input
JP6164361B2 (en) Terminal device, display control method, and program
JP6206580B2 (en) Terminal device, display control method, and program
US20130236117A1 (en) Apparatus and method for providing blurred image
CN108604128B (en) Processing method and mobile device
US20160148342A1 (en) Movement of displayed element from one display to another
US9860452B2 (en) Usage of first camera to determine parameter for action associated with second camera
US9817490B2 (en) Presenting user interface based on location of input from body part
JP2015032261A (en) Display device and control method
US10902265B2 (en) Imaging effect based on object depth information
JP6251075B2 (en) Translation device
US20200126517A1 (en) Image adjustment method, apparatus, device and computer readable storage medium
JP2016111521A (en) Information processing device, information processing program and information processing method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14849512

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14849512

Country of ref document: EP

Kind code of ref document: A1