CN114791784A - Blind person auxiliary image browsing method and system - Google Patents
Blind person auxiliary image browsing method and system Download PDFInfo
- Publication number
- CN114791784A CN114791784A CN202210349196.XA CN202210349196A CN114791784A CN 114791784 A CN114791784 A CN 114791784A CN 202210349196 A CN202210349196 A CN 202210349196A CN 114791784 A CN114791784 A CN 114791784A
- Authority
- CN
- China
- Prior art keywords
- image
- user
- rgb color
- browsing
- converting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000006243 chemical reaction Methods 0.000 claims abstract description 17
- 238000013507 mapping Methods 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 6
- 230000006870 function Effects 0.000 description 11
- 239000003086 colorant Substances 0.000 description 6
- 210000004556 brain Anatomy 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 239000000047 product Substances 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 108091008702 infrared receptors Proteins 0.000 description 4
- 230000001960 triggered effect Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000003796 beauty Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 239000006227 byproduct Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000033764 rhythmic process Effects 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 230000015541 sensory perception of touch Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04812—Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04842—Selection of displayed objects or displayed text elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/165—Management of the audio stream, e.g. setting of volume, audio stream path
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/24—Indexing scheme for image data processing or generation, in general involving graphical user interfaces [GUIs]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Position Input By Displaying (AREA)
Abstract
A blind-aided image browsing method and system, the method comprising: acquiring a browsing position of a user on an image and an RGB color value of the browsing position; converting the obtained RGB color values into corresponding sound information; and playing the corresponding sound information. The system comprises: the information acquisition module is used for acquiring a browsing position of a user on an image and an RGB color value of the browsing position; the conversion module is used for converting the acquired RGB color values into corresponding sound information; and the audio playing module is used for playing the corresponding sound information. The method and the system can convert the color information in the image into the sound information which can be captured by the blind person, help the blind person to realize the browsing and the identification of the image, do not need special equipment, reduce the cost and the operation complexity and improve the portability.
Description
Technical Field
The invention relates to the technical field of computer and blind person assistant reading, in particular to a blind person assistant image browsing method and a blind person assistant image browsing system.
Background
Aiming at the special requirement of blind people for reading, auxiliary reading is provided mainly by products for converting vision into touch sense and auditory sense. In addition to braille, for example, there is a tactile graphic display terminal that allows a blind person to perceive an image, the surface of which is composed of dots that can be raised and retracted, a conventional picture is changed into a graphic that can be touched by controlling the change of these dots by a computer, and a blind person user reads characters and recognizes a picture by touching braille or a tactile graphic composed of these raised dots. In the aspect of hearing, due to the development and popularization of electronic products, blind users can acquire information through sound by means of screen reading software in the electronic products.
A known prior art discloses a text detection and recognition method for blind person assisted reading, which detects the position of a finger on a reading text by tracking with a camera, extracts the text, and converts the text into a voice output to assist the blind person user in reading. In addition, the blind person assistant reading ring is provided, the ring collects information and transmits the collected information to an upper computer in communication connection with the ring, and the upper computer converts the collected text language into sound information and reads the sound information to a blind person user.
Disclosure of Invention
Technical problem
However, the above prior art has many drawbacks. For example, a complex system composed of various devices such as an image acquisition device and various functional modules is required to realize the functions of scene detection, finger positioning, text extraction, sound conversion and the like, and basically only text information can be extracted, and image information such as shapes, lines and the like appearing in the reading process cannot be extracted and transmitted, so that not only are the operation complex and the cost high due to a large amount of hardware, but also the extracted information is only limited to characters and cannot process the image information. In addition, although the tactile graphic display terminal in the prior art can assist the blind users in identifying images, the tactile graphic display terminal also has the problems of high equipment cost and low portability, and the users cannot sense the color intensity information only through the tactile sense.
Means for solving the problems
In view of the above problems, the present invention provides a blind-person-assisted image browsing method capable of assisting a blind-person user in browsing the whole information of images in an electronic product without requiring special equipment and capable of reducing the complexity of operation.
According to one aspect of the invention, a blind-person-assisted image browsing method is provided, which comprises the following steps:
acquiring a browsing position of a user on an image and an RGB color value of the browsing position;
converting the obtained RGB color values into corresponding sound information; and
and playing the corresponding sound information.
The technical scheme has the advantages that the color information in the image is converted into the sound information which can be captured by the blind, so that the blind can abstract corresponding color distribution in the brain through sound, the color information in the image is sensed, and the image browsing and recognition are realized.
In some preferred embodiments, converting the obtained RGB color values into corresponding sound information specifically includes: converting the obtained RGB color values into gray values; and converting the gray value into corresponding sound frequency to obtain the corresponding sound information.
The technical scheme has the advantages that the color can be converted into the gray value which has less information content compared with RGB (red, green and blue), and then the gray value is converted into the corresponding sound frequency, namely the scale, so that the information needing to be memorized is reduced, the memory burden of the blind user is relieved, the blind user can receive the corresponding scale in the sense of hearing after touching the color and abstract the corresponding gray color in the brain, and then the blind user can feel the light and shade change of the color to further perceive the graphic image.
In some preferred embodiments, converting the acquired RGB color values into gray scale values specifically includes calculating gray scale values corresponding to the RGB color values by using the following formula:
(R×30+G×59+B×100)/100(1)
where R represents a color intensity value of red, G represents a color intensity value of green, and B represents a color intensity value of blue.
The technical scheme has the advantages that the color values are converted into the gray values with less information carrying capacity, the processing complexity of a system and a human brain is reduced, the colors and the intensity of the colors are distinguished as far as possible, and the information capacity is enriched.
In some preferred embodiments, converting the gray value into the corresponding sound frequency specifically includes: and establishing a mapping relation between the gray values and the sound frequencies, so that each gray value corresponds to a unique sound frequency. In some preferred embodiments, the sound frequency increases with increasing gray scale value. And in some preferred embodiments, the gray scale value ranges from 0 to 255, when the gray scale value is 0, the corresponding sound frequency is 150Hz, and the corresponding sound frequency is increased by 15Hz for each increase of 1 in the gray scale value.
The technical scheme has the beneficial effects that the one-to-one corresponding relation between the gray value and the scale is established, for example, the gradual increase of the gray value corresponds to the gradual increase of the sound frequency, so that a blind user can obtain the perception of color gray, and the light and shade change of the color is felt only through the scale.
In some preferred embodiments, the acquiring the browsing position of the user on the image and the RGB color value of the browsing position specifically includes: displaying the image on an electronic device with a touch screen, so that the user can browse the image by touching the touch screen with a hand, acquiring touch point coordinates of the touch screen touched by the user through the touch screen, and acquiring RGB color values of the touch point coordinates through a system API.
The technical scheme has the advantages that the touch screen of the existing electronic equipment is utilized to realize image display and user browsing information, namely the touch point coordinates of the browsing position and the RGB color information of the touch point coordinates, so that special equipment is not needed, the cost and the operation complexity are reduced, and the portability is improved.
In some preferred embodiments, the acquiring the browsing position of the user on the image and the RGB color value of the browsing position specifically includes: displaying the image on an electronic device with an electronic screen, enabling the user to browse the image on the electronic screen by means of cursor movement by operating a mouse, and acquiring cursor coordinates of the cursor in the electronic screen and RGB color values of the cursor coordinates through a system API.
The technical scheme has the advantages that the electronic screen of the conventional electronic equipment is utilized to realize the display of the image and the acquisition of the browsing information of the user, namely the cursor coordinate of the browsing position and the RGB color information of the cursor coordinate, so that special equipment is not needed, and the cost and the operation complexity are reduced.
In some preferred embodiments, the electronic device has a camera or an infrared receptor, and when the camera or the infrared receptor is blocked during the use of the electronic device, the display content of the electronic screen is locked, so that the browsing position of the user on the image and the RGB color value of the browsing position can be acquired based on the operation of the user.
The technical scheme has the advantages that for example, a camera or an infrared receptor equipped for electronic equipment such as a smart phone is only required to be blocked by hands, or a camera carried by electronic equipment such as a personal computer and the like can be used for locking the display content and the functions of a screen, and rolling and clicking events can not be triggered, so that the auxiliary image browsing function according to the aspect can be started simply and conveniently, the operation complexity is reduced, and meanwhile, special equipment is not required.
In another aspect of the present invention, there is provided a blind-person-assisted image browsing system, including:
the information acquisition module is used for acquiring a browsing position of a user on an image based on user operation and acquiring RGB color values of the browsing position;
the conversion module is used for converting the acquired RGB color values into corresponding sound information; and
and the audio playing module is used for playing the corresponding sound information.
In some preferred embodiments, the conversion module comprises: the color conversion module is used for converting the acquired RGB color values into gray values; and the audio conversion module is used for converting the gray value into corresponding sound frequency so as to obtain the corresponding sound information.
In some preferred embodiments, the information acquisition module includes a touch screen, and the image is displayed on the touch screen, so that the user can browse the image by touching the touch screen with a hand, and touch point coordinates of the touch screen touched by the user are acquired through the touch screen, and RGB color values of pixels of the touch point coordinates are acquired through a system API.
In some preferred embodiments, the information acquisition module includes an electronic screen, the image is displayed on the electronic screen, so that the user can browse the image on the electronic screen by means of cursor movement by operating a mouse, and cursor coordinates of the cursor in the electronic screen and RGB color values of the cursor coordinates are acquired through a system API.
According to still another aspect of the present invention, there is provided an electronic apparatus including:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement any one of the methods described above in accordance with an aspect of the present invention.
According to a further aspect of the invention, there is provided a computer readable medium having stored thereon a computer program which, when executed by a processor, carries out any of the methods according to one aspect of the invention described above.
Advantageous effects of the invention
The method and the system can convert the color information in the image into the sound information which can be captured by the blind person, help the blind person to realize the browsing and the identification of the image, do not need special equipment, reduce the cost and the operation complexity and improve the portability.
Drawings
Fig. 1 is a flowchart showing a main flow of a blind-person-aided image browsing method according to a preferred embodiment of the present invention;
FIG. 2 is an operation flowchart showing an application example of the blind-person assisted image browsing method according to a preferred embodiment of the present invention;
fig. 3 is a flowchart illustrating a touch reading process performed on a touch action of a user in a blind-person-assisted image browsing method according to a preferred embodiment of the present invention;
fig. 4 is a flowchart showing a specific example of converting color information into sound information in the blind-person-aided image browsing method according to a preferred embodiment of the present invention; and
fig. 5 is a block diagram showing a blind-person-assisted image browsing system according to a preferred embodiment of the present invention.
Detailed Description
The technical solution of the present invention will be clearly and completely described below with reference to the specific embodiments of the present invention and the accompanying drawings. It is to be understood that the described embodiments are only a few of the presently preferred embodiments of the invention, and not all embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.
The color is the classification of the spatially distributed electromagnetic waves by the brain according to the wavelength, for example, red represents the classification of the spatially distributed electromagnetic waves with the wavelength of about 650nm, and the intensity of red corresponds to the amplitude of the red electromagnetic waves, so that the information carried in the color is very rich. Through the current auxiliary reading products, the information which can be acquired by the blind people is greatly different from the information amount captured by the actual vision, and especially in the aspect of images, the blind people cannot perceive the color and the strength of the color through the existing auxiliary reading products, so that much information is lost when the images are identified. Therefore, it is very meaningful to assist the blind in obtaining color information. The invention converts the color into the sound information, for example, firstly converts the color into the gray color, then converts the gray color into the information form which can be captured by the blind, namely the scale, and utilizes the one-to-one corresponding relation of the gray color and the scale to ensure that the blind user listens to the sound of the corresponding scale after touching the color, thereby helping the blind user abstract the corresponding gray color in the brain through continuous or frequent sound feedback and further feeling the light and shade change of the color, so that the user can further distinguish the graph and obtain the integral information of the image.
As shown in fig. 1, the blind-person-assisted image browsing method of the present invention mainly comprises: step S1: acquiring a browsing position of a user on an image and an RGB color value of the browsing position; step S2: converting the obtained RGB color values into corresponding sound information; and step S3: and playing the corresponding sound information.
The steps are described in detail below with reference to practical examples.
The blind-person-assisted image browsing method of the embodiment can be applied to electronic equipment such as a smart phone and a personal computer, and the method of the embodiment can be integrated into the electronic equipment in the form of a browser plug-in, so that the blind-person-assisted image browsing method is simple to install, low in cost and easy to use. Taking a smart phone as an example, an image is displayed on a touch screen of the smart phone, and with the method of the embodiment, a user can browse the image by touching the screen with a finger.
As shown in fig. 2, the mobile phone system enters the auxiliary browsing mode to operate the blind auxiliary image browsing method of the present embodiment. After entering the auxiliary browsing mode, the mobile phone system starts to monitor the on/off state of the touch reading switch in real time, and sets the function of the screen according to the on/off state of the touch reading switch. The touch reading switch can be any preset operation which can be realized by the blind user, such as blocking a camera at the top of a screen or an infrared receptor area of a mobile phone by hands, or touching a specific part of the mobile phone for a specific number of times, and the like. In this embodiment, it is specified that the touch switch is turned on when the camera on the top of the screen is shielded, and turned off when the camera is not shielded. The mobile phone system enters a touch reading mode to execute touch reading processing after detecting the opening of the touch reading switch, namely, screen functions and display pictures are locked, so that the screen pictures cannot roll along with the movement of fingers, a click event cannot be triggered, and the screen is only used for acquiring position coordinates touched by the fingers of a user in real time and acquiring color information of the position coordinates so as to acquire sound of a corresponding scale in subsequent steps. After the touch reading is finished, the user can cancel the shielding of the camera. When the camera is not shielded, the system judges that the touch reading switch is closed, the screen is unlocked and exits from the touch reading mode at the moment, a display picture of the screen can move along with the movement of the finger and can trigger a click event in response to the touch of the finger, and meanwhile, the system judges whether to exit from the auxiliary browsing mode of the method of the embodiment, if not, the on/off state of the touch reading switch is continuously monitored; if the auxiliary browsing mode is exited, other functions are normally executed, and the on/off state of the touch reading switch is no longer detected for executing the touch reading processing. When the system detects that the camera is continuously shielded, the touch reading mode is continued to execute touch reading processing, and the system continues to monitor the on/off state of the touch reading switch. When the auxiliary browsing mode of the method of the embodiment is not entered, the system does not lock the screen function and the picture according to the shielding of the camera, and all the functions of the mobile phone can be normally used. In addition, the auxiliary browsing mode is closed by default when a call is made, so that the touch reading function is prevented from being triggered by the fact that the face touches the screen during the call making and receiving.
As shown in fig. 3, after the touch reading switch is turned on, touch reading processing is started, that is, the position where the user touches the screen is acquired in real time, color information of the position is acquired, and the color information is converted into sound information to be played to the user in real time. Specifically, a user browses an image by touching a screen with a hand, the touch of the user on the screen triggers a touch event, the system acquires touch point coordinates of the touch screen of the user through screen x and screen y in the touch event, acquires RGB color values of the touch point coordinates according to a system API (application program interface), converts the color values into corresponding sound information, and plays the sound information. In this embodiment, the color value of the contact coordinate is converted into a gray value, the gray value is converted into a corresponding sound frequency, and then the sound of the frequency is played. The user does not shield the camera after finishing touch reading of one picture by listening to sound, can roll screen pictures by moving fingers at the moment, and shield the camera when needing to browse the next picture.
The conversion process of RGB color values into sound information is described in detail below in conjunction with fig. 4. First, the gray scale value corresponding to the RGB color value is calculated by using the formula (R × 30+ G × 59+ B × 100)/100, where R represents the color intensity value of red, G represents the color intensity value of green, and B represents the color intensity value of blue. The gray scale value ranges from 0 to 255, wherein the white gray scale value is 255, and the black gray scale value is 0. And then establishing a one-to-one mapping relation between the gray value and the sound frequency, converting the gray value into different sound frequencies according to the mapping relation, and further acquiring the sound frequencies corresponding to different colors. According to the investigation and practice, 150Hz to 4000Hz is the frequency range of sound that most people can hear and accept, so in the embodiment, a mapping between the gray value in the range of 0 to 255 and the sound in the frequency range of 150Hz to 4000Hz is established.
Specifically, in the present embodiment, the mapping relationship between the gradation value and the sound frequency is established as follows: so that a gradual increase of the grey value corresponds to a gradual increase of the sound frequency. That is, the smaller the grayscale value, i.e., the lower the sound frequency corresponding to the darker pixel (contact coordinates), the larger the grayscale value, i.e., the higher the sound frequency corresponding to the brighter pixel. The 0 gray scale value corresponds to the sound frequency of 150Hz, the interval is the gray scale value of 1, and the interval corresponding to the sound frequency is 15 Hz.
By using the above conversion formula and the established mapping, any color value can be converted into sound frequency. For example, when the touch point coordinates of the screen are red of R238, G18, B18, the RGB color values are converted into a gray value of 84 by the above formula, then the gray value 84 is converted into a sound frequency of 1410Hz according to the above mapping, and the sound of 1410Hz is played. When the user continues to move the finger and browse the cyan touch point with the color values of R160, G224 and B186, the gray value of 200 is obtained through calculation, and then the sound of 3150Hz is obtained and played. By converting the color of the user touch position into sound in real time and playing the sound to the user in this manner, the user can acquire the gradation value information of the entire screen by constantly receiving the fed-back sound information, thereby recognizing the entire information of the image.
One embodiment of the method of the present invention is described above. In this embodiment, the method of the present invention is applied to a smartphone in the form of App (application software), and converts the color of a contact point where a user touches a screen into a grayscale and further into a sound frequency, so as to convert the color information into sound information and feed the sound information back to the user, thereby helping the user obtain image information.
However, the present invention is not limited thereto. The blind-person-assisted image browsing method according to the present invention can also be applied to an electronic device such as a PC device, in which case the method of the present invention is applied to a browser of the device in the form of a browser plug-in, the image is displayed on an electronic screen of the electronic device, a user browses the image by means of cursor movement in the electronic screen by operating a mouse or the like to manipulate the device, then the position coordinates of the cursor and the RGB color values of the pixels of the cursor coordinates are acquired through a system API, the color values are converted into gray values via a specific formula and mapping and then converted into corresponding sound frequencies to be fed back to the user, so as to help the user to complete browsing by sound.
In addition, the blind-person auxiliary image browsing method can also be applied to browsing of entity images. For example, a user may take a physical image using a rear camera of a mobile phone and then browse the taken picture on the mobile phone using the blind-person-assisted image browsing method according to the present invention. Or, the user can track the touch position of the shooting finger on the entity image in real time by using a rear camera of the mobile phone, the system identifies the color of the touch point of the user on the image through the corresponding contact identification function and color identification function, and then the color is converted into corresponding sound information to be fed back to the user.
In addition, the formula used for converting the color value into the gradation value in the method of the present invention is not limited to the above formula, and the method of converting the color value into the gradation value is not limited to the aforementioned method of calculation according to the formula, and any method may be used as long as various colors can be sufficiently converted into the gradation value with distinction. In addition, the method of mapping the gradation values to the frequencies of sound is also not limited to the method in the foregoing embodiment, and any mapping or any other method may be used as long as it enables the user to distinguish different gradation values from sound.
Furthermore, the conversion of colors into sound information in the method of the present invention is not limited to the conversion of colors into corresponding sound frequencies. For example, the color information may also be converted into corresponding sound information such as volume, timbre and/or rhythm according to a specific formula and/or mapping, as long as the user can be helped to distinguish the color information from the sound information.
That is, any medium may be used to implement the method according to the present invention, as long as it can acquire the browsing position of the user on the image and the color value of the browsing position, and convert the color value into corresponding sound information to be fed back to the user.
By using the method, a user does not need to pay attention to whether the finger is flat or not, whether the contact area of the finger and the picture is straight or the sliding track is attached to the reading content or not, and the like, and does not need to wear additional wearing equipment in the image browsing process, so that the user can browse the image through a low-price, portable and easy-to-use way, recognize lines and shape elements in the image, perceive the beauty of the lines, and even read the cartoon works. In addition, the method can be applied to the field of education, and helps users to learn, practice writing and drawing.
The blind-person auxiliary image browsing method according to the present invention has been described above. A blind-person-aided image browsing system according to an embodiment of the present invention that implements the above method is described below with reference to fig. 5.
The blind-person-assisted image browsing system according to the present invention comprises: the information acquisition module is used for acquiring a browsing position of a user on an image based on the operation of the user and acquiring RGB color values of the browsing position; the conversion module is used for converting the acquired RGB color values into corresponding sound information; and the audio playing module is used for playing the sound with the corresponding sound frequency.
In some preferred embodiments, the conversion module comprises: the color conversion module is used for converting the acquired RGB color values into gray values; and the audio conversion module is used for converting the gray value into corresponding sound frequency.
In some preferred embodiments, the information acquisition module includes a touch screen, and the image is displayed on the touch screen, so that a user can browse the image by touching the touch screen with a hand, and acquire touch point coordinates of the touch screen touched by the user through the touch screen, and acquire RGB color values of the touch point coordinates through the system API.
In some preferred embodiments, the information acquisition module includes an electronic screen on which an image is displayed, so that a user can browse the image on the electronic screen by means of cursor movement by operating a mouse, and acquire cursor coordinates of a cursor in the electronic screen and RGB color values of the cursor coordinates through a system API.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A blind-aided image browsing method is characterized by comprising the following steps:
acquiring a browsing position of a user on an image and an RGB color value of the browsing position;
converting the obtained RGB color values into corresponding sound information; and
and playing the corresponding sound information.
2. The method of claim 1, wherein converting the obtained RGB color values into corresponding sound information specifically comprises:
converting the obtained RGB color values into gray values; and is
And converting the gray value into a corresponding sound frequency to obtain the corresponding sound information.
3. The method of claim 2, wherein converting the acquired RGB color values into grayscale values specifically comprises:
calculating the gray value corresponding to the RGB color value by using the following formula:
(R×30+G×59+B×100)/100(1)
where R represents a color intensity value of red, G represents a color intensity value of green, and B represents a color intensity value of blue.
4. The method according to claim 2 or 3, wherein converting the grey value into the corresponding sound frequency specifically comprises:
and establishing a mapping relation between the gray values and the sound frequencies, so that each gray value corresponds to a unique sound frequency.
5. The method of claim 4, wherein,
the sound frequency is increased with the gradual increase of the gray scale value.
6. The method according to claim 1 or 2, wherein the obtaining of the browsing location of the user on the image and the RGB color value of the browsing location specifically comprises:
displaying the image on an electronic device with a touch screen, such that the user can browse the image by touching the touch screen with a hand, acquiring touch point coordinates of the touch screen touched by the user through the touch screen, and acquiring RGB color values of pixels of the touch point coordinates through a system API, or
Displaying the image on an electronic device with an electronic screen, enabling the user to browse the image on the electronic screen by means of cursor movement by operating a mouse, and acquiring cursor coordinates of the cursor in the electronic screen and RGB color values of pixels of the cursor coordinates through a system API.
7. The method of claim 6, wherein,
the electronic device has a camera or an infrared sensor, and
when the camera or the infrared sensor is shielded in the using process of the electronic equipment, the display content of the electronic screen is locked, so that the browsing position of the user on the image can be obtained based on the operation of the user, and the RGB color value of the browsing position can be obtained.
8. A blind-assisted image browsing system, comprising:
the information acquisition module is used for acquiring a browsing position of a user on an image based on user operation and acquiring RGB color values of the browsing position;
the conversion module is used for converting the acquired RGB color values into corresponding sound information; and
and the audio playing module is used for playing the corresponding sound information.
9. The blind-aided image browsing system of claim 8, wherein the converting module comprises:
the color conversion module is used for converting the acquired RGB color values into gray values; and
and the audio conversion module is used for converting the gray value into corresponding sound frequency so as to obtain the corresponding sound information.
10. The blind-person-aided image browsing system according to claim 8 or 9, wherein,
the information acquisition module comprises a touch screen, the image is displayed on the touch screen, so that the user can browse the image by touching the touch screen with a hand, the touch point coordinate of the touch screen touched by the user is acquired through the touch screen, and the RGB color value of the pixel of the touch point coordinate is acquired through a system API (application programming interface), or
The information acquisition module comprises an electronic screen, wherein the image is displayed on the electronic screen, so that the user can browse the image on the electronic screen by means of cursor movement through operating a mouse, and cursor coordinates of the cursor in the electronic screen and RGB color values of the cursor coordinates are acquired through a system API.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210349196.XA CN114791784A (en) | 2022-04-01 | 2022-04-01 | Blind person auxiliary image browsing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210349196.XA CN114791784A (en) | 2022-04-01 | 2022-04-01 | Blind person auxiliary image browsing method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114791784A true CN114791784A (en) | 2022-07-26 |
Family
ID=82462229
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210349196.XA Pending CN114791784A (en) | 2022-04-01 | 2022-04-01 | Blind person auxiliary image browsing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114791784A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007105927A1 (en) * | 2006-03-16 | 2007-09-20 | Harmonicolor System Co., Ltd. | Method and apparatus for converting image to sound |
| CN101889921A (en) * | 2009-05-19 | 2010-11-24 | 北京保益互动科技发展有限公司 | Method for assisting visual handicapped to identify color and system thereof |
| CN102096822A (en) * | 2010-12-24 | 2011-06-15 | 华为终端有限公司 | Color discrimination method and device |
| CN105072281A (en) * | 2015-08-27 | 2015-11-18 | 上海理工大学 | Intelligent communication terminal and method for preventing touch screen from being mistakenly touched while communicating |
| US20160267813A1 (en) * | 2014-07-01 | 2016-09-15 | King Fahd University Of Petroleum And Minerals | Color detection system and method for color detection for the visually impaired |
-
2022
- 2022-04-01 CN CN202210349196.XA patent/CN114791784A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007105927A1 (en) * | 2006-03-16 | 2007-09-20 | Harmonicolor System Co., Ltd. | Method and apparatus for converting image to sound |
| CN101889921A (en) * | 2009-05-19 | 2010-11-24 | 北京保益互动科技发展有限公司 | Method for assisting visual handicapped to identify color and system thereof |
| CN102096822A (en) * | 2010-12-24 | 2011-06-15 | 华为终端有限公司 | Color discrimination method and device |
| US20160267813A1 (en) * | 2014-07-01 | 2016-09-15 | King Fahd University Of Petroleum And Minerals | Color detection system and method for color detection for the visually impaired |
| CN105072281A (en) * | 2015-08-27 | 2015-11-18 | 上海理工大学 | Intelligent communication terminal and method for preventing touch screen from being mistakenly touched while communicating |
Non-Patent Citations (1)
| Title |
|---|
| 孙立超 等: ""一种基于嵌入式的颜色识别转换音阶方法的研究及实现"", 《哈尔滨师范大学自然科学学报》, vol. 35, no. 5, 15 October 2019 (2019-10-15), pages 39 - 44 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108520241B (en) | Fingerprint acquisition method and device based on optical fingerprint technology and user terminal | |
| US11705160B2 (en) | Method and device for processing video | |
| CN109793498B (en) | Skin detection method and electronic equipment | |
| CN108712603B (en) | Image processing method and mobile terminal | |
| CN109348135A (en) | Photographic method, device, storage medium and terminal device | |
| KR20200034183A (en) | Display apparatus and control methods thereof | |
| EP2553656A2 (en) | A computing device interface | |
| CN110706179A (en) | Image processing method and electronic equipment | |
| CN111064891B (en) | Shooting control method and electronic equipment | |
| CN110807405A (en) | Detection method of candid camera device and electronic equipment | |
| CN112817443A (en) | Display interface control method, device and equipment based on gestures and storage medium | |
| KR20130088493A (en) | Method for providing user interface and video receving apparatus thereof | |
| CN108319914A (en) | Terminal and display control method, storage medium | |
| US10412307B2 (en) | Electronic device and image display method | |
| CN117130469A (en) | Space gesture recognition method, electronic equipment and chip system | |
| CN109639981B (en) | Image shooting method and mobile terminal | |
| CN106817608A (en) | Realize the method and device of local display | |
| CN107092852A (en) | Pressure detection method and device | |
| CN104902318B (en) | Control method for playing back and terminal device | |
| EP2685355A2 (en) | Camera cursor system | |
| US20130128118A1 (en) | Smart TV with Multiple Sub-Display Windows and the Method of the Same | |
| CN103150021A (en) | Electronic book reading control system and electronic book reading control method | |
| CN114791784A (en) | Blind person auxiliary image browsing method and system | |
| EP4290338A1 (en) | Method and apparatus for inputting information, and storage medium | |
| CN110519517A (en) | Copy bootstrap technique, electronic equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |