US20140156266A1 - System and method for enhancing comprehension and readability of text - Google Patents
System and method for enhancing comprehension and readability of text Download PDFInfo
- Publication number
- US20140156266A1 US20140156266A1 US14/131,838 US201314131838A US2014156266A1 US 20140156266 A1 US20140156266 A1 US 20140156266A1 US 201314131838 A US201314131838 A US 201314131838A US 2014156266 A1 US2014156266 A1 US 2014156266A1
- Authority
- US
- United States
- Prior art keywords
- sentence
- segment
- current
- text
- break points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 230000002708 enhancing effect Effects 0.000 title 1
- 239000012634 fragment Substances 0.000 claims description 9
- 238000007781 pre-processing Methods 0.000 claims description 4
- 230000011218 segmentation Effects 0.000 abstract description 9
- 239000003086 colorant Substances 0.000 abstract 1
- 230000001427 coherent effect Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000014509 gene expression Effects 0.000 description 4
- 238000003491 array Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 201000003723 learning disability Diseases 0.000 description 2
- 230000003278 mimic effect Effects 0.000 description 2
- 238000007639 printing Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000001902 propagating effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000003339 best practice Methods 0.000 description 1
- 210000001072 colon Anatomy 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G06F17/2705—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/211—Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
- G10L13/08—Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
- G10L13/10—Prosody rules derived from text; Stress or intonation
Definitions
- the present invention relates generally to computerized text presentation, and more particularly to computerized text presentation based on text segmentation.
- typesetting practices involve a mechanical arrangement of types to be printed on paper. Such arrangement includes a selection of typefaces, type size, and spacing between letters and line. With the advent of computers and word processing software, typesetting has become mostly automated. As such, users of modern word processing software rarely adjust the typesetting beyond font selection and line spacing.
- the present invention provides a method of parsing a sentence into a sequence of phrasal segments performed by a computer processor, the method comprising the steps of:
- the parsing of the sentence may be performed based on a grammar.
- the parsing step may comprise identifying words that are conjunctions, verbs and prepositions, and the parsing of the sentence may comprise placing candidate break points at the beginning of the sentence, at punctuation marks in the sentence, and before and after each word identified as a conjunction, verb or preposition.
- the parsing step may comprise a first sub-step of pre-processing the sentence to break it into fragments based on mid-sentence punctuation marks and including candidate break points at the beginning and end of each fragment, and a second sub-step of parsing each fragment to identify candidate break points based on punctuation and parts of speech.
- Candidate break points may be removed from before and after a conjunction, verb or preposition if the conjunction, verb or preposition is situated within a phrase that should not be broken. It may be that a phrase should not be broken if it is in a pre-determined set collected from external lexicographical sources.
- the present invention also provides a method for parsing a document, the document comprising one or more than one paragraph, each paragraph comprising one or more than one sentence, into a sequence of phrasal segments performed by a computer processor, the method comprising the steps of:
- the present invention also provides a method of reading a document to a user performed by a computer processor, the method comprising the steps of:
- the duration of each pause may be selected to be discernibly longer than the average pause between words.
- the duration of each pause may be at least 500 milliseconds. Alternatively the minimum duration of a pause may be anywhere between 100 and 500 milliseconds.
- the duration of each pause, other than the pauses between sentences, may be the same. Alternatively all pauses may have the same duration, or the duration of each pause may vary based on the attributes of the preceding segment.
- the current segment may be displayed to the user while the text-to-speech engine reads the current segment.
- a sentence, being the current sentence, containing the current segment may also be displayed, and the current segment may be highlighted to differentiate it from the remainder of the current sentence.
- At least a portion of a paragraph, being the current paragraph, containing the current segment may also be displayed, and the current sentence may be highlighted to differentiate it from the remainder of the current paragraph.
- a portion of the text preceding the current paragraph in the document may be displayed before the current paragraph, and the current paragraph may be highlighted to differentiate it from the text preceding the current paragraph.
- a portion of the text following the current paragraph in the document may be displayed after the current paragraph, and the current paragraph may be highlighted to differentiate it from the text following the current paragraph.
- the current segment may be highlighted to be most prominent.
- the reading of text by the text-to-speech engine may be muted, in a teleprompter mode, so that no words are sounded.
- the invention also includes systems comprising a programmable computer processor and a display, the processor running software adapted to perform the methods described above.
- the invention also includes a non-volatile computer-readable memory on which is stored software instructions adapted to run on a computer processor to cause the processor to perform the methods described above.
- FIG. 1 a is a block diagram showing the steps of sentence segmentation.
- FIG. 1 b is a diagram showing the segmentation of a sentence by way of example.
- FIGS. 2 a , 2 b , 2 c , 2 d , 3 a and 3 b show a portion of text displayed by a processor with varying highlighting.
- FIG. 3 c a portion of text displayed by a processor with a segment of the text having been selected by a user and highlighted by the processor.
- coherent semantic units should be visually coherent as well.
- a physical highlighter to visually mark the coherently salient set of words, or alternatively, letter grouping or other font changes to increase emphasis, but the process can be laborious and depends on good reading skills.
- the other considers the use of synthesized speech output to augment the reading that emulates the visual recognition of coherent units through the listening route.
- a preferred embodiment of the invention comprises methods implemented in software that runs on a programmable computer processor.
- the invention also includes systems comprising a programmable computer processor running software adapted to perform these methods.
- FIG. 1 a illustrates the high-level steps employed to segment a document 101 into a sequence of phrasal segments 108 .
- the algorithm employs two major steps, identified as step 1 and step 2 in FIG. 1 a .
- Step 1 is the pre-processing 103 of an input document 101 to create a set of sentences 104 based on a predefined set of sentence markers. For example, a period, question mark or exclamation mark may be used to identify the end of a sentence in English.
- each sentence may be broken up into smaller fragments 104 if there are mid-sentence punctuation marks such as commas, colons, and semi-colons.
- mid-sentence punctuation marks such as commas, colons, and semi-colons.
- contiguous portions of a sentence not having any such punctuation may be identified as sentence fragments or portions.
- the second step is to further segment each sentence portion into smaller phrasal segments 108 .
- the target segment length is generally expressed as a number of words, although in some embodiments it may be expressed, for example, as a number of syllables.
- Candidate break points are placed at the beginning and end of each sentence portion.
- Candidate break points within each sentence portion may then be chosen by parsing the sentence.
- One way to do this is by analyzing each word in context to determine what part of speech it is.
- the algorithm may attempt to identify parts of speech such as conjunctions, verbs, and prepositions because these parts of speech typically mark the inclusion of new information.
- Each word may be assigned a probability that it is each part of speech (e.g. conjunction, verb, preposition, noun, adverb, etc.) and then the word may be tagged as the part of speech having the highest probability. Suitable algorithms to perform this analysis and assign such a probability are well known to skilled persons.
- a conjunction signals a new clause
- a verb signals an event or state information
- a preposition signals additional information on the subject or the verb, so that candidate break points may be placed before and after each target word tagged as conjunction, a verb, or a preposition.
- a candidate break point may be removed from before and after a conjunction, verb or preposition if the candidate break point is situated within a phrase that should not be broken.
- Such phrases may be members of a pre-determined set collected from external lexicographical sources.
- Another condition for removing a candidate break point may be that the candidate break point is too close to a sentence fragment boundary, such as by being one or two words away from the boundary.
- a more sophisticated parser may be employed based on a grammar, such as a lexical functional grammar.
- Grammatically correct sentences typically present an idea in the form of [subject] [verb phrase], where the verb phrase describes the details of an event or the state relevant to the subject. The verb phrase can be further broken down (grammatically) into smaller parts depending on the complexity of the event or state.
- Such parsing also identifies, and can be used to tag, parts of speech, but it completely breaks a sentence into its constituent phrases and sub-phrases. Many such techniques are well-known to skilled persons.
- Candidate break points may be placed before an S-BAR (a clause within a sentence), VP (verb phrase), or PP (prepositional phrase).
- a parser may not correctly handle grammatically ill-formed text and may not select a sufficient number of candidate break points so as to lead to the most appropriate selection of final break points.
- the final choosing of break points 107 is made by eliminating some of the candidate break points based on the predefined segment size 102 .
- the predefined segment size 102 is considered to be the ideal number of words in each phrasal segment 108 for purposes of reading comprehension. It may depend on information provided to the system about the human user, or reader. For example, for a child the predefined segment size 102 may be set to three, but for normal adults, it may be set to five to seven words, depending on the person's abilities. Smaller or larger segment sizes may be appropriate in some situations.
- the method employs an integer-partitioning algorithm to choose the final break points by eliminating candidate break points, thereby leaving a number of final break points, so that the sum total of the differences between the target segment size and the number of words between each consecutive pair of final break points is minimal. Two break points are considered to be consecutive if there is no other break point between those two break points
- Each sequence of words between each consecutive pair of final break points is then identified as a phrasal segment 108 .
- FIG. 1 b illustrates by way of example the segmentation of a sentence 200 having 24 words into phrasal segments 108 according to step 2 of the segmentation process depicted in FIG. 1 a , using a segment size 102 of six.
- the parts-of-speech tagging 105 assigns a tag to every word in the sentence 200 to form an augmented sentence 201 .
- Candidate break points are then placed in the sentence, as described above, to produce a preliminarily segmented sentence 202 , in this case having 11 candidate break points.
- six candidate break points are then eliminated by choosing final breakpoints 107 , leaving five final breakpoints in a finally segmented sentence 203 .
- the phrasal segments 108 are then identified as each sequence of contiguous words between each consecutive pair of final break points, thereby resulting in four phrasal segments 108 having lengths of 6, 5, 6, and 7 words.
- one solution may be selected arbitrarily, or heuristics may be used. For example, the solution with the least variance in the differences may be selected (so if one solution has differences of 0, 0, 0, and 4 , and another has differences of 1, 1, 1, and 1, the latter may be selected).
- a processor may be adapted by software to use the segmentation of the text from a document into phrasal segments for various purposes.
- a system may employ the segmentation to highlight successive segments in the document for display to a user in sequence to facilitate the user reading the document. For example, the user may desire to read the text from a document, a portion of which is shown in FIG. 2 a .
- the system may also employ a text-to-speech (TTS) engine, which are widely available.
- TTS text-to-speech
- Such a TTS engine can be instructed by the system to convert one or a sequence of words into sounds that mimic how a person would speak the words so that they can be played aloud through acoustic speakers in electronic communication with the system.
- Such a TTS engine will speak the words provided to it at a rate, which may be adjustable, that is comparable to normal human speech. Rates of 120 to 180 words per minute are typical, although lower or higher rates may be suitable for some purposes.
- the system can create a pause in the speech heard by the user.
- the system may also introduce a short pause of a pre-determined duration between phrasal segments.
- the pause between segments is generally chosen to be discernibly longer than the average pause between words, so that a normal user would recognize that the inter-segment pause is longer than the average pause or gap between other words in the sentence. It may be chosen, for example, to have a duration of 500 milliseconds, although longer pauses may be appropriate in some cases, for example between sentences. Shorter pauses may also be appropriate in some cases, such as when a relatively high rate of reading words is employed.
- Such pauses can significantly improve the understanding of a listener of the text being spoken by providing extra time for the listener to digest the content of the current segment. It should be noted that this will often result in pauses being placed between words in a sentence that are not separated by any punctuation mark, which is a key advantage of the system over the prior art.
- the system may also highlight the spoken words in the text on a display visible to the user.
- “highlight” we mean that a portion that is highlighted is visually distinguished from all other portions of text displayed.
- four classes of highlighting are used in the display so that every word on the screen is highlighted in one manner, with words in the same class being highlighted in the same manner.
- Highlighting may emphasize the highlighted text, e.g. making it the most prominent text displayed, or de-emphasize it, e.g. making it the least prominent text on the display.
- the differentiation may be achieved with boldness of the font, the colour, or the size, for example, or by techniques such as “de-focusing” text.
- all the text shown in FIG. 2 a may be displayed at one time on a computer screen.
- the system may highlight the text of the sentence [ 10 ] that is currently being spoken by rendering it in a bold font that is bolder than the other text [ 11 ] in the same paragraph.
- the remainder of the text [ 12 ] on the display, from the preceding or following paragraphs, may be shown in a faded manner so that it is less prominent that the other text [ 11 ] in the same paragraph as the current sentence [ 10 ].
- This causes the reader who is listening to the current sentence [ 10 ] being spoken to focus on that sentence, but also provides context in the form of the rest of the paragraph [ 11 ], which is highlighted less prominently, and portions of other paragraphs [ 12 ] that are highlighted even less prominently than the remainder of the paragraph [ 11 ] containing the current sentence [ 10 ].
- each phrasal segment [ 14 ] within the current sentence [ 10 ] that is currently being read/spoken by the TTS engine may be further highlighted [ 13 ] such as in the manner shown in FIG. 2 c , so that it the most prominent portion of the display.
- Such a segment continues to be highlighted as the most prominent portion of the display for as long as it takes for the TTS engine to speak the words in that segment.
- the text for the phrasal segment [ 14 ] currently being spoken is rendered in white with a surrounding black background [ 13 ].
- a pause [ 15 ] may be introduced, and then the next phrasal segment [ 17 ], as shown in FIG. 2 d , may be fed to the TTS engine for speaking, and highlighted [ 16 ] on the screen in the same prominent manner in which the prior phrasal segment [ 14 ] had been highlighted, while the highlighting of the prior phrasal segment [ 14 ] is modified so that it is shown in the same manner (with the same form of highlighting) as the remainder of the current sentence.
- the phrasal segment corresponding to the words currently being spoken by the TTS engine is always the most prominent portion of the display.
- the next sentence is then highlighted in the same manner that the previous sentence had been highlighted, and the previous sentence is then highlighted in the same manner as the remainder of the paragraph.
- the next sentence in the next paragraph
- the remainder of the new paragraph is then highlighted as the remainder of the previous paragraph had been
- the highlighting of the previous paragraph is then modified to be the least prominent.
- the system may also scroll the display from time to time, or continuously, for example to keep the line(s) containing the current segment being read in the middle of the display.
- FIGS. 2 c and 2 d four distinct levels or classes of highlighting may be used as shown in FIGS. 2 c and 2 d where (1) the current phrasal segment [ 14 ] is most prominently highlighted, (2) the remainder of the sentence being spoken is less prominently highlighted, (3) the remainder of the paragraph containing that sentence is even less prominently highlighted than the remainder of the sentence being spoken, and (4) the remainder of the text on the display in previous or following paragraphs is least prominently highlighted (e.g. by being faded or de-focused).
- the method of highlighting shown in FIGS. 2 c and 2 d is a preferred method of providing such differential highlighting, but other approaches may rely on differences in colour, brightness and font size for example. It is also not necessary that the levels of highlighting be discrete. For example, other than for the current phrasal segment, the prominence (e.g. boldness) of words could be decreased proportionately with the distance of each word from the current phrasal segment.
- the system may instead operate in a teleprompter mode, muting the sound and just highlighting each successive phrasal segment as described above.
- the TTS engine may still be employed to determine the timing so that the highlighting is moved between phrasal segments as described above in a manner suitable for the person to read the highlighted text, while pausing briefly between phrasal segments.
- the user may be provided with a control to increase or decrease the reading speed.
- the system may also be used for non-linear reading.
- the user may select a sentence such as by clicking on a word in the sentence with a mouse or using a touch screen.
- the system may then highlight the sentence [ 19 ] as shown in FIG. 3 a and then may speak the sentence [ 19 ] via the TTS engine while highlighting [ 20 ] the segment [ 21 ] currently being spoken as shown in FIG. 3 b and as described above.
- the system may further allow the user to select a segment within the sentence, for example by clicking again on a particular word, or swiping a number of words [ 22 ], as shown in FIG. 3 c .
- the system may then highlight that segment [ 24 ] using the most prominent highlighting [ 23 ] as described above.
- the system may then use the TTS engine to speak the segment.
- a computer, computer system, client or server includes one or more computer processors, and may include separate memory, and one or more input and/or output (I/O) devices (or peripherals) that are in electronic communication with the one or more processor(s).
- the electronic communication may be facilitated by, for example, one or more busses, or other wired or wireless connections.
- the processors may be tightly coupled, e.g. by high-speed busses, or loosely coupled, e.g. by being connected by a wide-area network.
- a computer processor is a hardware device for performing digital computations.
- a programmable processor is adapted to execute software, which is typically stored in a computer-readable memory.
- Processors are generally semiconductor based microprocessors, in the form of microchips or chip sets. Processors may alternatively be completely implemented in hardware, with hard-wired functionality, or in a hybrid device, such as field-programmable gate arrays or programmable logic arrays. Processors may be general-purpose or special-purpose off-the-shelf commercial products, or customized application-specific integrated circuits (ASICs). Unless otherwise stated, or required in the context, any reference to software running on a programmable processor shall be understood to include purpose-built hardware that implements all the stated software functions completely in hardware.
- Computer-readable memory which may also be referred to as a computer-readable medium or a computer-readable storage medium, which terms have identical meanings herein, can include any one or a combination of non-transitory, tangible memory elements, such as random access memory (RAM), which may be DRAM, SRAM, SDRAM, etc., and nonvolatile memory elements, such as a ROM, PROM, FPROM, OTP NVM, EPROM, EEPROM, hard disk drive, solid state disk, magnetic tape, CDROM, DVD, etc.).
- RAM random access memory
- PROM PROM
- FPROM OTP NVM
- EPROM EPROM
- EEPROM electrically erasable programmable read-only memory
- a nonvolatile computer-readable memory refers to a computer-readable memory that can retain information stored in the memory when it is not powered.
- a computer-readable memory is a physical, tangible object that is a composition of matter, and it cannot be construed as an “activity” of any nature whatsoever.
- the storage of data, which may be computer instructions, or software, in a computer-readable memory physically transforms that computer-readable memory by physically modifying it to store the data or software that can later be read and used to cause a processor to perform the functions specified by the software or to otherwise make the data available for use by the processor. It is the express intent of the inventor that in any claim to a computer-readable memory, the computer-readable memory, being a physical object that has been transformed to record the elements recited as being stored thereon, is an essential element of the claim.
- Software may include one or more separate computer programs configured to provide a sequence, or a plurality of sequences, of instructions to the processors to cause the processors to perform computations, control other devices, receive input, send output, etc.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Human Computer Interaction (AREA)
- Machine Translation (AREA)
- Document Processing Apparatus (AREA)
Abstract
Description
- The present invention relates generally to computerized text presentation, and more particularly to computerized text presentation based on text segmentation.
- Traditional typesetting practices (involving movable type) involve a mechanical arrangement of types to be printed on paper. Such arrangement includes a selection of typefaces, type size, and spacing between letters and line. With the advent of computers and word processing software, typesetting has become mostly automated. As such, users of modern word processing software rarely adjust the typesetting beyond font selection and line spacing.
- As the resolution and quality of computer displays has increased, the display of text has approached the resolution and quality of type on paper. Studies have been done to examine the impact of various typographical features on the legibility, readability and comprehension of written material, in print or on screen. Academic debates on the best practices are on-going. Nevertheless, by and large, practices from the past several hundred years of printing typography remain state-of-the-art. In some cases, these printing practices are carried over to electronic reading devices, while in other cases they are neglected.
- As individuals desire to read large amounts of text and data, often from disparate sources and often under time pressures (real or perceived), traditional practices of displaying textual information are often deemed to be insufficient. To address this need, inventors have developed methods of displaying information using computer-based systems outside of typographical traditions. One of the first alternative views was called “fisheye” in which the foremost content of interest is enlarged or shown in great detail while the adjacent content, often just landmarks, is reduced in size proportionally as distance increases. This enables the user to perceive the entire content while focusing on the salient content.
- There have been many attempts at facilitating quick browsing or skimming without reading the whole text. Generally, methods relate to applying focus on salient units of text (typically keywords) within a typical textual layout by making these units stand out visually (by modifying the font or highlighting), and in some cases obscuring/defocusing the non-salient text. Text summarization methods have also been developed in an attempt to reduce the quantity of text to be read and these methods continue to be a major area of academic pursuit.
- Alternative presentation formats have gone as far as eliminating the traditional format entirely and presenting text sequentially one or a few words at a time, at a user-adjustable rate, in a speed-reading fashion. Here, the intent is to eliminate all but the most immediately salient word(s).
- In all of these alternative text presentations, the underlying assumption is that the reader is fully literate and able to read the displayed text. Irrespective of these alternative text presentations and the best typographical practices, there still remain a large number of individuals for whom reading is difficult, whether on paper or on an electronic display. There are many reasons for this that impact the type of solution that is most appropriate.
- Beginning in the 1980s the field of assistive technology for persons with disabilities regularly began using text-to-speech or synthetic speech to read computer-based text aloud, primarily for communication purposes for non-speaking persons. In the 1990s, this same technology became popular in the field of education to assist persons with learning disabilities particularly in conjunction with word prediction technology. In addition to visually adjusting the text itself, the linking of speech output to the text became commonplace to aid in the reading of text. In particular, word-by-word highlighting synchronized with synthesized speech (text-to-speech) became the accepted norm. Research in the 90s provided evidence that speaking text with word-by-word highlighting was more effective in improving reading by persons with learning disabilities than reading plain text alone.
- The method of highlighting text either by whole sentence or by individual words and having it spoken aloud with the aid of synthesized speech is now commonplace in reading applications for consumer computer systems (including tablet-style) and standalone electronic devices such as eReaders and other reading/learning devices.
- It is an object of the invention to provide a system and method to enhance comprehension and readability of text by presenting the text to a user as a sequence of segments that allows the user to focus on one segment at a time.
- The present invention provides a method of parsing a sentence into a sequence of phrasal segments performed by a computer processor, the method comprising the steps of:
-
- (a) providing a target segment length expressed as a number of words or syllables;
- (b) parsing the sentence to identify candidate break points based on punctuation and parts of speech and including candidate break points at the beginning and end of the sentence;
- (c) eliminating candidate break points, leaving a plurality of final break points, so that every word in the sentence is situated between two final break points and the sum of the differences between (i) the target segment length and (ii) the number of words, if the target segment length is expressed as a number of words, or the number of syllables, if the target segment length is expressed as a number of syllables, between each consecutive pair of final break points is minimal; and
- (d) identifying each sequence of words between each consecutive pair of final break points as a segment.
- The parsing of the sentence may be performed based on a grammar.
- The parsing step may comprise identifying words that are conjunctions, verbs and prepositions, and the parsing of the sentence may comprise placing candidate break points at the beginning of the sentence, at punctuation marks in the sentence, and before and after each word identified as a conjunction, verb or preposition.
- The parsing step may comprise a first sub-step of pre-processing the sentence to break it into fragments based on mid-sentence punctuation marks and including candidate break points at the beginning and end of each fragment, and a second sub-step of parsing each fragment to identify candidate break points based on punctuation and parts of speech.
- Candidate break points may be removed from before and after a conjunction, verb or preposition if the conjunction, verb or preposition is situated within a phrase that should not be broken. It may be that a phrase should not be broken if it is in a pre-determined set collected from external lexicographical sources.
- The present invention also provides a method for parsing a document, the document comprising one or more than one paragraph, each paragraph comprising one or more than one sentence, into a sequence of phrasal segments performed by a computer processor, the method comprising the steps of:
-
- (a) parsing the document into paragraphs based on paragraph breaks;
- (b) parsing the paragraphs into sentences based on punctuation; and
- (c) parsing each sentence into a sequence of phrasal segments as described above.
- The present invention also provides a method of reading a document to a user performed by a computer processor, the method comprising the steps of:
-
- (a) parsing the document into a sequence of phrasal segments using the methods described above and identifying one segment as a current segment;
- (b) instructing a text-to-speech engine to read the current segment aloud to the user;
- (c) pausing for a pre-determined duration after the current segment has been read;
- (d) setting the current segment to be the next segment in the sequence; and
- (e) repeating steps (b), (c), and (d) until all segments have been read or until the user instructs the processor to stop.
- The duration of each pause may be selected to be discernibly longer than the average pause between words. The duration of each pause may be at least 500 milliseconds. Alternatively the minimum duration of a pause may be anywhere between 100 and 500 milliseconds. The duration of each pause, other than the pauses between sentences, may be the same. Alternatively all pauses may have the same duration, or the duration of each pause may vary based on the attributes of the preceding segment.
- The current segment may be displayed to the user while the text-to-speech engine reads the current segment.
- A sentence, being the current sentence, containing the current segment may also be displayed, and the current segment may be highlighted to differentiate it from the remainder of the current sentence.
- At least a portion of a paragraph, being the current paragraph, containing the current segment may also be displayed, and the current sentence may be highlighted to differentiate it from the remainder of the current paragraph.
- A portion of the text preceding the current paragraph in the document may be displayed before the current paragraph, and the current paragraph may be highlighted to differentiate it from the text preceding the current paragraph.
- A portion of the text following the current paragraph in the document may be displayed after the current paragraph, and the current paragraph may be highlighted to differentiate it from the text following the current paragraph.
- In general, the current segment may be highlighted to be most prominent.
- The reading of text by the text-to-speech engine may be muted, in a teleprompter mode, so that no words are sounded.
- The invention also includes systems comprising a programmable computer processor and a display, the processor running software adapted to perform the methods described above.
- The invention also includes a non-volatile computer-readable memory on which is stored software instructions adapted to run on a computer processor to cause the processor to perform the methods described above.
-
FIG. 1 a is a block diagram showing the steps of sentence segmentation. -
FIG. 1 b is a diagram showing the segmentation of a sentence by way of example. -
FIGS. 2 a, 2 b, 2 c, 2 d, 3 a and 3 b show a portion of text displayed by a processor with varying highlighting. -
FIG. 3 c a portion of text displayed by a processor with a segment of the text having been selected by a user and highlighted by the processor. - In view of the foregoing, and with modern typesetting practices being largely automated and standardized, notwithstanding the legibility issues being partially an issue of aesthetic choice, grouping coherent units visually or via speech can improve readability and comprehension of text. First, coherent semantic units should be visually coherent as well. Traditionally, one can employ the use of a physical highlighter to visually mark the coherently salient set of words, or alternatively, letter grouping or other font changes to increase emphasis, but the process can be laborious and depends on good reading skills. The other considers the use of synthesized speech output to augment the reading that emulates the visual recognition of coherent units through the listening route. This is intended to mimic a good public speaker, where pauses are inserted in places that are not necessarily standard clauses, but rather mid-clause break points, for phrasing, emphasis and provision of time for the comprehension of what has just been said. Neither of these features has been automated previously and yet can be implemented for reading on screen.
- A preferred embodiment of the invention comprises methods implemented in software that runs on a programmable computer processor. The invention also includes systems comprising a programmable computer processor running software adapted to perform these methods.
-
FIG. 1 a illustrates the high-level steps employed to segment adocument 101 into a sequence ofphrasal segments 108. The algorithm employs two major steps, identified asstep 1 andstep 2 inFIG. 1 a.Step 1 is thepre-processing 103 of aninput document 101 to create a set ofsentences 104 based on a predefined set of sentence markers. For example, a period, question mark or exclamation mark may be used to identify the end of a sentence in English. - During pre-processing 103, each sentence may be broken up into
smaller fragments 104 if there are mid-sentence punctuation marks such as commas, colons, and semi-colons. In general, contiguous portions of a sentence not having any such punctuation may be identified as sentence fragments or portions. - The second step is to further segment each sentence portion into smaller
phrasal segments 108. This involves placingcandidate break points 106 at the beginning and end of each sentence and between certain words in the sentence, and then choosingfinal breakpoints 107 from those candidate break points, taking into account a pre-definedtarget segment size 102, or segment length. The target segment length is generally expressed as a number of words, although in some embodiments it may be expressed, for example, as a number of syllables. - Candidate break points are placed at the beginning and end of each sentence portion. Candidate break points within each sentence portion may then be chosen by parsing the sentence. One way to do this is by analyzing each word in context to determine what part of speech it is. In order to identify
phrasal segments 108, the algorithm may attempt to identify parts of speech such as conjunctions, verbs, and prepositions because these parts of speech typically mark the inclusion of new information. Each word may be assigned a probability that it is each part of speech (e.g. conjunction, verb, preposition, noun, adverb, etc.) and then the word may be tagged as the part of speech having the highest probability. Suitable algorithms to perform this analysis and assign such a probability are well known to skilled persons. In general, a conjunction signals a new clause, a verb signals an event or state information, and a preposition signals additional information on the subject or the verb, so that candidate break points may be placed before and after each target word tagged as conjunction, a verb, or a preposition. - A candidate break point may be removed from before and after a conjunction, verb or preposition if the candidate break point is situated within a phrase that should not be broken. Such phrases may be members of a pre-determined set collected from external lexicographical sources. Another condition for removing a candidate break point may be that the candidate break point is too close to a sentence fragment boundary, such as by being one or two words away from the boundary.
- Where sufficient processing power is available, a more sophisticated parser may be employed based on a grammar, such as a lexical functional grammar. Grammatically correct sentences typically present an idea in the form of [subject] [verb phrase], where the verb phrase describes the details of an event or the state relevant to the subject. The verb phrase can be further broken down (grammatically) into smaller parts depending on the complexity of the event or state.
- Such parsing also identifies, and can be used to tag, parts of speech, but it completely breaks a sentence into its constituent phrases and sub-phrases. Many such techniques are well-known to skilled persons. Candidate break points may be placed before an S-BAR (a clause within a sentence), VP (verb phrase), or PP (prepositional phrase). However, a parser may not correctly handle grammatically ill-formed text and may not select a sufficient number of candidate break points so as to lead to the most appropriate selection of final break points.
- The final choosing of
break points 107 is made by eliminating some of the candidate break points based on thepredefined segment size 102. Thepredefined segment size 102 is considered to be the ideal number of words in eachphrasal segment 108 for purposes of reading comprehension. It may depend on information provided to the system about the human user, or reader. For example, for a child thepredefined segment size 102 may be set to three, but for normal adults, it may be set to five to seven words, depending on the person's abilities. Smaller or larger segment sizes may be appropriate in some situations. - If a sentence has twenty words and the preferred segment size is five, then in the ideal case there would be four segments of five words per segment. However, the selection of segments is constrained by the locations of the candidate break points so that the number of segments and the size of each segment do not generally correspond to the ideal case. The method employs an integer-partitioning algorithm to choose the final break points by eliminating candidate break points, thereby leaving a number of final break points, so that the sum total of the differences between the target segment size and the number of words between each consecutive pair of final break points is minimal. Two break points are considered to be consecutive if there is no other break point between those two break points
- Each sequence of words between each consecutive pair of final break points is then identified as a
phrasal segment 108. -
FIG. 1 b illustrates by way of example the segmentation of asentence 200 having 24 words intophrasal segments 108 according tostep 2 of the segmentation process depicted inFIG. 1 a, using asegment size 102 of six. The parts-of-speech tagging 105 assigns a tag to every word in thesentence 200 to form anaugmented sentence 201. Candidate break points are then placed in the sentence, as described above, to produce a preliminarily segmentedsentence 202, in this case having 11 candidate break points. Using thetarget segment size 102 of six, six candidate break points are then eliminated by choosingfinal breakpoints 107, leaving five final breakpoints in a finally segmentedsentence 203. Thephrasal segments 108 are then identified as each sequence of contiguous words between each consecutive pair of final break points, thereby resulting in fourphrasal segments 108 having lengths of 6, 5, 6, and 7 words. - In some cases, there may be more than one solution that minimizes the total difference. In that case, one solution may be selected arbitrarily, or heuristics may be used. For example, the solution with the least variance in the differences may be selected (so if one solution has differences of 0, 0, 0, and 4, and another has differences of 1, 1, 1, and 1, the latter may be selected).
- A processor may be adapted by software to use the segmentation of the text from a document into phrasal segments for various purposes. A system may employ the segmentation to highlight successive segments in the document for display to a user in sequence to facilitate the user reading the document. For example, the user may desire to read the text from a document, a portion of which is shown in
FIG. 2 a. The system may also employ a text-to-speech (TTS) engine, which are widely available. Such a TTS engine can be instructed by the system to convert one or a sequence of words into sounds that mimic how a person would speak the words so that they can be played aloud through acoustic speakers in electronic communication with the system. Such a TTS engine will speak the words provided to it at a rate, which may be adjustable, that is comparable to normal human speech. Rates of 120 to 180 words per minute are typical, although lower or higher rates may be suitable for some purposes. - By pausing between the completion of the speaking of one segment and the provision of the next segment to the TTS engine, the system can create a pause in the speech heard by the user. In addition to creating pauses between sentences, the system may also introduce a short pause of a pre-determined duration between phrasal segments. The pause between segments is generally chosen to be discernibly longer than the average pause between words, so that a normal user would recognize that the inter-segment pause is longer than the average pause or gap between other words in the sentence. It may be chosen, for example, to have a duration of 500 milliseconds, although longer pauses may be appropriate in some cases, for example between sentences. Shorter pauses may also be appropriate in some cases, such as when a relatively high rate of reading words is employed.
- Such pauses can significantly improve the understanding of a listener of the text being spoken by providing extra time for the listener to digest the content of the current segment. It should be noted that this will often result in pauses being placed between words in a sentence that are not separated by any punctuation mark, which is a key advantage of the system over the prior art.
- In addition to speaking the text, the system may also highlight the spoken words in the text on a display visible to the user. By “highlight”, we mean that a portion that is highlighted is visually distinguished from all other portions of text displayed. In the depicted embodiment, four classes of highlighting are used in the display so that every word on the screen is highlighted in one manner, with words in the same class being highlighted in the same manner. Highlighting may emphasize the highlighted text, e.g. making it the most prominent text displayed, or de-emphasize it, e.g. making it the least prominent text on the display. The differentiation may be achieved with boldness of the font, the colour, or the size, for example, or by techniques such as “de-focusing” text.
- For example, all the text shown in
FIG. 2 a may be displayed at one time on a computer screen. As shown inFIG. 2 b, the system may highlight the text of the sentence [10] that is currently being spoken by rendering it in a bold font that is bolder than the other text [11] in the same paragraph. - The remainder of the text [12] on the display, from the preceding or following paragraphs, may be shown in a faded manner so that it is less prominent that the other text [11] in the same paragraph as the current sentence [10]. This causes the reader who is listening to the current sentence [10] being spoken to focus on that sentence, but also provides context in the form of the rest of the paragraph [11], which is highlighted less prominently, and portions of other paragraphs [12] that are highlighted even less prominently than the remainder of the paragraph [11] containing the current sentence [10].
- In addition, each phrasal segment [14] within the current sentence [10] that is currently being read/spoken by the TTS engine may be further highlighted [13] such as in the manner shown in
FIG. 2 c, so that it the most prominent portion of the display. Such a segment continues to be highlighted as the most prominent portion of the display for as long as it takes for the TTS engine to speak the words in that segment. - In
FIG. 2 c, the text for the phrasal segment [14] currently being spoken is rendered in white with a surrounding black background [13]. After the current phrasal segment [14] has been spoken, a pause [15] may be introduced, and then the next phrasal segment [17], as shown inFIG. 2 d, may be fed to the TTS engine for speaking, and highlighted [16] on the screen in the same prominent manner in which the prior phrasal segment [14] had been highlighted, while the highlighting of the prior phrasal segment [14] is modified so that it is shown in the same manner (with the same form of highlighting) as the remainder of the current sentence. In this manner, the phrasal segment corresponding to the words currently being spoken by the TTS engine is always the most prominent portion of the display. When the last segment of a sentence has been spoken, the next sentence is then highlighted in the same manner that the previous sentence had been highlighted, and the previous sentence is then highlighted in the same manner as the remainder of the paragraph. Similarly, when the last segment of a paragraph has been spoken, the next sentence (in the next paragraph) is then highlighted as the previous sentence had been, and the remainder of the new paragraph is then highlighted as the remainder of the previous paragraph had been, and the highlighting of the previous paragraph is then modified to be the least prominent. The system may also scroll the display from time to time, or continuously, for example to keep the line(s) containing the current segment being read in the middle of the display. - In general, four distinct levels or classes of highlighting may be used as shown in
FIGS. 2 c and 2 d where (1) the current phrasal segment [14] is most prominently highlighted, (2) the remainder of the sentence being spoken is less prominently highlighted, (3) the remainder of the paragraph containing that sentence is even less prominently highlighted than the remainder of the sentence being spoken, and (4) the remainder of the text on the display in previous or following paragraphs is least prominently highlighted (e.g. by being faded or de-focused). The method of highlighting shown inFIGS. 2 c and 2 d is a preferred method of providing such differential highlighting, but other approaches may rely on differences in colour, brightness and font size for example. It is also not necessary that the levels of highlighting be discrete. For example, other than for the current phrasal segment, the prominence (e.g. boldness) of words could be decreased proportionately with the distance of each word from the current phrasal segment. - Rather than speaking the words via an acoustic speaker to the user, the system may instead operate in a teleprompter mode, muting the sound and just highlighting each successive phrasal segment as described above. The TTS engine may still be employed to determine the timing so that the highlighting is moved between phrasal segments as described above in a manner suitable for the person to read the highlighted text, while pausing briefly between phrasal segments. The user may be provided with a control to increase or decrease the reading speed.
- The system may also be used for non-linear reading. In this mode, the user may select a sentence such as by clicking on a word in the sentence with a mouse or using a touch screen. The system may then highlight the sentence [19] as shown in
FIG. 3 a and then may speak the sentence [19] via the TTS engine while highlighting [20] the segment [21] currently being spoken as shown inFIG. 3 b and as described above. - The system may further allow the user to select a segment within the sentence, for example by clicking again on a particular word, or swiping a number of words [22], as shown in
FIG. 3 c. The system may then highlight that segment [24] using the most prominent highlighting [23] as described above. The system may then use the TTS engine to speak the segment. - Generally, a computer, computer system, client or server, as will be well understood by a person skilled in the art, includes one or more computer processors, and may include separate memory, and one or more input and/or output (I/O) devices (or peripherals) that are in electronic communication with the one or more processor(s). The electronic communication may be facilitated by, for example, one or more busses, or other wired or wireless connections. In the case of multiple processors, the processors may be tightly coupled, e.g. by high-speed busses, or loosely coupled, e.g. by being connected by a wide-area network.
- A computer processor, or just “processor”, is a hardware device for performing digital computations. A programmable processor is adapted to execute software, which is typically stored in a computer-readable memory. Processors are generally semiconductor based microprocessors, in the form of microchips or chip sets. Processors may alternatively be completely implemented in hardware, with hard-wired functionality, or in a hybrid device, such as field-programmable gate arrays or programmable logic arrays. Processors may be general-purpose or special-purpose off-the-shelf commercial products, or customized application-specific integrated circuits (ASICs). Unless otherwise stated, or required in the context, any reference to software running on a programmable processor shall be understood to include purpose-built hardware that implements all the stated software functions completely in hardware.
- Computer-readable memory, which may also be referred to as a computer-readable medium or a computer-readable storage medium, which terms have identical meanings herein, can include any one or a combination of non-transitory, tangible memory elements, such as random access memory (RAM), which may be DRAM, SRAM, SDRAM, etc., and nonvolatile memory elements, such as a ROM, PROM, FPROM, OTP NVM, EPROM, EEPROM, hard disk drive, solid state disk, magnetic tape, CDROM, DVD, etc.). Memory may employ electronic, magnetic, optical, and/or other technologies but excludes transitory propagating signals so references to computer-readable memory exclude transitory propagating signals. Memory may be distributed such that at least two components are remote from one another, but are still all accessible by one or more processors. A nonvolatile computer-readable memory refers to a computer-readable memory that can retain information stored in the memory when it is not powered. A computer-readable memory is a physical, tangible object that is a composition of matter, and it cannot be construed as an “activity” of any nature whatsoever. The storage of data, which may be computer instructions, or software, in a computer-readable memory physically transforms that computer-readable memory by physically modifying it to store the data or software that can later be read and used to cause a processor to perform the functions specified by the software or to otherwise make the data available for use by the processor. It is the express intent of the inventor that in any claim to a computer-readable memory, the computer-readable memory, being a physical object that has been transformed to record the elements recited as being stored thereon, is an essential element of the claim.
- Software may include one or more separate computer programs configured to provide a sequence, or a plurality of sequences, of instructions to the processors to cause the processors to perform computations, control other devices, receive input, send output, etc.
- It should be emphasized that the above-described embodiments of the present invention, particularly, any “preferred” embodiments, are possible examples of implementations, merely set forth for a clear understanding of the principles of the invention. Many variations and modifications may be made to the above-described embodiment(s) of the invention as will be evident to those skilled in the art.
- Where, in this document, a list of items is prefaced by the expression “such as”, or is prefaced or followed by the expression “for example”, or “e.g.”, this is done to expressly convey and emphasize that the list is not exhaustive, irrespective of the length of the list. The absence of such an expression, or other similar expressions, is in no way intended to imply that a list is exhaustive. Unless otherwise expressly stated, such lists shall be read to include all comparable or equivalent variations of the items, and alternatives to the items, in the list that a skilled person would understand would be suitable for the purpose that the items are referenced.
- The words “comprises” and “comprising”, when used in this specification and the claims, are to used to specify the presence of stated features, elements, integers, steps or components, and do not preclude, nor imply the necessity for, the presence or addition of one or more other features, elements, integers, steps, components or groups thereof
- The scope of the claims that follow is not limited by the embodiments set forth in the description. The claims should be given the broadest purposive construction consistent with the description as a whole.
Claims (21)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/131,838 US8731905B1 (en) | 2012-02-22 | 2013-02-22 | System and method for enhancing comprehension and readability of text |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201261601766P | 2012-02-22 | 2012-02-22 | |
PCT/CA2013/000158 WO2013123583A1 (en) | 2012-02-22 | 2013-02-22 | System and method for enhancing comprehension and readability of text |
US14/131,838 US8731905B1 (en) | 2012-02-22 | 2013-02-22 | System and method for enhancing comprehension and readability of text |
Publications (2)
Publication Number | Publication Date |
---|---|
US8731905B1 US8731905B1 (en) | 2014-05-20 |
US20140156266A1 true US20140156266A1 (en) | 2014-06-05 |
Family
ID=49004874
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/131,838 Active US8731905B1 (en) | 2012-02-22 | 2013-02-22 | System and method for enhancing comprehension and readability of text |
Country Status (4)
Country | Link |
---|---|
US (1) | US8731905B1 (en) |
CA (1) | CA2833656C (en) |
GB (1) | GB2514725B (en) |
WO (1) | WO2013123583A1 (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150193436A1 (en) * | 2014-01-08 | 2015-07-09 | Kent D. Slaney | Search result processing |
US20180108356A1 (en) * | 2016-10-17 | 2018-04-19 | Honda Motor Co., Ltd. | Voice processing apparatus, wearable apparatus, mobile terminal, and voice processing method |
US20190121849A1 (en) * | 2017-10-20 | 2019-04-25 | MachineVantage, Inc. | Word replaceability through word vectors |
US11127171B2 (en) * | 2019-03-07 | 2021-09-21 | Microsoft Technology Licensing, Llc | Differentiating in-canvas markups of document-anchored content |
Families Citing this family (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9319444B2 (en) | 2009-06-22 | 2016-04-19 | Monotype Imaging Inc. | Font data streaming |
US8615709B2 (en) | 2010-04-29 | 2013-12-24 | Monotype Imaging Inc. | Initiating font subsets |
US9817615B2 (en) | 2012-12-03 | 2017-11-14 | Monotype Imaging Inc. | Network based font management for imaging devices |
US9569865B2 (en) | 2012-12-21 | 2017-02-14 | Monotype Imaging Inc. | Supporting color fonts |
EP2943894A2 (en) * | 2013-01-09 | 2015-11-18 | Monotype Imaging Inc. | Advanced text editor |
US9317777B2 (en) | 2013-10-04 | 2016-04-19 | Monotype Imaging Inc. | Analyzing font similarity for presentation |
US9691169B2 (en) | 2014-05-29 | 2017-06-27 | Monotype Imaging Inc. | Compact font hinting |
DE112014007207B4 (en) * | 2014-11-25 | 2019-12-24 | Mitsubishi Electric Corporation | Information presentation system |
US10115215B2 (en) | 2015-04-17 | 2018-10-30 | Monotype Imaging Inc. | Pairing fonts for presentation |
US11537262B1 (en) | 2015-07-21 | 2022-12-27 | Monotype Imaging Inc. | Using attributes for font recommendations |
US10699072B2 (en) * | 2016-08-12 | 2020-06-30 | Microsoft Technology Licensing, Llc | Immersive electronic reading |
US11334750B2 (en) | 2017-09-07 | 2022-05-17 | Monotype Imaging Inc. | Using attributes for predicting imagery performance |
US10909429B2 (en) | 2017-09-27 | 2021-02-02 | Monotype Imaging Inc. | Using attributes for identifying imagery for selection |
US11657602B2 (en) | 2017-10-30 | 2023-05-23 | Monotype Imaging Inc. | Font identification from imagery |
CN108305636B (en) * | 2017-11-06 | 2019-11-15 | 腾讯科技(深圳)有限公司 | A kind of audio file processing method and processing device |
CN109213974B (en) * | 2018-08-22 | 2022-12-20 | 北京慕华信息科技有限公司 | Electronic document conversion method and device |
CN110851564B (en) * | 2019-10-10 | 2023-06-09 | 安徽听见科技有限公司 | Voice data processing method and related device |
CN112750423B (en) * | 2019-10-29 | 2023-11-17 | 阿里巴巴集团控股有限公司 | Personalized speech synthesis model construction method, device and system and electronic equipment |
US20210406471A1 (en) * | 2020-06-25 | 2021-12-30 | Seminal Ltd. | Methods and systems for abridging arrays of symbols |
RU2769427C1 (en) * | 2021-04-05 | 2022-03-31 | Анатолий Владимирович Буров | Method for automated analysis of text and selection of relevant recommendations to improve readability thereof |
US11170154B1 (en) | 2021-04-09 | 2021-11-09 | Cascade Reading, Inc. | Linguistically-driven automated text formatting |
CN117113974B (en) * | 2023-04-26 | 2024-05-24 | 荣耀终端有限公司 | Text segmentation method, device, chip, electronic equipment and medium |
Family Cites Families (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5724498A (en) * | 1995-06-07 | 1998-03-03 | Adobe Systems Incorporated | Method and apparatus for concealing character modifications made for text composition purposes |
US5802533A (en) | 1996-08-07 | 1998-09-01 | Walker; Randall C. | Text processor |
US6279017B1 (en) | 1996-08-07 | 2001-08-21 | Randall C. Walker | Method and apparatus for displaying text based upon attributes found within the text |
US6154757A (en) | 1997-01-29 | 2000-11-28 | Krause; Philip R. | Electronic text reading environment enhancement method and apparatus |
US6130968A (en) | 1997-10-03 | 2000-10-10 | Mcian; Peter | Method of enhancing the readability of rapidly displayed text |
US7069508B1 (en) | 2000-07-13 | 2006-06-27 | Language Technologies, Inc. | System and method for formatting text according to linguistic, visual and psychological variables |
US7490092B2 (en) * | 2000-07-06 | 2009-02-10 | Streamsage, Inc. | Method and system for indexing and searching timed media information based upon relevance intervals |
US7159172B1 (en) | 2000-11-08 | 2007-01-02 | Xerox Corporation | Display for rapid text reading |
US20080141126A1 (en) | 2006-11-17 | 2008-06-12 | Vincent Lee Johnson | Method and system to aid in viewing digital content |
US8794972B2 (en) | 2008-08-07 | 2014-08-05 | Lynn M. LoPucki | System and method for enhancing comprehension and readability of legal text |
US8385652B2 (en) * | 2010-03-31 | 2013-02-26 | Microsoft Corporation | Segmentation of textual lines in an image that include western characters and hieroglyphic characters |
US20110313762A1 (en) * | 2010-06-20 | 2011-12-22 | International Business Machines Corporation | Speech output with confidence indication |
US20120290288A1 (en) * | 2011-05-09 | 2012-11-15 | Xerox Corporation | Parsing of text using linguistic and non-linguistic list properties |
-
2013
- 2013-02-22 WO PCT/CA2013/000158 patent/WO2013123583A1/en active Application Filing
- 2013-02-22 US US14/131,838 patent/US8731905B1/en active Active
- 2013-02-22 CA CA2833656A patent/CA2833656C/en active Active
- 2013-02-22 GB GB1416621.9A patent/GB2514725B/en active Active
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150193436A1 (en) * | 2014-01-08 | 2015-07-09 | Kent D. Slaney | Search result processing |
US20180108356A1 (en) * | 2016-10-17 | 2018-04-19 | Honda Motor Co., Ltd. | Voice processing apparatus, wearable apparatus, mobile terminal, and voice processing method |
US20190121849A1 (en) * | 2017-10-20 | 2019-04-25 | MachineVantage, Inc. | Word replaceability through word vectors |
US10915707B2 (en) * | 2017-10-20 | 2021-02-09 | MachineVantage, Inc. | Word replaceability through word vectors |
US11127171B2 (en) * | 2019-03-07 | 2021-09-21 | Microsoft Technology Licensing, Llc | Differentiating in-canvas markups of document-anchored content |
Also Published As
Publication number | Publication date |
---|---|
GB2514725A (en) | 2014-12-03 |
CA2833656C (en) | 2014-03-18 |
GB201416621D0 (en) | 2014-11-05 |
GB2514725B (en) | 2015-11-04 |
WO2013123583A1 (en) | 2013-08-29 |
CA2833656A1 (en) | 2013-08-29 |
US8731905B1 (en) | 2014-05-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8731905B1 (en) | System and method for enhancing comprehension and readability of text | |
AU2016202974B2 (en) | Automatically creating a mapping between text data and audio data | |
US9548052B2 (en) | Ebook interaction using speech recognition | |
JP7506092B2 (en) | System and method for simultaneously presenting target language content in two formats and improving target language listening comprehension | |
US20130222396A1 (en) | System and method for creating and displaying an animated flow of text and other media from an input of conventional text | |
KR20160111275A (en) | Foreign language learning system and foreign language learning method | |
US9818450B2 (en) | System and method of subtitling by dividing script text into two languages | |
KR102078626B1 (en) | Hangul learning method and device | |
US10276150B2 (en) | Correction system, method of correction, and computer program product | |
Akasheh et al. | Artificial intelligence-generated Arabic subtitles: insights from Veed. io’s automatic speech recognition system of Jordanian Arabic | |
US20080243510A1 (en) | Overlapping screen reading of non-sequential text | |
WO2008066361A1 (en) | Language learning contents provider system | |
US20230069113A1 (en) | Text Summarization Method and Text Summarization System | |
Campbell | On the structure of spoken language | |
KR20170009487A (en) | Chunk-based language learning method and electronic device to do this | |
JP2007156888A (en) | Information presentation system and information presentation program | |
JP2009075526A (en) | Comprehensive english learning system using speech synthesis | |
US20220036625A1 (en) | Method and electronic device for providing sign language | |
KR20220136801A (en) | Method and apparatus for providing associative chinese learning contents using images | |
JP2017117149A (en) | Electronic book display device | |
Brierley et al. | Non-traditional prosodic features for automated phrase break prediction | |
Caldwell et al. | Understanding WCAG 2.0 | |
Narayanan et al. | Automatic Detection of Phrase Boundaries and Highlighting Phrases in Text | |
WO2023059818A1 (en) | Acoustic-based linguistically-driven automated text formatting | |
CN114880495A (en) | Method, device and system for highlighting content |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: QUILLSOFT LTD., CANADA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:TSANG, VIVIAN;JACOB, DAVID;SHEIN, FRASER;REEL/FRAME:031934/0593 Effective date: 20120227 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2551) Year of fee payment: 4 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2552); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY Year of fee payment: 8 |