US20070011147A1 - Systems and methods for retrieving data - Google Patents
Systems and methods for retrieving data Download PDFInfo
- Publication number
- US20070011147A1 US20070011147A1 US11/473,542 US47354206A US2007011147A1 US 20070011147 A1 US20070011147 A1 US 20070011147A1 US 47354206 A US47354206 A US 47354206A US 2007011147 A1 US2007011147 A1 US 2007011147A1
- Authority
- US
- United States
- Prior art keywords
- data
- enterprise
- databases
- software
- search
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/80—Information retrieval; Database structures therefor; File system structures therefor of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML
- G06F16/83—Querying
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Definitions
- Knowledge Management is about the protection, development and exploitation of knowledge assets. KM encompasses theories, models, processes and tools that support the efficient and effective evaluation, acquisition, dissemination, development and exploitation of knowledge resources in organizational processes and decision making.
- Knowledge engineering is the use of information assets regarding situations, users, products and technologies that an organization owns (or needs to develop as part of its strategic plan) and then uses to implement organizational processes efficiently and effectively.
- Knowledge Engineering Systems and Tools provide the means of combining individuals' knowledge in the pursuit of personal and organizational objectives.
- knowledge engineering supports person-to-person and group-to-group communications resulting in the development of new knowledge via integration.
- These more advanced computer collaboration and data sharing methodologies can create a level and quality of exchange not normally achievable within the conventional data-sharing environment.
- the present invention is directed a system and method for obtaining information responsive to a legal discovery request directed at an enterprise.
- the legal discovery request corresponds to a document subpoena directed to the enterprise or a request for production of documents served on the enterprise in litigation.
- a plurality of databases is associated with the enterprise.
- the databases potentially include information responsive to the legal discovery request, and include a plurality of structured databases and at least one unstructured database.
- a plurality of taxonomies are provided, each of which describes a structure of one of the structured databases.
- An ontology is provided, based at least in part on the taxonomies, that describes data relationships between the plurality of databases associated with the enterprise.
- a native application is identified, which is configured to access at least one but not all of the plurality of databases associated with the enterprise.
- the native application is modified to include search functionality for accessing all of the plurality of databases associated with the enterprise.
- a request is received from the user, via the modified native application, to search the plurality of databases associated with the enterprise for information responsive to the legal discovery request. Information included in the request, the ontology and the taxonomies are then used to search the plurality of databases associated with the enterprise.
- a list of search results responsive to the search request is presented to the user with the modified native application. In response to a user selection of a search result from the list, an application required to view the selected search result is automatically launched if the modified native application lacks functionality required to display the selected search result.
- the present invention may be implemented in the context of a software application that is currently available in a current version and which was previously available in an old version, where the current version of the software stores data in a current data storage format and the old version of the software previously stored in an old data storage format which is different from the current data storage format.
- a data metamorphosis system and method in accordance with the present invention is used in this context to support usage of data stored in the old data storage format with the current version of the software.
- an XML (eXtended Markup Language) definition is provided for each of a plurality of object classes used to store data in the old data storage format and the new data storage format, and data format rules are provided for each XML definition.
- At least some of the XML definitions and format rules are used to dynamically convert the data in the old storage format to the current data storage format, thereby allowing usage of the data in the old storage format by the current version of the software.
- the present invention is directed to a system and method for collecting and retrieving data for an enterprise with a plurality of databases including a plurality of structured databases and at least one unstructured database.
- a plurality of taxonomies is provided, each of which describes a structure of one of the structured databases.
- An ontology is provided, based at least in part on the taxonomies, that describes data relationships between the plurality of databases of the enterprise.
- a software agent is associated with each of the plurality of databases of the enterprise. Each software agent uses information included in a search request, the ontology and the taxonomies to search an associated database and generate search results.
- Each software agent duplicates and archives data retrieved from the associated database based on the search results, and wraps the retrieved data with a standardized object format that facilitates data transformation and secure access of the retrieved data at a field level.
- Each software agent also logs events that occur with respect to an application that corresponds to the database associated with the software agent. The events logged by the software agent include, for example, server status, server operations performed, and network status.
- the present invention is directed to a network having a plurality of nodes, where nodes enter and exit the network in a manner that is unpredictable.
- Data availability is improved in the network by fragmenting data across the network nodes in order to increase a likelihood that the data will be available irrespective of a combination of nodes that are connected to the network at any given time.
- Each fragment of data includes a header with control information that specifies how the fragment connects with other fragments stored in the network.
- FIG. 1 is a flow diagram depicting knowledge flow or data flow through the repository of the present invention.
- FIG. 2 is a block diagram showing program components and logical storage areas used for implementing the search technology of the present invention.
- FIG. 3 is a flow diagram illustrating steps performed by the search technology of the present invention.
- FIG. 4 is an exemplary screen shot illustrating a native application which has been modified to include a Search Interface for accessing the search functionality of the present invention.
- FIG. 5 is a further exemplary screen shot illustrating a native application which has been modified to include the Search Interface for accessing the search functionality of the present invention.
- FIG. 6 is a still further exemplary screen shot illustrating a native application which has been modified to include the Search Interface for accessing the search functionality of the present invention.
- FIG. 7 is a graphical user interface for mapping data between a current version of a software application which stores data in a current data storage format and an old version of the software which previously stored the data in an old data storage format.
- FIG. 8 is a further example of the graphical user interface for mapping data between a current version of a software application which stores data in a current data storage format and an old version of the software which previously stored the data in an old data storage format.
- FIG. 9 is a graphical user interface depicting certain core functions that the user can select in order to map data between a current version of a software application which stores data in a current data storage format and an old version of the software which previously stored the data in an old data storage format.
- FIG. 10 is a graphical user interface depicting a user-defined function (created from the core functions of FIG. 9 ) developed by a user for mapping data between a current version of a software application which stores data in a current data storage format and an old version of the software which previously stored the data in an old data storage format.
- FIG. 11 is a graphical user interface depicting a visual function builder for mapping data between a current version of a software application which stores data in a current data storage format and an old version of the software which previously stored the data in an old data storage format.
- FIG. 12 depicts an XML Schema generated using the user-interface shown in FIGS. 7-11 .
- FIG. 13 is a block diagram showing the components of a data collector agent of the present invention.
- FIG. 14 is a flow diagram illustrating steps performed by the data collector agent of the present invention.
- FIG. 15 is a block diagram illustrating the object transmission format used by the data collector agent for transmitting information.
- FIG. 16 is a flow diagram illustrating steps performed for fragmenting data in order to improve availability in accordance with the present invention.
- FIG. 17 is a block diagram illustrating a fragmented object store header format, in accordance with the present invention.
- the present invention is implemented as an object-oriented, agent-based scalable digital repository that creates a storage virtualization layer over any combination of storage devices, databases or other knowledge sources.
- the invention combines aspects of relational, object-oriented and XML databases to create a content integrating repository.
- the invention uses data mining methods and graphical interface tools, and interfaces with most disparate enterprise data sources requiring only point-and-click methods during installation.
- Other ‘content integrating’ knowledge managers require substantial entry of SQL code and XML in order to accomplish the connection to the enterprises' disparate data sources.
- Existing data may be made accessible to the present invention by describing the format and structure of the data via XML oriented methods and then having the invention scan and build metadata indices.
- New data may be inserted into the repository via API calls or using a utility program that pulls data directly from the data source without altering existing programs.
- Policy Templates control the metadata index content and determine where a particular data class will be stored within the repository. Support for Information Lifecycle Management and Digital Rights Management are provided in certain embodiments.
- the software supports interfaces to enterprise databases and applications, plus web service middleware such as Web Sphere.
- the software binds tightly with applications. This ability is important to compliance-oriented solutions or to solutions where creations of an audit trail and secure storage on data is required. This is performed via a module that binds tightly to the data-source application (such as an Exchange Mail Server) and records all activity of the server as well all executed data transactions.
- the data-source application such as an Exchange Mail Server
- the software preferably includes an auto-archiving ability.
- data can be automatically moved off-line or to other storage locations. As time passes some data no longer is actively used in the enterprise. This data is archived or placed on storage that no longer allows modification of the data.
- This media can be, but is not limited to, certain types of disk drives or tape storage.
- the software allows the data to be moved to off-line storage while maintaining metadata indices online with the location of all the data. Off-line refers to storage media that requires human or mechanical intervention in order to access the data via means of a computer (tape stored on a shelf must be mounted in a tape drive by an operator). This maintains database performance and levels and reduces resource utilization as information accumulates.
- Compliance data may be moved offline, unless used to support the findings of an audit, after, e.g., a pre-set period of time such as three years, but be able to be located on-line for a longer period, e.g., seven years (per Sarbanes-Oxley and SEC Rule 17).
- the software allows the data to be moved to off-line storage while maintaining the metadata indices online with the location of all the data. This is performed without explicit user action via policy.
- the software also supports the ‘marking’ of audit/case support information causing it to remain online per compliance requirements so that it can be easily is retrieved as supporting information.
- the software requires no changes to existing data sources or applications for their integration into the repository used by the present invention.
- the software preferably meets the stringent requirements of the rules and regulations recently established for compliance with Sarbanes-Oxley, HIPPA, SEC and other regulators.
- the software offers an auditable, secure chain of evidence from data generation at the data source to destruction of the data per policy.
- the data is always stored in the archive in the original data format.
- the software preferably supports various aspects of compliance support, including the ability to extend the on-line and destruction times of selected data items per regulations should they support audit findings.
- the software uses separate meta-data indices from the data per requirements, and provides a packaging function that will copy the metadata and source data to a specified destination (CD-ROM, DVD, Disk Drive, etc.) with (optionally) a search engine to aid the receiver of the knowledge in processing of the data.
- the software supports Digital Rights Management (DRM) of the data deposited in its repository. Policies are used to control the application of where and how DRM is applied within the repository.
- the software also preferably supports use of the XML standards for Digital Rights Management including (i) DPRL—Digital Property Rights Language, (ii) XrML—Extensible Rights Markup Language, and (iii) XACML—Extensible Access Control Language.
- the software supports packaged and custom applications, databases (Oracle, DB2, SQL Server, Informix, and Sybase), servers and storage classes (SAN/DAS. CAS. Tape), and is configurable for single companies, multiple divisions, multinational organizations, or ASPs supporting any combination of organizational types.
- the software includes policies and templates that may be configured to control each organizational unit's storage affinity and retention periods separately or in an aggregate form.
- the software also interfaces transparently with collaboration software such as Lotus Notes and Microsoft Exchange.
- the software is based on a Highly Available Architecture (HAA) that assures data being deposited or withdrawn from the repository is delivered and correct. Automatic synchronization and healing of virtualized storage areas is automatically performed per policies and rules after a partitioned network is healed.
- HAA Highly Available Architecture
- the software enables the transparent integration of data from diverse and disparate sources.
- the software is based on a Hierarchical Index Architecture that can handle any data type and any number of entries. This is enabled by using a hierarchy in the index structure that allows a distributed method for the index.
- the index or metadata can be replicated like data in a database, and assigned affinity to geographical locations or logical locations. This allows an index to be generated near the data source and then ‘distributed’ for search operations via the network according to policy.
- the index can be merged into a central index or intelligently summed to allow the rapid locating of those distributed data sources which contain items of interest to be recalled.
- the software supports policy definition for retention and has the ability to manage data from non-structured sources such as email and Instant Messaging also via policy. Any source of data (document scanners, fiche readers, 5ESS switch, etc.) can become an information provider to the repository of the present invention.
- the software provides a unified view of repository data growth for the monitoring and control of the all storage areas and types that make up the repository. Views by location, data classes, storage type, etc. are possible.
- An added feature is to create knowledge relationships based upon chained events. This allows the creation of alerts or other defined actions that require correlation of a complex series of events. For example, a keyword match of selected words in an email, followed by a phone call to a particular number, then ending with a stock trade could be a possible alert.
- the software provides the ability for charge-back information to be supplied to budgetary programs and applications.
- the software has a language semantic engine that improves the ‘fuzzy’ search results beyond the stemming methods used in other search processing engines.
- the software has the ability to dynamically transform data upon retrieval. Using the taxonomy and ontology information contained in a knowledge encyclopedia, the software combines this information with specific transformation rules to convert the data as necessary to meet the requirements for the information to be utilized by the retrieving application.
- the software supports a multiple language ability for serving international enterprises that exchange data freely across borders.
- the search engine supports queries with intelligence to improve the relevancy of the search results across different languages.
- the software uses an object-oriented digital repository. This allows integration between the application logic and the archived data to be the most transparent possible, this is known as transparent persistence.
- transparent persistence The ability to keep the persistence and application logic in the same object-oriented paradigm simplifies modeling, design tool requirements, and visualization of the system architecture and design.
- the persisting may apply not only to data but also the entire object, including its implemented behavior. This supports calling the persisted object's methods on a remote database server, thereby providing advantages in scalability and distribution.
- stored procedures or COM objects would need to be implemented to accomplish this result, forcing some duplication of coding efforts or a more awkward architecture.
- the present invention reduces development time by allowing developers to focus on object persistence, not the decomposition of objects to rows and parent/child relationships in one or more tables. In most cases, a developer simply saves an object to the repository.
- the software utilizes a highly scalable architecture that provides the ability to handle the demands of enterprise applications and distributed native data access.
- the software uses optimization of queries and transparent dynamic caching methods to deliver users a non-disruptive, responsive approach to enterprise knowledge search and retrieval.
- the software scales by the simple addition of more agents to handle increased transactional loads.
- the agents utilize network traffic monitoring, node loading measures, and transaction counts creating a feedback loop that enables intelligent decision making about when, where and if additional agents are needed.
- the software can also generally throttle operations in order to maintain throughput on other critical applications operating in the network.
- the performance control mechanism is also capable of throttling only certain types of operations based upon type, user, group, data of day, node, network segment, etc.
- the software is hardware/software platform neutral or agnostic.
- the software runs on mainframes to embedded platforms and supports operation on z/OS, z/VM, OS/400, AIX, Linux, Solaris, Windows and etc.
- the software provides for support of policy or template-based control of data retention. It also uses policies to control data affinity or the ability to direct selected data types to particular storage types or physical locations.
- the policy allows for the replication of data among locations and storage types. For example, using the metamorphose ability with replication it is possible to replicate database tables with unlike structures by establishing policy rules on how to convert the data.
- the software leverages the abilities and functionality of the databases and storage technology that it interfaces with during virtualization of data sources.
- the indexing and storage processes can be tasked to the database and storage technology that delivers the best performance for that data or information type.
- the software uses available technologies to locate and retrieve knowledge from the repository.
- the software's unique architecture allows the user to leverage the data mining abilities, unique search abilities and other features of the database and applications it utilizes as data sources.
- the software includes features that allow the specification of external programs, routines, etc. for specific data search problems. For example, the software can be extended to annotate and process video streams real-time to create easily searchable video knowledge repositories.
- the default search methods implemented in the software use the latest ranking and fuzzy search technology to enable the user the greatest ease of use.
- the software supports basic and advanced security modes of operation. From simple password based protection to advanced Digital Rights Management methods on individual data items fields.
- the software also uses compressed, encrypted communications (selectable by policy) for movement of data within the archive or between storage locations. This is also used to create a true chain of evidence for data relating to compliance.
- the ‘chain of evidence’ provides a secured log of all types of access attempted or performed on any information deposited within the repository.
- the software supports XML open standards and eliminates the risk of application obsolescence. Data remains accessible even after the application is retired. In addition, the software will metamorphose data per a defined policy during retrieval to support the current version of an application or other data formatting requirements of a retrieval destination.
- the software is in compliance with ISO 14721 and DoD 5015.2 for Digital Archives.
- the software provides transparency such that all data sources appear and act as one data source.
- the software can access archived data through existing application interfaces, online, and in real time.
- offline data requests are queued via librarian utilities or via operator interfaces and managed without user intervention.
- Off-line (long-term archived items) can be queued for retrieval and then presented to the requesting user once they have placed on-line either in the on-line archive or in a temporary cache per policy.
- the software allows data items to be versioned in the repository. This keeps a complete audit trail of the data item thought all of its changes. Policies are used to determine the version number allowed, expiration dates, always keep the original and other options. Compliance data is preferably never modified, and CRC or MD5 signature methods are used on the data to ensure this does not happen. Encryption via other algorithms is also available.
- FIG. 1 a flow diagram is shown depicting knowledge flow or data flow through the repository of the present invention.
- Corporate compliance data from different sources within an enterprise is generated from a source application, device or database (e.g., VoIP application 102 , e-mail application 104 , statement/report application 106 , image database 108 , transaction database 110 , internet forms database 112 , or video database 114 ) and deposited to storage repository 100 using XML based data descriptions and policies.
- a list of different source data formats that may be generated by source applications and deposited into repository 100 are shown in Appendix A.
- the different data sources within the enterprise may include structured databases and unstructured databases.
- storage areas 120 , 122 and 124 are used for storing data in repository 100 .
- NAS/SAN arrays, databases or any other device may be used for storage. Data that has been deposited into repository 100 may also be withdrawn.
- the software of the present invention is capable of returning the data to the source application, device or database.
- a native application i.e., an interface on application/device/database 102 , 104 , 106 , 108 , 110 , 112 or 114
- Lotus Notes or Microsoft Exchange in the case of e-mail application 104
- FIG. 2 is a block diagram showing the program components and logical storage areas used for implementing the search technology of the present invention.
- the architecture shown includes four program components (data collector agents 202 , data storage agents 204 , management and control agents 206 , and process flow management agent 208 ) and utilizes six different logical storage areas (discussed below) to perform its functions.
- FIG. 2 shows the functional deployment in a network of the program and storage area components of the present invention. Multiple instances of each the components may exist in a network to provide scalability, reliability and accessibility to the data in repository 100 .
- the six logical storage areas used by the architecture are:
- An Evidence Chain Repository (ECR) 210 which holds an audit log of all activities for a given item deposited into repository 100 . This log also includes the activities of the data source associated with a given item of data, such as when the server is taken down or purges are run, etc.
- MTR MetaTag Repository
- MTR 212 holds XML based descriptions of the various native formats of the data supplied to repository 100 by sources 102 , 104 , 106 , 108 , 110 , 112 .
- MTR 212 also holds (i) security templates/policies, (ii) processing templates/policies, (iii) indexing rules/requirements, (iv) taxonomy and ontology of the enterprise data, (v) data metamorphosis rules, (vi) workflow processing triggers, (vii) data correlation rules, and (viii) storage area affinity and control policies.
- a MetaData Repository (MDR) 214 which holds indexes to the data stored in repository 100 or registered with repository 100 for purposes of searching.
- ODR Object Data Repository
- a Workflow Repository (WCR) 218 which holds workflow process rules for targeted data classes.
- An Evidence Data Repository (EDR) 220 which holds data linked and kept separate during a workflow process for purposes of isolation or easy retrieval.
- step 302 the search technology of the present invention is invoked through a native application associated with one of sources 102 , 104 , 106 , 108 , 110 , 112 , which has been modified to include a Search Interface for invoking the search functionality of the present invention in order to simultaneously access all of the plurality of data sources associated with the enterprise.
- Exemplary screen shots illustrating a native application Microsoft Outlook in the example shown
- FIGS. 4-6 Exemplary screen shots illustrating a native application (Microsoft Outlook in the example shown) which has been modified (e.g., using a plug-in) to include the Search Interface for accessing the search functionality of the present invention are shown in FIGS. 4-6 .
- the user enters search parameters (for searching all of the plurality of data sources associated with the enterprise) into the Search Interface.
- the search parameters represent search terms and a search strategy designed to elicit information responsive to a legal discovery request directed at an enterprise.
- the legal discovery request corresponds to a document subpoena directed to the enterprise or a request for production of documents served on the enterprise in litigation.
- the search parameters are provided to metadata storage agent 204 .
- metadata storage agent 204 returns to the user a search results list representative of data that (i) matches the search request, and (ii) which the user is authorized to access. An example of such a search results list is shown in FIG. 6 .
- step 310 the user is presented with an option to narrow the search criteria by entry of additional parameters. If the user opts to narrow the search criteria, the additional parameters are input by the user via the Search Interface, and in step 312 , the additional search parameters are provided to metadata storage agent 204 . In step 314 , metadata storage agent 204 returns to the user a search results list representative of data that (i) matches the narrowed search request, and (ii) which the user is authorized to access. Steps 312 - 314 may be repeated in order to successively narrow the search, as desired by the user.
- step 316 the user can display an item corresponding to one of the entries on the search results list (see FIG. 6 ) by clicking on the item via the Search Interface.
- the Search Interface accesses the metadata storage agent 204 (step 318 ), which in turn returns the location or locations of source data (which may be from data sources 102 , 104 , 106 , 108 , 110 , 112 , 114 and/or 100 and/or 120 , 122 , 124 ) corresponding to the item selected by the user (step 320 ) and a display method associated with the source data (step 322 ) (e.g., the identify of an application required to display the data).
- source data which may be from data sources 102 , 104 , 106 , 108 , 110 , 112 , 114 and/or 100 and/or 120 , 122 , 124
- the Search Interface uses the data location information and the display method information to display the source data to the user.
- the native application used for implementing the Search Interface lacks functionality required for displaying the selected source data
- an application required to view the selected search result is automatically launched by the Search Interface and used for displaying the selected source data to the user.
- step 326 the user can select an item corresponding to one of the entries on the search results list (see FIG. 6 ) in order to save the data to a storage medium for distribution.
- This media is not part of the repository, but a writable storage media attached to the user's computer or network that they have security authorization to write to.
- An example would be an in-house corporate counsel locating all activities of a given broker and then using this facility to prepare a CD-ROM to send to external counsel.
- the Search Interface accesses metadata agent 204 which in turn returns the location or locations of source data (which may be data sources 102 , 104 , 106 , 108 , 110 , 112 , 114 and/or 100 and/or 120 , 122 , 124 ) corresponding to the item selected by the user (step 330 ) and the storage method currently used for storing the source data (step 332 ).
- the Search Interface uses the data location information and the storage method information to access the source data and then store the data on a storage medium selected by the user.
- a plurality of taxonomies is provided within MetaTag Repository 212 .
- Each of the taxonomies describes a structure of a structured database corresponding to one of the data sources ( 102 - 112 ).
- An ontology is also provided within MetaTag Repository 212 , based at least in part on the taxonomies.
- the ontology describes data relationships between the plurality of databases associated with the enterprise (e.g., data sources 102 - 112 ).
- the ontology and the taxonomies are accessed and used to search the plurality of source databases associated with the enterprise.
- the software of the present invention includes ‘data metamorphosis’ functionality which supports usage of data stored in the old data storage format with the current version of the software.
- the invention includes an XML definition for each of a plurality of object classes used to store data in the old data storage format and the new data storage format, and data format rules for each XML definition.
- At least some of the XML definitions and format rules are used to dynamically convert the data in the old storage format to the current data storage format, thereby allowing usage of the data in the old storage format by the current version of the application.
- Table I below shows a sample XML definition for customer order data stored by an old version (e.g., version 1 ) of an ordering system software application
- Table II below shows a sample XML definition for customer order data stored by a current version (e.g., version 7 ) of the ordering system software application.
- the present invention includes a graphical user interface (shown in FIGS. 7-11 ) for generating the W3C organization XML definitions. These definitions in conjunction with code (format rules) are stored in MetaTag Repository 212 , and provide the present invention with the ability to dynamically alter data formats during the retrieval of source data from storage locations.
- the user's authority to access (and/or share) data is determined in the context of a collaboration environment that changes dynamically based on the occurrence of events external to the data collaboration environment.
- the collaboration environment is defined to include roles, activities and external event specifications. Each role defines data access privileges granted to a user during a session, each activity defines with whom the user can share data, and each external event specification defines an event external to the collaboration environment which must occur before an activity is permitted for a defined role during a session.
- Role This names the ‘role’ of the user being granted Specification access to the information contained in the system.
- a given individual can be assigned several ‘roles’.
- an individual may be assigned the following roles: administrator, auditing, security; while another individual may simply be assigned the role: user.
- Activity Control Complete access to system information, including Specification defining security for others for the information item, and all other functions Read - user only to read information and not alter any.
- Event may write new information, but not modify existing or read existing information Modify - User may read and alter existing information, but not create new information Meta - User may access only metadata for the information, but not the source information Delete - User may delete information Audit - Read the audit log concerning activities for the information Event Defines an external ‘condition’ such as a Specification Defense Department ‘DEFCON’ (Defense Condition). DEFCON 1 is the highest alert level (war) with DEFCON 5 being a peaceful alert level. Any ‘conditions’ can be defined to cover information sharing activities. For example, an event can be defined to permit military or civilian agencies that otherwise do not share data to automatically share during emergencies or other defined events/conditions.
- Each data collector agent 202 includes a first portion 202 a which is customized to the data source associated with the agent; an agent API portion 202 b , and an interface portion 202 c for interfacing with other agents.
- step 1402 the agent starts-up and begins initialization.
- step 1404 data collection agent 202 sends a broadcast message to request a connection to a data storage agent 205 and, in step 1406 data collector agent waits for a connection with data storage agent 205 .
- the storage agent virtualizes the storage destinations for the system.
- the storage agent uses the MetaTag Repository, as the source for its configuration and management.
- the storage agent stores and retrieves data to and from the Object Data Repository and OCR.
- This component provides reliability and scalability to the data storage function of the invention by recovering from communications outages, network-partitioning events, and server crashes using rules defined in the MetaTag Repository.
- the metadata agent manages the MetaData Repository via the information in the MetaTag Repository.
- the metadata agent maintains the integrity of the metadata structure and ensures the ‘freshness’ of the metadata at all levels of the hierarchy.
- step 1408 data collector agent 202 establishes a connection to the data source associated with the agent (e.g., source 102 ), and in step 1412 extracts metadata values which may include, without limitation, major indicies of databases, word dicitionaries for unstructured data, or time code annotations for video streams from the data source associated with agent 202 .
- step 1414 the agent 202 transmits the collected metadata values to a data storage manager for collection in MetaData repository 214 .
- agent 202 transmits source data (from the data source associated with agent 202 ) to a data storage manager for collection in object data repository 216 . If auditing of the source associated with agent 202 has been specified, then in steps 1420 and 1422 , the agent 202 collects auditing data from the data source associated with agent 202 , and transmits the audit data to a data storage manager for collection in evidence chain repository 220 .
- agent 202 and evidence chain repository 220 log various events that occur with respect to the application associated with the data source such as, e.g., server status, server operations performed, and network status, application changes, stops and starts, when data is accessed, by who, when and what operations was attempted.
- the evidence chain repository also collects information from the data storage agents 204 when information is accessed within the repository 100 . This includes the type of access (read, write, etc.), who accessed and the time.
- FIG. 15 is a block diagram illustrating the object transmission format used by the data collector agent 202 for transmitting information to the data storage and metadata agents 204 .
- availability of a data object is improved in the present invention using a fragmented networked storage method that exploits the statistical stability of a large number of storage points.
- the concept improves retrievability of a data object over simple replication and is applicable to networks where nodes enter and exit the network in a manner that is unpredictable.
- this storage method the number of segments to break a data object into is specified. Thus, if two fragments (backups) are specified and 32 replicates exist for each fragment, this would correspond to a 2-times storage overhead.
- P Q the probability that a given Data Object will be available
- n the total number of data fragments
- m the minimum count of data fragments needed to reconstruct the data object
- N the total number of network storage positions (network nodes with storage and storage agents 204 )
- M is the count of networked storage positions unavailable.
- the probability that a given data object will be available is equal to the number of ways that unavailable fragments can be arranged on unreachable networked storage positions multiplied by the number of ways in which available fragments can be arranged on reachable networked storage points, divided by the total number of ways all fragments can be stored on all networked storage positions.
- 2 simple replicated copies yield an availability probability of 99%
- 2 fragments stored via 32 sub-fragments yield an availability probability of 99.99998%.
- FIG. 16 illustrates steps performed for fragmenting data in order to improve availability in accordance with the present invention.
- the MetaTag Repository 212 for an object class has fields that denote the number of replicates and fragments for each replicated piece.
- the data manager agent 206 then handles the distribution of the data object across the nodes required to meet the specifications.
- Each fragment is assigned a unique identification which is held in MetaData repository 214 .
- the metadata repository 214 is replicated in multiple locations to assure its availability.
- the original data object is not deleted until verification of that data is stored according to specified requirements of the object class for replication and fragmentation.
- the fragmented object store object header (shown in FIG. 17 ) is structured similar to the transmission object of the agent system. As shown in FIG. 17 , each fragment of data includes a header with control information that specifies how the fragment connects with other fragments stored in the network.
- the Source Data Unique Identifier in combination with the Replicate ID and the Fragment ID creates a unique identifier for each fragmented piece of the data object ( FIG. 16 ).
- the MetaTag Repository 212 specifies the number of replicates (RC—replicate count) and the number of fragments for each replicate (FC—fragment count).
- RC replica count
- FC fragment count
- the object can be reassembled from the fragments if the metadata repository 214 is destroyed or unavailable by scanning each object data repository 216 instance and reading the header replicate/fragment IDs.
- a complete metadata entry can be constructed to read the entire data object once sufficient nodes have scanned their object data repositories 216 . The process is reversible even with deliberate tampering by destruction of the metadata indices which point to the data locations.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- General Engineering & Computer Science (AREA)
- Human Resources & Organizations (AREA)
- Entrepreneurship & Innovation (AREA)
- Strategic Management (AREA)
- Economics (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Computational Linguistics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A system and method for obtaining information responsive to a legal discovery request directed at an enterprise having a plurality of databases that potentially include information responsive to the legal discovery request, including structured databases and at least one unstructured database. A plurality of taxonomies are provided, each of which describes a structure of one of the structured databases. An ontology is provided that describes data relationships between the databases. A native application is identified, which is configured to access at least one but not all of the databases associated with the enterprise. The native application is modified to include search functionality for accessing all of the databases associated with the enterprise. A request is received from the user, via the modified native application, to search the plurality of databases associated with the enterprise for information responsive to the legal discovery request. Information included in the request, the ontology and the taxonomies are then used to search the plurality of databases associated with the enterprise. A list of search results responsive to the search request is presented with the modified native application. In response to a user selection of a search result from the list, an application required to view the selected search result is automatically launched if the modified native application lacks functionality required to display the selected search result.
Description
- The present application claims priority to U.S. Provisional Application No. 60/692,987 filed Jun. 22, 2005 entitled “ARK System, Including Features and Functions Thereof,” incorporated herein by reference in its entirety.
- Today's enterprises that must swiftly and decisively react to demands for data due to regulations, legal entanglements, or for matters requiring precise, select business intelligence data. Among other things, today's enterprises often require strategic targeted information deployment that enables precision situational awareness during evolving tactical events requiring rapid and decisive action. When the need for such information arises, the information needs to be identified and provided to the locations and persons who require it without a fog of extraneous information.
- The distinction between the three terms—data, information and knowledge is relevant to establish a common understanding of terms and concepts used in knowledge control. First, data represents facts, parameters or statistics that can be analyzed to produce information. Information is data related to a particular context that creates meaning. In other words, data represents facts and becomes information when embedded in a context of relevance to a recipient. In contrast to data that can be characterized as a property of things, knowledge is a property of agents (people or computer systems) predisposing them to act in particular way in circumstances defined by the context.
- Knowledge Management (KM) is about the protection, development and exploitation of knowledge assets. KM encompasses theories, models, processes and tools that support the efficient and effective evaluation, acquisition, dissemination, development and exploitation of knowledge resources in organizational processes and decision making.
- Knowledge engineering is the use of information assets regarding situations, users, products and technologies that an organization owns (or needs to develop as part of its strategic plan) and then uses to implement organizational processes efficiently and effectively.
- Knowledge Engineering Systems and Tools provide the means of combining individuals' knowledge in the pursuit of personal and organizational objectives. In the simplest form, knowledge engineering supports person-to-person and group-to-group communications resulting in the development of new knowledge via integration. These more advanced computer collaboration and data sharing methodologies can create a level and quality of exchange not normally achievable within the conventional data-sharing environment.
- Enterprises are caught in the middle of explosions with respect to government regulations, litigation, and data to be managed, all of which are occurring simultaneously. These intertwined issues require a level of knowledge, information and data management not formally required for day-to-day business at a company. In order to meet these challenges, a company must utilize resources at their highest efficiency and select tools to readily manage, engineer and integrate the data of the enterprise into a knowledge solution. The present invention addresses shortcomings in existing knowledge management solutions, and provides today's companies with tools useful for overcoming current knowledge management challenges.
- The present invention is directed a system and method for obtaining information responsive to a legal discovery request directed at an enterprise. The legal discovery request corresponds to a document subpoena directed to the enterprise or a request for production of documents served on the enterprise in litigation. A plurality of databases is associated with the enterprise. The databases potentially include information responsive to the legal discovery request, and include a plurality of structured databases and at least one unstructured database. A plurality of taxonomies are provided, each of which describes a structure of one of the structured databases. An ontology is provided, based at least in part on the taxonomies, that describes data relationships between the plurality of databases associated with the enterprise. A native application is identified, which is configured to access at least one but not all of the plurality of databases associated with the enterprise. The native application is modified to include search functionality for accessing all of the plurality of databases associated with the enterprise. A request is received from the user, via the modified native application, to search the plurality of databases associated with the enterprise for information responsive to the legal discovery request. Information included in the request, the ontology and the taxonomies are then used to search the plurality of databases associated with the enterprise. A list of search results responsive to the search request is presented to the user with the modified native application. In response to a user selection of a search result from the list, an application required to view the selected search result is automatically launched if the modified native application lacks functionality required to display the selected search result.
- In accordance with a further aspect, the present invention may be implemented in the context of a software application that is currently available in a current version and which was previously available in an old version, where the current version of the software stores data in a current data storage format and the old version of the software previously stored in an old data storage format which is different from the current data storage format. A data metamorphosis system and method in accordance with the present invention is used in this context to support usage of data stored in the old data storage format with the current version of the software. In accordance with the invention, an XML (eXtended Markup Language) definition is provided for each of a plurality of object classes used to store data in the old data storage format and the new data storage format, and data format rules are provided for each XML definition. When the current version of the software recalls data in the old data storage format, at least some of the XML definitions and format rules are used to dynamically convert the data in the old storage format to the current data storage format, thereby allowing usage of the data in the old storage format by the current version of the software.
- In accordance with a still further aspect, the present invention is directed to a system and method of providing a data collaboration environment that changes dynamically based on the occurrence of events external to the data collaboration environment. The collaboration environment is defined to include roles, activities and external event specifications. Each role defines data access privileges granted to a user during a session, each activity defines with whom the user can share data, and each external event specification defines an event external to the collaboration environment which must occur before an activity is permitted for a defined role during a session. Data sharing privileges granted to a user change dynamically in response to the occurrence of an external event identified in one of the defined external event specifications.
- In accordance with yet a further aspect, the present invention is directed to a system and method for collecting and retrieving data for an enterprise with a plurality of databases including a plurality of structured databases and at least one unstructured database. A plurality of taxonomies is provided, each of which describes a structure of one of the structured databases. An ontology is provided, based at least in part on the taxonomies, that describes data relationships between the plurality of databases of the enterprise. A software agent is associated with each of the plurality of databases of the enterprise. Each software agent uses information included in a search request, the ontology and the taxonomies to search an associated database and generate search results. Each software agent duplicates and archives data retrieved from the associated database based on the search results, and wraps the retrieved data with a standardized object format that facilitates data transformation and secure access of the retrieved data at a field level. Each software agent also logs events that occur with respect to an application that corresponds to the database associated with the software agent. The events logged by the software agent include, for example, server status, server operations performed, and network status.
- In accordance with another aspect, the present invention is directed to a network having a plurality of nodes, where nodes enter and exit the network in a manner that is unpredictable. Data availability is improved in the network by fragmenting data across the network nodes in order to increase a likelihood that the data will be available irrespective of a combination of nodes that are connected to the network at any given time. Each fragment of data includes a header with control information that specifies how the fragment connects with other fragments stored in the network.
-
FIG. 1 is a flow diagram depicting knowledge flow or data flow through the repository of the present invention. -
FIG. 2 is a block diagram showing program components and logical storage areas used for implementing the search technology of the present invention. -
FIG. 3 is a flow diagram illustrating steps performed by the search technology of the present invention. -
FIG. 4 is an exemplary screen shot illustrating a native application which has been modified to include a Search Interface for accessing the search functionality of the present invention. -
FIG. 5 is a further exemplary screen shot illustrating a native application which has been modified to include the Search Interface for accessing the search functionality of the present invention. -
FIG. 6 is a still further exemplary screen shot illustrating a native application which has been modified to include the Search Interface for accessing the search functionality of the present invention. -
FIG. 7 is a graphical user interface for mapping data between a current version of a software application which stores data in a current data storage format and an old version of the software which previously stored the data in an old data storage format. -
FIG. 8 is a further example of the graphical user interface for mapping data between a current version of a software application which stores data in a current data storage format and an old version of the software which previously stored the data in an old data storage format. -
FIG. 9 is a graphical user interface depicting certain core functions that the user can select in order to map data between a current version of a software application which stores data in a current data storage format and an old version of the software which previously stored the data in an old data storage format. -
FIG. 10 is a graphical user interface depicting a user-defined function (created from the core functions ofFIG. 9 ) developed by a user for mapping data between a current version of a software application which stores data in a current data storage format and an old version of the software which previously stored the data in an old data storage format. -
FIG. 11 is a graphical user interface depicting a visual function builder for mapping data between a current version of a software application which stores data in a current data storage format and an old version of the software which previously stored the data in an old data storage format. -
FIG. 12 depicts an XML Schema generated using the user-interface shown inFIGS. 7-11 . -
FIG. 13 is a block diagram showing the components of a data collector agent of the present invention. -
FIG. 14 is a flow diagram illustrating steps performed by the data collector agent of the present invention. -
FIG. 15 is a block diagram illustrating the object transmission format used by the data collector agent for transmitting information. -
FIG. 16 is a flow diagram illustrating steps performed for fragmenting data in order to improve availability in accordance with the present invention. -
FIG. 17 is a block diagram illustrating a fragmented object store header format, in accordance with the present invention. - A. Overview
- In one embodiment, the present invention is implemented as an object-oriented, agent-based scalable digital repository that creates a storage virtualization layer over any combination of storage devices, databases or other knowledge sources. The invention combines aspects of relational, object-oriented and XML databases to create a content integrating repository. The invention uses data mining methods and graphical interface tools, and interfaces with most disparate enterprise data sources requiring only point-and-click methods during installation. Other ‘content integrating’ knowledge managers require substantial entry of SQL code and XML in order to accomplish the connection to the enterprises' disparate data sources.
- Certain embodiments of the present invention also include the capability to dynamically metamorphose data between storage locations within the repository or upon retrieval from the repository to an application. Data is stored in an open format and searchable metadata indices are automatically generated for all data inserted into the repository. The indices can be stored separately from the data and can be interfaced to archive librarian utilities to manage and control both on-line and off-line data.
- Existing data may be made accessible to the present invention by describing the format and structure of the data via XML oriented methods and then having the invention scan and build metadata indices. New data may be inserted into the repository via API calls or using a utility program that pulls data directly from the data source without altering existing programs.
- Policy Templates control the metadata index content and determine where a particular data class will be stored within the repository. Support for Information Lifecycle Management and Digital Rights Management are provided in certain embodiments.
- The software of the present invention preferably includes the following functionality:
- 1. The software supports interfaces to enterprise databases and applications, plus web service middleware such as Web Sphere.
- 2. The software binds tightly with applications. This ability is important to compliance-oriented solutions or to solutions where creations of an audit trail and secure storage on data is required. This is performed via a module that binds tightly to the data-source application (such as an Exchange Mail Server) and records all activity of the server as well all executed data transactions.
- 3. The software preferably includes an auto-archiving ability. Through policy definition, data can be automatically moved off-line or to other storage locations. As time passes some data no longer is actively used in the enterprise. This data is archived or placed on storage that no longer allows modification of the data. This media can be, but is not limited to, certain types of disk drives or tape storage. The software allows the data to be moved to off-line storage while maintaining metadata indices online with the location of all the data. Off-line refers to storage media that requires human or mechanical intervention in order to access the data via means of a computer (tape stored on a shelf must be mounted in a tape drive by an operator). This maintains database performance and levels and reduces resource utilization as information accumulates. Compliance data may be moved offline, unless used to support the findings of an audit, after, e.g., a pre-set period of time such as three years, but be able to be located on-line for a longer period, e.g., seven years (per Sarbanes-Oxley and SEC Rule 17). The software allows the data to be moved to off-line storage while maintaining the metadata indices online with the location of all the data. This is performed without explicit user action via policy. The software also supports the ‘marking’ of audit/case support information causing it to remain online per compliance requirements so that it can be easily is retrieved as supporting information.
- 4. The software requires no changes to existing data sources or applications for their integration into the repository used by the present invention.
- 5. The software preferably meets the stringent requirements of the rules and regulations recently established for compliance with Sarbanes-Oxley, HIPPA, SEC and other regulators. In one embodiment, the software offers an auditable, secure chain of evidence from data generation at the data source to destruction of the data per policy. The data is always stored in the archive in the original data format. The software preferably supports various aspects of compliance support, including the ability to extend the on-line and destruction times of selected data items per regulations should they support audit findings. In order to accomplish these objectives, the software uses separate meta-data indices from the data per requirements, and provides a packaging function that will copy the metadata and source data to a specified destination (CD-ROM, DVD, Disk Drive, etc.) with (optionally) a search engine to aid the receiver of the knowledge in processing of the data.
- 6. The software supports Digital Rights Management (DRM) of the data deposited in its repository. Policies are used to control the application of where and how DRM is applied within the repository. The software also preferably supports use of the XML standards for Digital Rights Management including (i) DPRL—Digital Property Rights Language, (ii) XrML—Extensible Rights Markup Language, and (iii) XACML—Extensible Access Control Language.
- 7. The software supports packaged and custom applications, databases (Oracle, DB2, SQL Server, Informix, and Sybase), servers and storage classes (SAN/DAS. CAS. Tape), and is configurable for single companies, multiple divisions, multinational organizations, or ASPs supporting any combination of organizational types. The software includes policies and templates that may be configured to control each organizational unit's storage affinity and retention periods separately or in an aggregate form. The software also interfaces transparently with collaboration software such as Lotus Notes and Microsoft Exchange.
- 8. The software is based on a Highly Available Architecture (HAA) that assures data being deposited or withdrawn from the repository is delivered and correct. Automatic synchronization and healing of virtualized storage areas is automatically performed per policies and rules after a partitioned network is healed.
- 9. The software enables the transparent integration of data from diverse and disparate sources.
- 10. The software is based on a Hierarchical Index Architecture that can handle any data type and any number of entries. This is enabled by using a hierarchy in the index structure that allows a distributed method for the index. The index or metadata can be replicated like data in a database, and assigned affinity to geographical locations or logical locations. This allows an index to be generated near the data source and then ‘distributed’ for search operations via the network according to policy. The index can be merged into a central index or intelligently summed to allow the rapid locating of those distributed data sources which contain items of interest to be recalled.
- 11. The software supports policy definition for retention and has the ability to manage data from non-structured sources such as email and Instant Messaging also via policy. Any source of data (document scanners, fiche readers, 5ESS switch, etc.) can become an information provider to the repository of the present invention.
- 12. The software provides a unified view of repository data growth for the monitoring and control of the all storage areas and types that make up the repository. Views by location, data classes, storage type, etc. are possible. An added feature is to create knowledge relationships based upon chained events. This allows the creation of alerts or other defined actions that require correlation of a complex series of events. For example, a keyword match of selected words in an email, followed by a phone call to a particular number, then ending with a stock trade could be a possible alert. The software provides the ability for charge-back information to be supplied to budgetary programs and applications.
- 13. The software has a language semantic engine that improves the ‘fuzzy’ search results beyond the stemming methods used in other search processing engines. The software utilizes an analytic engine that employs: (i) prefix/suffix recognition (which masks certain semantic components), (ii) text segmentation (which provides improved recall for non-white space languages), (iii) component segmentation (which breaks words into linguistic constituents, e.g., paragraph, sentence, word, compound word, URLs, etc.), (iv) normalization (for handling inflection differences (mouse/mice), (v) variant spelling matching (e.g., Color=Colour), and (vi) lemmatization which matches conceptually similar words (‘demonstrate’=‘demonstration’ but does not match ‘demon’).
- 14. The software automatically builds searchable metadata indices based upon policy templates for particular data sources or classes as data is inserted. It allows for the creation of multiple indices to group similar data for rapid searches and retrieval. The software allows for the scanning of existing data for inclusion into the metadata indices. The metadata is gathered from sources controlled via policy templates. Compliance specific features such as keyword controls, attachment scanning plus indexing, and linking of receipts, etc., for email and IM are provided along with other features.
- 15. The software has the ability to dynamically transform data upon retrieval. Using the taxonomy and ontology information contained in a knowledge encyclopedia, the software combines this information with specific transformation rules to convert the data as necessary to meet the requirements for the information to be utilized by the retrieving application.
- 16. The software supports a multiple language ability for serving international enterprises that exchange data freely across borders. The search engine supports queries with intelligence to improve the relevancy of the search results across different languages.
- 17. The software uses an object-oriented digital repository. This allows integration between the application logic and the archived data to be the most transparent possible, this is known as transparent persistence. The ability to keep the persistence and application logic in the same object-oriented paradigm simplifies modeling, design tool requirements, and visualization of the system architecture and design. The persisting may apply not only to data but also the entire object, including its implemented behavior. This supports calling the persisted object's methods on a remote database server, thereby providing advantages in scalability and distribution. In a relational database, by contrast, stored procedures or COM objects would need to be implemented to accomplish this result, forcing some duplication of coding efforts or a more awkward architecture. The present invention reduces development time by allowing developers to focus on object persistence, not the decomposition of objects to rows and parent/child relationships in one or more tables. In most cases, a developer simply saves an object to the repository.
- 18. The software utilizes a highly scalable architecture that provides the ability to handle the demands of enterprise applications and distributed native data access. The software uses optimization of queries and transparent dynamic caching methods to deliver users a non-disruptive, responsive approach to enterprise knowledge search and retrieval. The software scales by the simple addition of more agents to handle increased transactional loads. The agents utilize network traffic monitoring, node loading measures, and transaction counts creating a feedback loop that enables intelligent decision making about when, where and if additional agents are needed. The software can also generally throttle operations in order to maintain throughput on other critical applications operating in the network. The performance control mechanism is also capable of throttling only certain types of operations based upon type, user, group, data of day, node, network segment, etc.
- 19. The software is hardware/software platform neutral or agnostic. The software runs on mainframes to embedded platforms and supports operation on z/OS, z/VM, OS/400, AIX, Linux, Solaris, Windows and etc.
- 20. The software provides for support of policy or template-based control of data retention. It also uses policies to control data affinity or the ability to direct selected data types to particular storage types or physical locations. The policy allows for the replication of data among locations and storage types. For example, using the metamorphose ability with replication it is possible to replicate database tables with unlike structures by establishing policy rules on how to convert the data.
- 21. The software leverages the abilities and functionality of the databases and storage technology that it interfaces with during virtualization of data sources. The indexing and storage processes can be tasked to the database and storage technology that delivers the best performance for that data or information type.
- 22. The software uses available technologies to locate and retrieve knowledge from the repository. The software's unique architecture allows the user to leverage the data mining abilities, unique search abilities and other features of the database and applications it utilizes as data sources. The software includes features that allow the specification of external programs, routines, etc. for specific data search problems. For example, the software can be extended to annotate and process video streams real-time to create easily searchable video knowledge repositories. The default search methods implemented in the software use the latest ranking and fuzzy search technology to enable the user the greatest ease of use.
- 23. The software supports basic and advanced security modes of operation. From simple password based protection to advanced Digital Rights Management methods on individual data items fields. The software also uses compressed, encrypted communications (selectable by policy) for movement of data within the archive or between storage locations. This is also used to create a true chain of evidence for data relating to compliance. The ‘chain of evidence’ provides a secured log of all types of access attempted or performed on any information deposited within the repository.
- 24. The software supports XML open standards and eliminates the risk of application obsolescence. Data remains accessible even after the application is retired. In addition, the software will metamorphose data per a defined policy during retrieval to support the current version of an application or other data formatting requirements of a retrieval destination. The software is in compliance with ISO 14721 and DoD 5015.2 for Digital Archives.
- 25. The software provides transparency such that all data sources appear and act as one data source. The software can access archived data through existing application interfaces, online, and in real time. In addition, offline data requests are queued via librarian utilities or via operator interfaces and managed without user intervention. Off-line (long-term archived items) can be queued for retrieval and then presented to the requesting user once they have placed on-line either in the on-line archive or in a temporary cache per policy.
- 26. The software allows data items to be versioned in the repository. This keeps a complete audit trail of the data item thought all of its changes. Policies are used to determine the version number allowed, expiration dates, always keep the original and other options. Compliance data is preferably never modified, and CRC or MD5 signature methods are used on the data to ensure this does not happen. Encryption via other algorithms is also available.
- B. System Architecture
- Referring now to
FIG. 1 , a flow diagram is shown depicting knowledge flow or data flow through the repository of the present invention. Corporate compliance data from different sources within an enterprise is generated from a source application, device or database (e.g.,VoIP application 102,e-mail application 104, statement/report application 106,image database 108,transaction database 110,internet forms database 112, or video database 114) and deposited tostorage repository 100 using XML based data descriptions and policies. A list of different source data formats that may be generated by source applications and deposited intorepository 100 are shown in Appendix A. The different data sources within the enterprise may include structured databases and unstructured databases. - Referring still to
FIG. 1 ,storage areas repository 100. NAS/SAN arrays, databases or any other device may be used for storage. Data that has been deposited intorepository 100 may also be withdrawn. Unlike other digital repositories tasked for compliance issues, the software of the present invention is capable of returning the data to the source application, device or database. This allows users to access and work with data inrepository 100 utilizing an interface on a native application (i.e., an interface on application/device/database -
FIG. 2 is a block diagram showing the program components and logical storage areas used for implementing the search technology of the present invention. The architecture shown includes four program components (data collector agents 202,data storage agents 204, management andcontrol agents 206, and process flow management agent 208) and utilizes six different logical storage areas (discussed below) to perform its functions.FIG. 2 shows the functional deployment in a network of the program and storage area components of the present invention. Multiple instances of each the components may exist in a network to provide scalability, reliability and accessibility to the data inrepository 100. The six logical storage areas used by the architecture are: - 1. An Evidence Chain Repository (ECR) 210, which holds an audit log of all activities for a given item deposited into
repository 100. This log also includes the activities of the data source associated with a given item of data, such as when the server is taken down or purges are run, etc. - 2. A MetaTag Repository (MTR) (Knowledge Encyclopedia) 212, which holds XML based descriptions of the various native formats of the data supplied to
repository 100 bysources MTR 212 also holds (i) security templates/policies, (ii) processing templates/policies, (iii) indexing rules/requirements, (iv) taxonomy and ontology of the enterprise data, (v) data metamorphosis rules, (vi) workflow processing triggers, (vii) data correlation rules, and (viii) storage area affinity and control policies. - 3. A MetaData Repository (MDR) 214, which holds indexes to the data stored in
repository 100 or registered withrepository 100 for purposes of searching. - 4. An Object Data Repository (ODR) 216, which holds stored data.
- 5. A Workflow Repository (WCR) 218, which holds workflow process rules for targeted data classes.
- 6. An Evidence Data Repository (EDR) 220, which holds data linked and kept separate during a workflow process for purposes of isolation or easy retrieval.
- Referring now to
FIG. 3 , a flow diagram is shown illustrating various steps performed by the search technology of the present invention. Instep 302, the search technology of the present invention is invoked through a native application associated with one ofsources FIGS. 4-6 . - In
step 304, the user enters search parameters (for searching all of the plurality of data sources associated with the enterprise) into the Search Interface. In one embodiment, the search parameters represent search terms and a search strategy designed to elicit information responsive to a legal discovery request directed at an enterprise. In this embodiment, the legal discovery request corresponds to a document subpoena directed to the enterprise or a request for production of documents served on the enterprise in litigation. Next, instep 306, the search parameters are provided tometadata storage agent 204. Instep 308,metadata storage agent 204 returns to the user a search results list representative of data that (i) matches the search request, and (ii) which the user is authorized to access. An example of such a search results list is shown inFIG. 6 . - In
step 310, the user is presented with an option to narrow the search criteria by entry of additional parameters. If the user opts to narrow the search criteria, the additional parameters are input by the user via the Search Interface, and instep 312, the additional search parameters are provided tometadata storage agent 204. Instep 314,metadata storage agent 204 returns to the user a search results list representative of data that (i) matches the narrowed search request, and (ii) which the user is authorized to access. Steps 312-314 may be repeated in order to successively narrow the search, as desired by the user. - In
step 316, the user can display an item corresponding to one of the entries on the search results list (seeFIG. 6 ) by clicking on the item via the Search Interface. In response to a user “clicking on” an item in the search results list for display, the Search Interface accesses the metadata storage agent 204 (step 318), which in turn returns the location or locations of source data (which may be fromdata sources step 324, the Search Interface uses the data location information and the display method information to display the source data to the user. In the event that the native application used for implementing the Search Interface lacks functionality required for displaying the selected source data, an application required to view the selected search result is automatically launched by the Search Interface and used for displaying the selected source data to the user. - In
step 326, the user can select an item corresponding to one of the entries on the search results list (seeFIG. 6 ) in order to save the data to a storage medium for distribution. This media is not part of the repository, but a writable storage media attached to the user's computer or network that they have security authorization to write to. An example would be an in-house corporate counsel locating all activities of a given broker and then using this facility to prepare a CD-ROM to send to external counsel. - In
step 328, the Search Interface accessesmetadata agent 204 which in turn returns the location or locations of source data (which may bedata sources step 334, the Search Interface uses the data location information and the storage method information to access the source data and then store the data on a storage medium selected by the user. - A plurality of taxonomies is provided within
MetaTag Repository 212. Each of the taxonomies describes a structure of a structured database corresponding to one of the data sources (102-112). An ontology is also provided withinMetaTag Repository 212, based at least in part on the taxonomies. The ontology describes data relationships between the plurality of databases associated with the enterprise (e.g., data sources 102-112). When a search request is received from the user in step 306 (or step 312), the ontology and the taxonomies are accessed and used to search the plurality of source databases associated with the enterprise. - C. Data Metamorphosis
- It often happens that a current software application (available in a current version) was previously available in an old version, where the current version of the software stores data in a current data storage format and the old version of the software previously stored the data in an old data storage format which is different from the current data storage format. The software of the present invention includes ‘data metamorphosis’ functionality which supports usage of data stored in the old data storage format with the current version of the software. The invention includes an XML definition for each of a plurality of object classes used to store data in the old data storage format and the new data storage format, and data format rules for each XML definition. When the current version of the software recalls data in the old data storage format, at least some of the XML definitions and format rules are used to dynamically convert the data in the old storage format to the current data storage format, thereby allowing usage of the data in the old storage format by the current version of the application.
- Table I below shows a sample XML definition for customer order data stored by an old version (e.g., version 1) of an ordering system software application, and Table II below shows a sample XML definition for customer order data stored by a current version (e.g., version 7) of the ordering system software application. The present invention includes a graphical user interface (shown in
FIGS. 7-11 ) for generating the W3C organization XML definitions. These definitions in conjunction with code (format rules) are stored inMetaTag Repository 212, and provide the present invention with the ability to dynamically alter data formats during the retrieval of source data from storage locations.TABLE I Sample XML Definition Of A Customer Order From Version 1 Of The Ordering System <xsd:schema xmlns:xsd=“http://www.w3.org/2001/XMLSchema”> <xsd:annotation> <xsd:documentation xml:lang=“en”> Order System Version 1 Sample Definitiion </xsd:documentation> </xsd:annotation> <xsd:element name=“customerOrder” type=“customerOrderType”/> <xsd:element name=“comment” type=“xsd:string”/> <xsd:complexType name=“customerOrderType”> <xsd:sequence> <xsd:element name=“shipTo” type=“USAddress”/> <xsd:element name=“billTo” type=“USAddress”/> <xsd:element ref=“comment” minOccurs=“0”/> <xsd:element name=“items” type=“Items”/> </xsd:sequence> <xsd:attribute name=“orderDate” type=“xsd:date”/> </xsd:complexType> <xsd:complexType name=“USAddress”> <xsd:sequence> <xsd:element name=“name” type=“xsd:string”/> <xsd:element name=“street” type=“xsd:string”/> <xsd:element name=“city” type=“xsd:string”/> <xsd:element name=“state” type=“xsd:string”/> <xsd:element name=“zip” type=“xsd:decimal”/> </xsd:sequence> <xsd:attribute name=“country” type=“xsd:NMTOKEN” fixed=“US”/> </xsd:complexType> <xsd:complexType name=“Items”> <xsd:sequence> <xsd:element name=“item” minOccurs=“0” maxOccurs=“unbounded”> <xsd:complexType> <xsd:sequence> <xsd:element name=“productName” type=“xsd:string”/> <xsd:element name=“quantity”> <xsd:simpleType> <xsd:restrictionbase=“xsd:positiveInteger”> <xsd:maxExclusive value=“100”/> </xsd:restriction> </xsd:simpleType> </xsd:element> <xsd:element name=“USPrice” type=“xsd:decimal”/> <xsd:element ref=“comment” minOccurs=“0”/> <xsd:element name=“shipDate” type=“xsd:date” minOccurs=“0”/> </xsd:sequence> <xsd:attribute name=“partNum” type=“SKU” use=“required”/> </xsd:complexType> </xsd:element> </xsd:sequence> </xsd:complexType> <!-- Stock Keeping Unit, a code for identifying products --> <xsd:simpleType name=“SKU”> <xsd:restrictionbase=“xsd:string”> <xsd:pattern value=“\d{3}−[A-Z]{2}”/> </xsd:restriction> </xsd:simpleType> </xsd:schema> -
TABLE II Sample XML Definition Of A Customer Order From Version 7 Of The Ordering System <xsd:schema xmlns:xsd=“http://www.w3.org/2001/XMLSchema”> <xsd:annotation> <xsd:documentation xml:lang=“en”> Order System Version 7 Sample Definitiion </xsd:documentation> </xsd:annotation> <xsd:element name=“customerOrder” type=“customerOrderType”/> <xsd:element name=“comment” type=“xsd:string”/> <xsd:complexType name=“customerOrderType”> <xsd:sequence> <xsd:element name=“customerNumber”> <xsd:simpleType> <xsd:restriction base=“xsd:positiveInteger”> <xsd:maxExclusive value=“9999”/> </xsd:restriction> <xsd:element ref=“comment” minOccurs=“0”/> <xsd:element name=“items” type=“Items”/> </xsd:sequence> <xsd:attribute name=“orderDate” type=“xsd:date”/> </xsd:complexType> <xsd:complexType name=“Items”> <xsd:sequence> <xsd:element name=“item” minOccurs=“0” maxOccurs=“unbounded”> <xsd:complexType> <xsd:sequence> <xsd:element name=“productName” type=“xsd:string”/> <xsd:element name=“quantity”> <xsd:simpleType> <xsd:restriction base=“xsd:positiveInteger”> <xsd:maxExclusive value=“100”/> </xsd:restriction> </xsd:simpleType> </xsd:element> <xsd:element name=“USPrice” type=“xsd:decimal”/> <xsd:element ref=“comment” minOccurs=“0”/> <xsd:element name=“shipDate” type=“xsd:date” minOccurs=“0”/> </xsd:sequence> <xsd:attribute name=“partNum” type=“SKU” use=“required”/> </xsd:complexType> </xsd:element> </xsd:sequence> </xsd:complexType> <!-- Stock Keeping Unit, a code for identifying products --> <xsd:simpleType name=“SKU”> <xsd:restriction base=“xsd:string”> <xsd:pattern value=“\d{3}−[A-Z]{2}”/> </xsd:restriction> </xsd:simpleType> </xsd:schema> - The XML schemes described in Tables I and II layout the actual database schema for the ordering system. In
version 7, the schema for the database has been altered to reference a second database that contains the customer information. This requires that any access toversion 1 data be converted to the new format in order to be used in the same application programs. This is accomplished by adding a method within the data definition of theschema 1 object description contained in theMetaTag Repository 212. This method is illustrated in Table III below and added based upon on XML definitions and mapping created using the graphical user interface ofFIGS. 7-11 .TABLE III <xsl:metamorphosis ; Define Data Metamorphosis xmlns:xsl=′http://www.w3.org/1999/XSL/Transform′ version=′1.0′> <xsl:condition currentVersion=”1′, targetVersion=”7”/> ; Define Trigger Condition <xsl:template match=′results′> ; Define Method <xsl:for-each select=′$shipto::name′> ;Key to search customers <xsl:databaseQuery, customerMaster, key=”name”, $Select/> ; Lookup Name <xsl:RemoveFields> : Remove Ver 1 Fields<xsl:element name=′shipTo′ type=′USAddress′/> <xsl:element name=′billTo′ type=′USAddress′/> </xsl:RemoveFields> <xsl:ReplaceFields> : Add Version 7 Fields<xsd:element name=′customerNumber′ source=”customerMaster::CustID> </xsl:ReplaceFields> </xsl:for-each> </xsl:template> </xsl:transform> - D. Object-Oriented Reactive Secure Collaboration
- As mentioned above in connection with
FIG. 3 , in response to a search request,metadata storage agent 204 returns to the user a search results list representative of data that (i) matches the narrowed search request, and (ii) which the user is authorized to access. (Step 314). In one embodiment, the user's authority to access (and/or share) data is determined in the context of a collaboration environment that changes dynamically based on the occurrence of events external to the data collaboration environment. The collaboration environment is defined to include roles, activities and external event specifications. Each role defines data access privileges granted to a user during a session, each activity defines with whom the user can share data, and each external event specification defines an event external to the collaboration environment which must occur before an activity is permitted for a defined role during a session. Data sharing privileges granted to a user change dynamically in response to the occurrence of an external event identified in one of the defined external event specifications. Table IV below sets forth definitions for roles, activities and external event specifications, in accordance with specific examples of the invention.TABLE IV Name Description Role This names the ‘role’ of the user being granted Specification access to the information contained in the system. A given individual can be assigned several ‘roles’. For example, an individual may be assigned the following roles: administrator, auditing, security; while another individual may simply be assigned the role: user. Activity Control - Complete access to system information, including Specification defining security for others for the information item, and all other functions Read - user only to read information and not alter any. Write - user may write new information, but not modify existing or read existing information Modify - User may read and alter existing information, but not create new information Meta - User may access only metadata for the information, but not the source information Delete - User may delete information Audit - Read the audit log concerning activities for the information Event Defines an external ‘condition’ such as a Specification Defense Department ‘DEFCON’ (Defense Condition). DEFCON 1 is the highest alert level (war) withDEFCON 5 being a peaceful alert level. Any‘conditions’ can be defined to cover information sharing activities. For example, an event can be defined to permit military or civilian agencies that otherwise do not share data to automatically share during emergencies or other defined events/conditions. -
TABLE V Event Description Normal Information and other security per defined policy Crime A crime has been committed, information sharing allowing Alert coordination among local law official is now allowed Disaster A natural event has occurred resulting in loss of life Alert and property, Information sharing among all levels of government with limited media access is now allowed Attack An attack has occurred resulting in loss of life and Alert property, Information sharing among all levels of government with some media access is now allowed. Some other government levels also have CONTROL level access to allow enable data sharing on a as needed basis. -
TABLE VI Example Reactive Collaboration Setup Event Role Specification Information Type Activity Level Administrator Normal Chemical Sensor Status Control Video Surveillance Data Control Personnel Status Control Tollbooth/Tunnel Bridge Control License Scans Public Announcements Control Crime Chemical Sensor Status Control Video Surveillance Data Control Personnel Status Control Tollbooth/Tunnel Bridge Control License Scans Public Announcements Control Disaster Chemical Sensor Status Control Video Surveillance Data Control Personnel Status Control Tollbooth/Tunnel Bridge Control License Scans Public Announcements Control Attack Chemical Sensor Status Control Video Surveillance Data Control Personnel Status Control Tollbooth/Tunnel Bridge Control License Scans Public Announcements Control Responder Normal Chemical Sensor Status Meta Personnel Video Surveillance Data Meta Personnel Status Meta Tollbooth/Tunnel Bridge Meta License Scans Public Announcements Read Crime Chemical Sensor Status Meta Video Surveillance Data Read Personnel Status Meta Tollbooth/Tunnel Bridge Read License Scans Public Announcements Read Disaster Chemical Sensor Status Meta Video Surveillance Data Read Personnel Status Read Tollbooth/Tunnel Bridge Read License Scans Public Announcements Read Attack Chemical Sensor Status Read Video Surveillance Data Read Personnel Status Read Tollbooth/Tunnel Bridge Read License Scans Public Announcements Read Local Agency Normal Chemical Sensor Status Modify, Write Video Surveillance Data Modify, Write Personnel Status Modify, Write Tollbooth/Tunnel Bridge Modify, Write License Scans Public Announcements Modify, Write Crime Chemical Sensor Status Modify, Write Video Surveillance Data Modify, Write Personnel Status Modify, Write Tollbooth/Tunnel Bridge Modify, Write License Scans Public Announcements Modify, Write Disaster Chemical Sensor Status Modify, Write Video Surveillance Data Modify, Write Personnel Status Modify, Write Tollbooth/Tunnel Bridge Modify, Write License Scans Public Announcements Modify, Write Attack Chemical Sensor Status Modify, Write Video Surveillance Data Modify, Write Personnel Status Modify, Write Tollbooth/Tunnel Bridge Modify, Write License Scans Public Announcements Modify, Write Other Agency Normal Chemical Sensor Status None Video Surveillance Data None Personnel Status None Tollbooth/Tunnel Bridge None License Scans Public Announcements Read Crime Chemical Sensor Status None Video Surveillance Data Read Personnel Status None Tollbooth/Tunnel Bridge Read License Scans Public Announcements Read Disaster Chemical Sensor Status Read Video Surveillance Data Read Personnel Status Read Tollbooth/Tunnel Bridge Read License Scans Public Announcements Read Attack Chemical Sensor Status Control Video Surveillance Data Control Personnel Status Control Tollbooth/Tunnel Bridge Control License Scans Public Announcements Control News Media Normal Chemical Sensor Status None Video Surveillance Data None Personnel Status None Tollbooth/Tunnel Bridge None License Scans Public Announcements Read Crime Chemical Sensor Status None Video Surveillance Data None Personnel Status None Tollbooth/Tunnel Bridge None License Scans Public Announcements Read Disaster Chemical Sensor Status None Video Surveillance Data None Personnel Status None Tollbooth/Tunnel Bridge None License Scans Public Announcements Read Attack Chemical Sensor Status None Video Surveillance Data None Personnel Status None Tollbooth/Tunnel Bridge None License Scans Public Announcements Read - E. Data Collector Agent
- Referring now to
FIG. 13 , a block diagram is shown illustrating the components of adata collector agent 202 of the present invention. Eachdata collector agent 202 includes afirst portion 202 a which is customized to the data source associated with the agent; anagent API portion 202 b, and an interface portion 202 c for interfacing with other agents. - Referring now to
FIG. 14 , a flow diagram is shown illustrating steps performed by thedata collector agent 202. Instep 1402, the agent starts-up and begins initialization. Instep 1404,data collection agent 202 sends a broadcast message to request a connection to adata storage agent 205 and, instep 1406 data collector agent waits for a connection withdata storage agent 205. The storage agent virtualizes the storage destinations for the system. The storage agent uses the MetaTag Repository, as the source for its configuration and management. The storage agent stores and retrieves data to and from the Object Data Repository and OCR. This component provides reliability and scalability to the data storage function of the invention by recovering from communications outages, network-partitioning events, and server crashes using rules defined in the MetaTag Repository. The metadata agent manages the MetaData Repository via the information in the MetaTag Repository. The metadata agent maintains the integrity of the metadata structure and ensures the ‘freshness’ of the metadata at all levels of the hierarchy. - Referring again to
FIG. 14 , once the connection is established between the data collector agent and thedata storage agent 205, data structure definitions, security and auditing parameters (corresponding to the data source associated with the data collector agent 202) are retrieved from MetaTag repository 212 (step 1408). Instep 1410,data collector agent 202 establishes a connection to the data source associated with the agent (e.g., source 102), and instep 1412 extracts metadata values which may include, without limitation, major indicies of databases, word dicitionaries for unstructured data, or time code annotations for video streams from the data source associated withagent 202. Instep 1414, theagent 202 transmits the collected metadata values to a data storage manager for collection inMetaData repository 214. If the user has requested storage of source data in repository 100 (step 1416), thenagent 202 transmits source data (from the data source associated with agent 202) to a data storage manager for collection inobject data repository 216. If auditing of the source associated withagent 202 has been specified, then insteps agent 202 collects auditing data from the data source associated withagent 202, and transmits the audit data to a data storage manager for collection inevidence chain repository 220. In one embodiment ofsteps agent 202 andevidence chain repository 220 log various events that occur with respect to the application associated with the data source such as, e.g., server status, server operations performed, and network status, application changes, stops and starts, when data is accessed, by who, when and what operations was attempted. The evidence chain repository also collects information from thedata storage agents 204 when information is accessed within therepository 100. This includes the type of access (read, write, etc.), who accessed and the time. -
FIG. 15 is a block diagram illustrating the object transmission format used by thedata collector agent 202 for transmitting information to the data storage andmetadata agents 204. - F. Improving Data Availability By Fragmenting Data Across Network Nodes
- In one embodiment, availability of a data object is improved in the present invention using a fragmented networked storage method that exploits the statistical stability of a large number of storage points. The concept improves retrievability of a data object over simple replication and is applicable to networks where nodes enter and exit the network in a manner that is unpredictable. In this storage method, the number of segments to break a data object into is specified. Thus, if two fragments (backups) are specified and 32 replicates exist for each fragment, this would correspond to a 2-times storage overhead.
- Possible availability of data object is computed in accordance with the equation below:
where, PQ is the probability that a given Data Object will be available, n is the total number of data fragments, m is the minimum count of data fragments needed to reconstruct the data object, N is the total number of network storage positions (network nodes with storage and storage agents 204), and M is the count of networked storage positions unavailable. - The probability that a given data object will be available (retrievable) is equal to the number of ways that unavailable fragments can be arranged on unreachable networked storage positions multiplied by the number of ways in which available fragments can be arranged on reachable networked storage points, divided by the total number of ways all fragments can be stored on all networked storage positions. Using this method of fragmented storage, 2 simple replicated copies yield an availability probability of 99%, while 2 fragments stored via 32 sub-fragments yield an availability probability of 99.99998%.
- Using the distributed nature of the agents described above, implementation of this fragmentation method becomes possible. Each
data storage agent 205 of the network becomes a possible storage point (or node) of one of the fragments of the data object that must be stored in a highly reliable fashion and secure. Security of the data is increased since only a fragment of the total information piece is stored at any given location. This creates a virtual ‘data shredder’ that is reversible.FIG. 16 illustrates steps performed for fragmenting data in order to improve availability in accordance with the present invention. - The
MetaTag Repository 212 for an object class has fields that denote the number of replicates and fragments for each replicated piece. Thedata manager agent 206 then handles the distribution of the data object across the nodes required to meet the specifications. Each fragment is assigned a unique identification which is held inMetaData repository 214. Themetadata repository 214 is replicated in multiple locations to assure its availability. The original data object is not deleted until verification of that data is stored according to specified requirements of the object class for replication and fragmentation. The fragmented object store object header (shown inFIG. 17 ) is structured similar to the transmission object of the agent system. As shown inFIG. 17 , each fragment of data includes a header with control information that specifies how the fragment connects with other fragments stored in the network. The Source Data Unique Identifier in combination with the Replicate ID and the Fragment ID creates a unique identifier for each fragmented piece of the data object (FIG. 16 ). For a given data object, theMetaTag Repository 212 specifies the number of replicates (RC—replicate count) and the number of fragments for each replicate (FC—fragment count). To recreate the entire data object a collection of the fragments must be accessed via thedata storage agents 204. A complete data object requires allfragments 1 to FC of any replicate 1 to RC matching the Source Data Unique Identifier. - The object can be reassembled from the fragments if the
metadata repository 214 is destroyed or unavailable by scanning eachobject data repository 216 instance and reading the header replicate/fragment IDs. A complete metadata entry can be constructed to read the entire data object once sufficient nodes have scanned theirobject data repositories 216. The process is reversible even with deliberate tampering by destruction of the metadata indices which point to the data locations. - Finally, it will be appreciated by those skilled in the art that changes could be made to the embodiments described above without departing from the broad inventive concept thereof. It is understood, therefore, that this invention is not limited to the particular embodiments disclosed, but is intended to cover modifications within the spirit and scope of the present invention as defined in the appended claims.
Appendix A ACT! (4.02-6) ADO Adobe Acrobat (PDF) ANSI Text ASCII Source Code ASCII Text ASCII Transcript AutoCAD BDE Btrieve Comma Separated Values (CSV) CompuServe E-mail dBASE (III, IV) DisplayWrite (4, 5) Eudora Excel (2.0-2000, XP, 2003) Firefox Flash FoxPro (2, 2.5) Framework 3 FTP HTML Raw Codes HTML Text Only HTTP IBM DCA/RFT IBM DB2 IBM FFT IFilters Informix Interbase JPEG EXIF Lotus 123 (3, 4, 5 96, 97) Lotus Ami Pro Lotus cc: Mail Lotus Manuscript Lotus Notes Lotus WordPro (2.0, 3.0, 97) MASS - 11PC Microsoft Access (2000, XP, 2003) Microsoft Binders Microsoft EML Microsoft Excel (2.0-5.0, 95, 97, 2000, XP, 2003) Microsoft Exchange Microsoft Internet Explorer cache Microsoft Internet Mail Microsoft HTML Help (CHM) Microsoft MHT Microsoft MSG Microsoft Outlook (97, 98, 2000, XP, 2003) Microsoft Outlook Express (4, 5, 6) Microsoft PowerPoint (4.0, 95, 97, 2000, XP, 2003) Microsoft Personal Folders Microsoft Public Folders Microsoft RTF (Rich Text Format) Microsoft SQL Server Microsoft Visio Microsoft Word DOS (2.0-5.5) Microsoft Word (2.0-6.0, 95, 97, 2000, XP, 2003) Microsoft Wordpad Microsoft Works (1.0, 2.0, 3.0, 4.0) Mozilla MP3 Multimate (3.0, 4.0, 5.0, Advantage) My SQL Netscape Mail ODBC Open Access II OpenOffice Oracle Paradox (3.x, 4.x, 2000) Professional Write Public MAPI Q & A Write Revelation Sendmail mailbox SGML Text Only Spreadsheets SQL StarOffice Sybase Thunderbird TIFF Uniplex vCard Ventura publisher mark-up VIM Wang IWP Wang WP PLUS Windows Write (1.0, 2.0, 3.0) Word Perfect 4.2, 5.0, 6, 7, 8, 10, 2000) Wordstar (3, 4, 5, 6, 2000) XML XYWrite ZIP files
Claims (10)
1. A method for obtaining information responsive to a legal discovery request directed at an enterprise, wherein the legal discovery request is selected from the group consisting of a document subpoena directed to the enterprise or a request for production of documents served on the enterprise in litigation, wherein a plurality of databases is associated with the enterprise and potentially include information responsive to the legal discovery request, said plurality of databases including a plurality of structured databases and at least one unstructured database, said method comprising:
(a) providing a plurality of taxonomies, each of which describes a structure of one of the structured databases;
(b) providing an ontology, based at least in part on the taxonomies, that describes data relationships between the plurality of databases associated with the enterprise;
(c) identifying a native application that is configured to access at least one but not all of the plurality of databases associated with the enterprise;
(d) modifying the native application to include search functionality for accessing all of the plurality of databases associated with the enterprise;
(e) receiving from a user, via the modified native application, a request to search the plurality of databases associated with the enterprise for information responsive to the legal discovery request;
(f) using information included in the request, the ontology and the taxonomies to search the plurality of databases associated with the enterprise;
(g) presenting a list of search results responsive to the search request to the user with the modified native application; and
(h) in response to a user selection of a search result from the list, automatically launching an application required to view the selected search result if the modified native application lacks functionality required to display the selected search result.
2. A system for obtaining information responsive to a legal discovery request directed at an enterprise, wherein the legal discovery request is selected from the group consisting of a document subpoena directed to the enterprise or a request for production of documents served on the enterprise in litigation, wherein a plurality of databases is associated with the enterprise and potentially include information responsive to the legal discovery request, said plurality of databases including a plurality of structured databases and at least one unstructured database, said system comprising:
(a) a plurality of taxonomies, each of which describes a structure of one of the structured databases;
(b) an ontology, based at least in part on the taxonomies, that describes data relationships between the plurality of databases associated with the enterprise;
(c) a native application that includes a modification that adds functionality for accessing all of the plurality of databases associated with the enterprise; wherein the modification includes a user interface for receiving a user request to search the plurality of databases associated with the enterprise for information responsive to the legal discovery request;
(d) a data collector agent that uses information included in the request, the ontology and the taxonomies to search the plurality of databases associated with the enterprise and generate a list of search results responsive to the search request for presentation to the user via the modified native application; and
(e) wherein the modification to the native application automatically launches an application required to view a selected search result if the modified native application lacks functionality required to display the selected search result.
3. In a software application that is currently available in a current version and which was previously available in an old version, wherein the current version of the software stores data in a current data storage format and the old version of the software previously stored in an old data storage format which is different from the current data storage format, a data metamorphosis method that supports usage of data stored in the old data storage format with the current version of the software, the method comprising:
(a) providing an XML definition for each of a plurality of object classes used to store data in the old data storage format and the new data storage format;
(b) providing data format rules for each XML definition; and
(c) when the current version of the software recalls data in the old data storage format, using at least some of the XML definitions and format rules to dynamically convert the data in the old storage format to the current data storage format, thereby allowing usage of the data in the old storage format by the current version of the software.
4. In a software application that is currently available in a current version and which was previously available in an old version, wherein the current version of the software stores data in a current data storage format and the old version of the software previously stored in an old data storage format which is different from the current data storage format, a data metamorphosis system that supports usage of data stored in the old data storage format with the current version of the software, the system comprising:
(a) at least one repository that stores an XML definition for each of a plurality of object classes used to store data in the old data storage format and the new data storage format and data format rules for each XML definition; and
(b) a software agent, coupled to the at least one repository, that uses at least some of the XML definitions and format rules to dynamically convert the data in the old storage format to the current data storage format when the current version of the software recalls data in the old data storage format, thereby allowing usage of the data in the old storage format by the current version of the software.
5. A method of providing a data collaboration environment that changes dynamically based on occurrence of events external to the data collaboration environment, the method comprising:
(a) defining a collaboration environment that includes at least roles, activities and external event specifications;
(b) wherein each role defines data access privileges granted to a user during a session;
(c) wherein each activity defines with whom the user can share data;
(d) wherein each external event specification defines an event external to the collaboration environment which must occur before an activity is permitted for a defined role during a session; and
(e) dynamically changing data sharing privileges granted to a user in response to the occurrence of an external event identified in one of the defined external event specifications.
6. A system of providing a data collaboration environment that changes dynamically based on occurrence of events external to the data collaboration environment, the system comprising:
(a) at least one repository that defines a collaboration environment by storing information that specifies at least roles, activities and external event specifications; wherein each role defines data access privileges granted to a user during a session; each activity defines with whom the user can share data; and each external event specification defines an event external to the collaboration environment which must occur before an activity is permitted for a defined role during a session; and
(b) at least one software agent, coupled to the at least one repository, that dynamically changes data sharing privileges granted to a user in response to the occurrence of an external event identified in one of the defined external event specifications.
7. A method for collecting and retrieving data for an enterprise with a plurality of databases including a plurality of structured databases and at least one unstructured database, said method comprising:
(a) providing a plurality of taxonomies, each of which describes a structure of one of the structured databases;
(b) providing an ontology, based at least in part on the taxonomies, that describes data relationships between the plurality of databases of the enterprise;
(c) providing a software agent associated with each of the plurality of databases of the enterprise, wherein each software agent uses information included in a search request, the ontology and the taxonomies to search an associated database and generate search results, and each software agent duplicates and archives data retrieved from the associated database based on the search results;
(d) wherein each software agent wraps the retrieved data with a standardized object format that facilitates data transformation and secure access of the retrieved data at a field level; and
(e) wherein each software agent logs events that occur with respect to an application that corresponds to the database associated with the software agent, wherein the events logged by the software agent include at least one of: server status, server operations performed, and network status.
8. A system for collecting and retrieving data for an enterprise with a plurality of databases including a plurality of structured databases and at least one unstructured database, said system comprising:
(a) at least one repository that stores a plurality of taxonomies, each of which describes a structure of one of the structured databases, and an ontology, based at least in part on the taxonomies, that describes data relationships between the plurality of databases of the enterprise;
(b) a software agent associated with each of the plurality of databases of the enterprise, wherein each software agent uses information included in a search request, the ontology and the taxonomies to search an associated database and generate search results, wherein each software agent duplicates and archives data retrieved from the associated database based on the search results, wraps the retrieved data with a standardized object format that facilitates data transformation and secure access of the retrieved data at a field level, and logs events that occur with respect to an application that corresponds to the database associated with the software agent:
wherein the events logged by the software agent include at least one of: server status, server operations performed, and network status.
9. In a network having a plurality of nodes, wherein nodes enter and exit the network in a manner that is unpredictable, a method for improving data availability by fragmenting data across the network nodes, the method comprising:
(a) fragmenting the data across nodes in the network in order to increase a likelihood that the data will be available irrespective of a combination of nodes that are connected to the network at any given time; and
(b) wherein each fragment of data includes a header with control information that specifies how the fragment connects with other fragments stored in the network.
10. In a network having a plurality of nodes, wherein nodes enter and exit the network in a manner that is unpredictable, a system for improving data availability by fragmenting data across the network nodes, the system comprising:
a software agent that fragments the data across nodes in the network in order to increase a likelihood that the data will be available irrespective of a combination of nodes that are connected to the network at any given time;
wherein each fragment of data includes a header with control information that specifies how the fragment connects with other fragments stored in the network.
Priority Applications (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/473,542 US20070011147A1 (en) | 2005-06-22 | 2006-06-22 | Systems and methods for retrieving data |
US12/322,534 US8112453B2 (en) | 2005-06-22 | 2009-02-03 | Systems and methods for retrieving data |
US13/291,778 US20120117152A1 (en) | 2005-06-22 | 2011-11-08 | Systems and methods for retrieving data |
US14/246,730 US9602585B2 (en) | 2005-06-22 | 2014-04-07 | Systems and methods for retrieving data |
US15/464,558 US20170193068A1 (en) | 2005-06-22 | 2017-03-21 | Systems and methods for retrieving data |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US69298705P | 2005-06-22 | 2005-06-22 | |
US11/473,542 US20070011147A1 (en) | 2005-06-22 | 2006-06-22 | Systems and methods for retrieving data |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/322,534 Continuation US8112453B2 (en) | 2005-06-22 | 2009-02-03 | Systems and methods for retrieving data |
Publications (1)
Publication Number | Publication Date |
---|---|
US20070011147A1 true US20070011147A1 (en) | 2007-01-11 |
Family
ID=37595885
Family Applications (5)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/473,542 Abandoned US20070011147A1 (en) | 2005-06-22 | 2006-06-22 | Systems and methods for retrieving data |
US12/322,534 Active 2026-07-13 US8112453B2 (en) | 2005-06-22 | 2009-02-03 | Systems and methods for retrieving data |
US13/291,778 Abandoned US20120117152A1 (en) | 2005-06-22 | 2011-11-08 | Systems and methods for retrieving data |
US14/246,730 Active US9602585B2 (en) | 2005-06-22 | 2014-04-07 | Systems and methods for retrieving data |
US15/464,558 Abandoned US20170193068A1 (en) | 2005-06-22 | 2017-03-21 | Systems and methods for retrieving data |
Family Applications After (4)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/322,534 Active 2026-07-13 US8112453B2 (en) | 2005-06-22 | 2009-02-03 | Systems and methods for retrieving data |
US13/291,778 Abandoned US20120117152A1 (en) | 2005-06-22 | 2011-11-08 | Systems and methods for retrieving data |
US14/246,730 Active US9602585B2 (en) | 2005-06-22 | 2014-04-07 | Systems and methods for retrieving data |
US15/464,558 Abandoned US20170193068A1 (en) | 2005-06-22 | 2017-03-21 | Systems and methods for retrieving data |
Country Status (2)
Country | Link |
---|---|
US (5) | US20070011147A1 (en) |
WO (1) | WO2007002412A2 (en) |
Cited By (34)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070032887A1 (en) * | 2005-07-26 | 2007-02-08 | Brother Kogyo Kabushiki Kaisha | Information management system, information processing device, and program |
US20070106653A1 (en) * | 2005-10-12 | 2007-05-10 | Yu Sun | Search engine |
US20070233708A1 (en) * | 2006-03-28 | 2007-10-04 | Andrew Baio | Accessing an events repository |
US20070260636A1 (en) * | 2006-03-28 | 2007-11-08 | Andrew Baio | Creating and viewing private events in an envents repository |
US20080065740A1 (en) * | 2006-09-08 | 2008-03-13 | Andrew Baio | Republishing group event data |
US20080065599A1 (en) * | 2006-09-08 | 2008-03-13 | Andrew Baio | Generating event data display code |
US20080301471A1 (en) * | 2007-05-31 | 2008-12-04 | Marc Demarest | Systems and methods in electronic evidence management for creating and maintaining a chain of custody |
US20080306645A1 (en) * | 2007-06-05 | 2008-12-11 | Snap-On, Incorporated | Methods and Systems for Providing Open Access to Vehicle Data |
US20090094020A1 (en) * | 2007-10-05 | 2009-04-09 | Fujitsu Limited | Recommending Terms To Specify Ontology Space |
US20090150333A1 (en) * | 2007-12-10 | 2009-06-11 | International Business Machines Corporation | In-context collaboration tool for resource management systems |
US20100023997A1 (en) * | 2008-07-25 | 2010-01-28 | International Business Machines Corporation | Method of using xpath and ontology engine in authorization control of assets and resources |
US20110040726A1 (en) * | 2007-09-17 | 2011-02-17 | Nicholas Daryl Crosbie | Layout Manager |
US20110099233A1 (en) * | 2009-10-26 | 2011-04-28 | Microsoft Corporation | Scalable queues on a scalable structured storage system |
US20110112974A1 (en) * | 2009-11-11 | 2011-05-12 | International Business Machines Corporation | Distributed Policy Distribution For Compliance Functionality |
US20110119668A1 (en) * | 2009-11-16 | 2011-05-19 | Microsoft Corporation | Managing virtual hard drives as blobs |
US8280892B2 (en) | 2007-10-05 | 2012-10-02 | Fujitsu Limited | Selecting tags for a document by analyzing paragraphs of the document |
US20140007261A1 (en) * | 2007-06-29 | 2014-01-02 | Microsoft Corporation | Business application search |
US20140258468A1 (en) * | 2013-03-05 | 2014-09-11 | Fuji Xerox Co., Ltd. | Relay apparatus, client apparatus, and computer-readable medium |
US20140280496A1 (en) * | 2013-03-14 | 2014-09-18 | Thoughtwire Holdings Corp. | Method and system for managing data-sharing sessions |
US8898113B2 (en) | 2012-11-21 | 2014-11-25 | International Business Machines Corporation | Managing replicated data |
US20140351232A1 (en) * | 2013-05-21 | 2014-11-27 | Sap Ag | Accessing enterprise data using a natural language-based search |
US20140372481A1 (en) * | 2013-06-17 | 2014-12-18 | Microsoft Corporation | Cross-model filtering |
US20150066865A1 (en) * | 2013-08-27 | 2015-03-05 | Bank Of America Corporation | Archive information management |
US9742843B2 (en) | 2013-03-14 | 2017-08-22 | Thoughtwire Holdings Corp. | Method and system for enabling data sharing between software systems |
US20180158734A1 (en) * | 2014-03-21 | 2018-06-07 | Semiconductor Components Industries, Llc | Method of separating electronic devices having a back layer and apparatus |
US10282380B2 (en) * | 2014-07-16 | 2019-05-07 | Hitachi, Ltd. | Data retrieval/distribution method and apparatus, and storage medium |
US10313433B2 (en) | 2013-03-14 | 2019-06-04 | Thoughtwire Holdings Corp. | Method and system for registering software systems and data-sharing sessions |
US10372442B2 (en) | 2013-03-14 | 2019-08-06 | Thoughtwire Holdings Corp. | Method and system for generating a view incorporating semantically resolved data values |
US20190278570A1 (en) * | 2018-03-08 | 2019-09-12 | Microsoft Technology Licensing, Llc | Annotating Features of a Resource to Facilitate Consumption in Different Computing Environments |
US10417586B2 (en) * | 2007-08-31 | 2019-09-17 | Red Hat, Inc. | Attaching ownership to data |
US20190303868A1 (en) * | 2015-01-21 | 2019-10-03 | Palantir Technologies Inc. | Systems and methods for accessing and storing snapshots of a remote application in a document |
US20210117445A1 (en) * | 2013-03-15 | 2021-04-22 | BeulahWorks, LLC | Technologies for data capture and data analysis |
US20220060433A1 (en) * | 2014-05-27 | 2022-02-24 | Magnet Forensics Investco Inc. | Systems and methods for locating application-specific data on a remote endpoint computer |
US11983189B2 (en) * | 2020-03-27 | 2024-05-14 | At&T Intellectual Property I, L.P. | Data pipeline controller |
Families Citing this family (206)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2007002412A2 (en) | 2005-06-22 | 2007-01-04 | Affiniti, Inc. | Systems and methods for retrieving data |
US8438611B2 (en) | 2007-10-11 | 2013-05-07 | Varonis Systems Inc. | Visualization of access permission status |
US8438612B2 (en) | 2007-11-06 | 2013-05-07 | Varonis Systems Inc. | Visualization of access permission status |
US8001140B2 (en) * | 2008-02-22 | 2011-08-16 | Tigerlogic Corporation | Systems and methods of refining a search query based on user-specified search keywords |
US9129036B2 (en) | 2008-02-22 | 2015-09-08 | Tigerlogic Corporation | Systems and methods of identifying chunks within inter-related documents |
US8126880B2 (en) | 2008-02-22 | 2012-02-28 | Tigerlogic Corporation | Systems and methods of adaptively screening matching chunks within documents |
US7933896B2 (en) * | 2008-02-22 | 2011-04-26 | Tigerlogic Corporation | Systems and methods of searching a document for relevant chunks in response to a search request |
US8145632B2 (en) | 2008-02-22 | 2012-03-27 | Tigerlogic Corporation | Systems and methods of identifying chunks within multiple documents |
US8078630B2 (en) | 2008-02-22 | 2011-12-13 | Tigerlogic Corporation | Systems and methods of displaying document chunks in response to a search request |
US8924374B2 (en) * | 2008-02-22 | 2014-12-30 | Tigerlogic Corporation | Systems and methods of semantically annotating documents of different structures |
US7937395B2 (en) * | 2008-02-22 | 2011-05-03 | Tigerlogic Corporation | Systems and methods of displaying and re-using document chunks in a document development application |
US8359533B2 (en) | 2008-02-22 | 2013-01-22 | Tigerlogic Corporation | Systems and methods of performing a text replacement within multiple documents |
US8924421B2 (en) * | 2008-02-22 | 2014-12-30 | Tigerlogic Corporation | Systems and methods of refining chunks identified within multiple documents |
US8001162B2 (en) * | 2008-02-22 | 2011-08-16 | Tigerlogic Corporation | Systems and methods of pipelining multiple document node streams through a query processor |
CA2721212A1 (en) * | 2008-04-20 | 2009-10-29 | Tigerlogic Corporation | Systems and methods of identifying chunks from multiple syndicated content providers |
JP4714778B2 (en) * | 2009-02-25 | 2011-06-29 | 富士通株式会社 | Archive device, conversion device, and conversion program |
US8938717B1 (en) * | 2009-03-16 | 2015-01-20 | Xilinx, Inc. | Updating an installed computer program |
CN102656553B (en) * | 2009-09-09 | 2016-02-10 | 瓦欧尼斯***有限公司 | Enterprise Data manages |
US10229191B2 (en) | 2009-09-09 | 2019-03-12 | Varonis Systems Ltd. | Enterprise level data management |
US20110061093A1 (en) * | 2009-09-09 | 2011-03-10 | Ohad Korkus | Time dependent access permissions |
US20110145210A1 (en) * | 2009-12-10 | 2011-06-16 | Negti Systems, Inc. | System and Method for Managing One or More Databases |
US10296596B2 (en) | 2010-05-27 | 2019-05-21 | Varonis Systems, Inc. | Data tagging |
CN108920502B (en) | 2010-05-27 | 2021-11-23 | 瓦欧尼斯***有限公司 | Data classification |
US8442982B2 (en) * | 2010-11-05 | 2013-05-14 | Apple Inc. | Extended database search |
US9680839B2 (en) | 2011-01-27 | 2017-06-13 | Varonis Systems, Inc. | Access permissions management system and method |
EP2668562A4 (en) | 2011-01-27 | 2015-05-20 | Varonis Systems Inc | Access permissions management system and method |
US8832126B2 (en) * | 2011-05-30 | 2014-09-09 | Google Inc. | Custodian suggestion for efficient legal e-discovery |
US9489457B2 (en) * | 2011-07-14 | 2016-11-08 | Nuance Communications, Inc. | Methods and apparatus for initiating an action |
US20130246376A1 (en) * | 2012-03-16 | 2013-09-19 | Infosys Limited | Methods for managing data intake and devices thereof |
EP2711793B1 (en) * | 2012-09-19 | 2018-10-31 | Siemens Aktiengesellschaft | Method for operating an operating device for controlling a technical installation |
US8935652B2 (en) | 2013-01-30 | 2015-01-13 | International Business Machines Corporation | System advisor for requirements gathering and analysis in web solutions |
US10565533B2 (en) | 2014-05-09 | 2020-02-18 | Camelot Uk Bidco Limited | Systems and methods for similarity and context measures for trademark and service mark analysis and repository searches |
US9965547B2 (en) * | 2014-05-09 | 2018-05-08 | Camelot Uk Bidco Limited | System and methods for automating trademark and service mark searches |
US11100124B2 (en) | 2014-05-09 | 2021-08-24 | Camelot Uk Bidco Limited | Systems and methods for similarity and context measures for trademark and service mark analysis and repository searches |
US10289867B2 (en) | 2014-07-27 | 2019-05-14 | OneTrust, LLC | Data processing systems for webform crawling to map processing activities and related methods |
US9729583B1 (en) | 2016-06-10 | 2017-08-08 | OneTrust, LLC | Data processing systems and methods for performing privacy assessments and monitoring of new versions of computer code for privacy compliance |
US10181051B2 (en) | 2016-06-10 | 2019-01-15 | OneTrust, LLC | Data processing systems for generating and populating a data inventory for processing data access requests |
US10191907B2 (en) | 2015-02-27 | 2019-01-29 | Ricoh Company, Ltd. | Legal discovery tool implemented in a mobile device |
US20160253346A1 (en) * | 2015-02-27 | 2016-09-01 | Ricoh Company, Ltd. | Legal Discovery Tool |
CN104700221A (en) * | 2015-03-18 | 2015-06-10 | 浪潮集团有限公司 | Method for acquiring mass data change logs in system in SAAS (software as a service) mode |
US10423996B2 (en) | 2016-04-01 | 2019-09-24 | OneTrust, LLC | Data processing systems and communication systems and methods for the efficient generation of privacy risk assessments |
US20220164840A1 (en) | 2016-04-01 | 2022-05-26 | OneTrust, LLC | Data processing systems and methods for integrating privacy information management systems with data loss prevention tools or other tools for privacy design |
US11244367B2 (en) | 2016-04-01 | 2022-02-08 | OneTrust, LLC | Data processing systems and methods for integrating privacy information management systems with data loss prevention tools or other tools for privacy design |
US10176503B2 (en) | 2016-04-01 | 2019-01-08 | OneTrust, LLC | Data processing systems and methods for efficiently assessing the risk of privacy campaigns |
US9892444B2 (en) | 2016-04-01 | 2018-02-13 | OneTrust, LLC | Data processing systems and communication systems and methods for the efficient generation of privacy risk assessments |
US10706447B2 (en) | 2016-04-01 | 2020-07-07 | OneTrust, LLC | Data processing systems and communication systems and methods for the efficient generation of privacy risk assessments |
US10176502B2 (en) | 2016-04-01 | 2019-01-08 | OneTrust, LLC | Data processing systems and methods for integrating privacy information management systems with data loss prevention tools or other tools for privacy design |
US9892443B2 (en) | 2016-04-01 | 2018-02-13 | OneTrust, LLC | Data processing systems for modifying privacy campaign data via electronic messaging systems |
US11004125B2 (en) | 2016-04-01 | 2021-05-11 | OneTrust, LLC | Data processing systems and methods for integrating privacy information management systems with data loss prevention tools or other tools for privacy design |
US9898769B2 (en) | 2016-04-01 | 2018-02-20 | OneTrust, LLC | Data processing systems and methods for operationalizing privacy compliance via integrated mobile applications |
US10685140B2 (en) | 2016-06-10 | 2020-06-16 | OneTrust, LLC | Consent receipt management systems and related methods |
US10169609B1 (en) | 2016-06-10 | 2019-01-01 | OneTrust, LLC | Data processing systems for fulfilling data subject access requests and related methods |
US10798133B2 (en) | 2016-06-10 | 2020-10-06 | OneTrust, LLC | Data processing systems for data-transfer risk identification, cross-border visualization generation, and related methods |
US10496846B1 (en) | 2016-06-10 | 2019-12-03 | OneTrust, LLC | Data processing and communications systems and methods for the efficient implementation of privacy by design |
US11277448B2 (en) | 2016-06-10 | 2022-03-15 | OneTrust, LLC | Data processing systems for data-transfer risk identification, cross-border visualization generation, and related methods |
US10440062B2 (en) | 2016-06-10 | 2019-10-08 | OneTrust, LLC | Consent receipt management systems and related methods |
US11343284B2 (en) | 2016-06-10 | 2022-05-24 | OneTrust, LLC | Data processing systems and methods for performing privacy assessments and monitoring of new versions of computer code for privacy compliance |
US11328092B2 (en) | 2016-06-10 | 2022-05-10 | OneTrust, LLC | Data processing systems for processing and managing data subject access in a distributed environment |
US10839102B2 (en) | 2016-06-10 | 2020-11-17 | OneTrust, LLC | Data processing systems for identifying and modifying processes that are subject to data subject access requests |
US10706176B2 (en) | 2016-06-10 | 2020-07-07 | OneTrust, LLC | Data-processing consent refresh, re-prompt, and recapture systems and related methods |
US11636171B2 (en) | 2016-06-10 | 2023-04-25 | OneTrust, LLC | Data processing user interface monitoring systems and related methods |
US10353673B2 (en) | 2016-06-10 | 2019-07-16 | OneTrust, LLC | Data processing systems for integration of consumer feedback with data subject access requests and related methods |
US11403377B2 (en) | 2016-06-10 | 2022-08-02 | OneTrust, LLC | Privacy management systems and methods |
US10776518B2 (en) | 2016-06-10 | 2020-09-15 | OneTrust, LLC | Consent receipt management systems and related methods |
US11625502B2 (en) | 2016-06-10 | 2023-04-11 | OneTrust, LLC | Data processing systems for identifying and modifying processes that are subject to data subject access requests |
US10509920B2 (en) | 2016-06-10 | 2019-12-17 | OneTrust, LLC | Data processing systems for processing data subject access requests |
US11475136B2 (en) | 2016-06-10 | 2022-10-18 | OneTrust, LLC | Data processing systems for data transfer risk identification and related methods |
US11227247B2 (en) | 2016-06-10 | 2022-01-18 | OneTrust, LLC | Data processing systems and methods for bundled privacy policies |
US10565236B1 (en) | 2016-06-10 | 2020-02-18 | OneTrust, LLC | Data processing systems for generating and populating a data inventory |
US10944725B2 (en) | 2016-06-10 | 2021-03-09 | OneTrust, LLC | Data processing systems and methods for using a data model to select a target data asset in a data migration |
US10706131B2 (en) | 2016-06-10 | 2020-07-07 | OneTrust, LLC | Data processing systems and methods for efficiently assessing the risk of privacy campaigns |
US11188862B2 (en) | 2016-06-10 | 2021-11-30 | OneTrust, LLC | Privacy management systems and methods |
US10873606B2 (en) | 2016-06-10 | 2020-12-22 | OneTrust, LLC | Data processing systems for data-transfer risk identification, cross-border visualization generation, and related methods |
US11544667B2 (en) | 2016-06-10 | 2023-01-03 | OneTrust, LLC | Data processing systems for generating and populating a data inventory |
US10452864B2 (en) | 2016-06-10 | 2019-10-22 | OneTrust, LLC | Data processing systems for webform crawling to map processing activities and related methods |
US10282559B2 (en) | 2016-06-10 | 2019-05-07 | OneTrust, LLC | Data processing systems for identifying, assessing, and remediating data processing risks using data modeling techniques |
US11416590B2 (en) | 2016-06-10 | 2022-08-16 | OneTrust, LLC | Data processing and scanning systems for assessing vendor risk |
US10776514B2 (en) | 2016-06-10 | 2020-09-15 | OneTrust, LLC | Data processing systems for the identification and deletion of personal data in computer systems |
US10289866B2 (en) | 2016-06-10 | 2019-05-14 | OneTrust, LLC | Data processing systems for fulfilling data subject access requests and related methods |
US10726158B2 (en) | 2016-06-10 | 2020-07-28 | OneTrust, LLC | Consent receipt management and automated process blocking systems and related methods |
US10678945B2 (en) | 2016-06-10 | 2020-06-09 | OneTrust, LLC | Consent receipt management systems and related methods |
US11100444B2 (en) | 2016-06-10 | 2021-08-24 | OneTrust, LLC | Data processing systems and methods for providing training in a vendor procurement process |
US10878127B2 (en) | 2016-06-10 | 2020-12-29 | OneTrust, LLC | Data subject access request processing systems and related methods |
US10997318B2 (en) | 2016-06-10 | 2021-05-04 | OneTrust, LLC | Data processing systems for generating and populating a data inventory for processing data access requests |
US11438386B2 (en) | 2016-06-10 | 2022-09-06 | OneTrust, LLC | Data processing systems for data-transfer risk identification, cross-border visualization generation, and related methods |
US10706379B2 (en) | 2016-06-10 | 2020-07-07 | OneTrust, LLC | Data processing systems for automatic preparation for remediation and related methods |
US10997315B2 (en) | 2016-06-10 | 2021-05-04 | OneTrust, LLC | Data processing systems for fulfilling data subject access requests and related methods |
US10437412B2 (en) | 2016-06-10 | 2019-10-08 | OneTrust, LLC | Consent receipt management systems and related methods |
US10346637B2 (en) | 2016-06-10 | 2019-07-09 | OneTrust, LLC | Data processing systems for the identification and deletion of personal data in computer systems |
US10708305B2 (en) | 2016-06-10 | 2020-07-07 | OneTrust, LLC | Automated data processing systems and methods for automatically processing requests for privacy-related information |
US11341447B2 (en) | 2016-06-10 | 2022-05-24 | OneTrust, LLC | Privacy management systems and methods |
US10282692B2 (en) | 2016-06-10 | 2019-05-07 | OneTrust, LLC | Data processing systems for identifying, assessing, and remediating data processing risks using data modeling techniques |
US10853501B2 (en) | 2016-06-10 | 2020-12-01 | OneTrust, LLC | Data processing and scanning systems for assessing vendor risk |
US11057356B2 (en) | 2016-06-10 | 2021-07-06 | OneTrust, LLC | Automated data processing systems and methods for automatically processing data subject access requests using a chatbot |
US11392720B2 (en) | 2016-06-10 | 2022-07-19 | OneTrust, LLC | Data processing systems for verification of consent and notice processing and related methods |
US11651104B2 (en) | 2016-06-10 | 2023-05-16 | OneTrust, LLC | Consent receipt management systems and related methods |
US11651106B2 (en) | 2016-06-10 | 2023-05-16 | OneTrust, LLC | Data processing systems for fulfilling data subject access requests and related methods |
US11157600B2 (en) | 2016-06-10 | 2021-10-26 | OneTrust, LLC | Data processing and scanning systems for assessing vendor risk |
US10353674B2 (en) | 2016-06-10 | 2019-07-16 | OneTrust, LLC | Data processing and communications systems and methods for the efficient implementation of privacy by design |
US11038925B2 (en) | 2016-06-10 | 2021-06-15 | OneTrust, LLC | Data processing systems for data-transfer risk identification, cross-border visualization generation, and related methods |
US10713387B2 (en) | 2016-06-10 | 2020-07-14 | OneTrust, LLC | Consent conversion optimization systems and related methods |
US11222142B2 (en) | 2016-06-10 | 2022-01-11 | OneTrust, LLC | Data processing systems for validating authorization for personal data collection, storage, and processing |
US10909488B2 (en) | 2016-06-10 | 2021-02-02 | OneTrust, LLC | Data processing systems for assessing readiness for responding to privacy-related incidents |
US10204154B2 (en) | 2016-06-10 | 2019-02-12 | OneTrust, LLC | Data processing systems for generating and populating a data inventory |
US10949565B2 (en) | 2016-06-10 | 2021-03-16 | OneTrust, LLC | Data processing systems for generating and populating a data inventory |
US11336697B2 (en) | 2016-06-10 | 2022-05-17 | OneTrust, LLC | Data processing systems for data-transfer risk identification, cross-border visualization generation, and related methods |
US11354435B2 (en) | 2016-06-10 | 2022-06-07 | OneTrust, LLC | Data processing systems for data testing to confirm data deletion and related methods |
US10896394B2 (en) | 2016-06-10 | 2021-01-19 | OneTrust, LLC | Privacy management systems and methods |
US10614247B2 (en) | 2016-06-10 | 2020-04-07 | OneTrust, LLC | Data processing systems for automated classification of personal information from documents and related methods |
US10949170B2 (en) | 2016-06-10 | 2021-03-16 | OneTrust, LLC | Data processing systems for integration of consumer feedback with data subject access requests and related methods |
US11416589B2 (en) | 2016-06-10 | 2022-08-16 | OneTrust, LLC | Data processing and scanning systems for assessing vendor risk |
US11238390B2 (en) | 2016-06-10 | 2022-02-01 | OneTrust, LLC | Privacy management systems and methods |
US11144622B2 (en) | 2016-06-10 | 2021-10-12 | OneTrust, LLC | Privacy management systems and methods |
US10796260B2 (en) | 2016-06-10 | 2020-10-06 | OneTrust, LLC | Privacy management systems and methods |
US10282700B2 (en) | 2016-06-10 | 2019-05-07 | OneTrust, LLC | Data processing systems for generating and populating a data inventory |
US11295316B2 (en) | 2016-06-10 | 2022-04-05 | OneTrust, LLC | Data processing systems for identity validation for consumer rights requests and related methods |
US11074367B2 (en) | 2016-06-10 | 2021-07-27 | OneTrust, LLC | Data processing systems for identity validation for consumer rights requests and related methods |
US10769301B2 (en) | 2016-06-10 | 2020-09-08 | OneTrust, LLC | Data processing systems for webform crawling to map processing activities and related methods |
US11418492B2 (en) | 2016-06-10 | 2022-08-16 | OneTrust, LLC | Data processing systems and methods for using a data model to select a target data asset in a data migration |
US11481710B2 (en) | 2016-06-10 | 2022-10-25 | OneTrust, LLC | Privacy management systems and methods |
US11354434B2 (en) | 2016-06-10 | 2022-06-07 | OneTrust, LLC | Data processing systems for verification of consent and notice processing and related methods |
US11087260B2 (en) | 2016-06-10 | 2021-08-10 | OneTrust, LLC | Data processing systems and methods for customizing privacy training |
US10803200B2 (en) | 2016-06-10 | 2020-10-13 | OneTrust, LLC | Data processing systems for processing and managing data subject access in a distributed environment |
US10181019B2 (en) | 2016-06-10 | 2019-01-15 | OneTrust, LLC | Data processing systems and communications systems and methods for integrating privacy compliance systems with software development and agile tools for privacy design |
US11416109B2 (en) | 2016-06-10 | 2022-08-16 | OneTrust, LLC | Automated data processing systems and methods for automatically processing data subject access requests using a chatbot |
US10275614B2 (en) | 2016-06-10 | 2019-04-30 | OneTrust, LLC | Data processing systems for generating and populating a data inventory |
US11210420B2 (en) | 2016-06-10 | 2021-12-28 | OneTrust, LLC | Data subject access request processing systems and related methods |
US10565161B2 (en) | 2016-06-10 | 2020-02-18 | OneTrust, LLC | Data processing systems for processing data subject access requests |
US10586075B2 (en) | 2016-06-10 | 2020-03-10 | OneTrust, LLC | Data processing systems for orphaned data identification and deletion and related methods |
US10885485B2 (en) | 2016-06-10 | 2021-01-05 | OneTrust, LLC | Privacy management systems and methods |
US11151233B2 (en) | 2016-06-10 | 2021-10-19 | OneTrust, LLC | Data processing and scanning systems for assessing vendor risk |
US10592648B2 (en) | 2016-06-10 | 2020-03-17 | OneTrust, LLC | Consent receipt management systems and related methods |
US10346638B2 (en) | 2016-06-10 | 2019-07-09 | OneTrust, LLC | Data processing systems for identifying and modifying processes that are subject to data subject access requests |
US10503926B2 (en) | 2016-06-10 | 2019-12-10 | OneTrust, LLC | Consent receipt management systems and related methods |
US10430740B2 (en) | 2016-06-10 | 2019-10-01 | One Trust, LLC | Data processing systems for calculating and communicating cost of fulfilling data subject access requests and related methods |
US10242228B2 (en) | 2016-06-10 | 2019-03-26 | OneTrust, LLC | Data processing systems for measuring privacy maturity within an organization |
US11562097B2 (en) | 2016-06-10 | 2023-01-24 | OneTrust, LLC | Data processing systems for central consent repository and related methods |
US11416798B2 (en) | 2016-06-10 | 2022-08-16 | OneTrust, LLC | Data processing systems and methods for providing training in a vendor procurement process |
US10762236B2 (en) | 2016-06-10 | 2020-09-01 | OneTrust, LLC | Data processing user interface monitoring systems and related methods |
US10235534B2 (en) | 2016-06-10 | 2019-03-19 | OneTrust, LLC | Data processing systems for prioritizing data subject access requests for fulfillment and related methods |
US11025675B2 (en) | 2016-06-10 | 2021-06-01 | OneTrust, LLC | Data processing systems and methods for performing privacy assessments and monitoring of new versions of computer code for privacy compliance |
US11301796B2 (en) | 2016-06-10 | 2022-04-12 | OneTrust, LLC | Data processing systems and methods for customizing privacy training |
US11138299B2 (en) | 2016-06-10 | 2021-10-05 | OneTrust, LLC | Data processing and scanning systems for assessing vendor risk |
US11146566B2 (en) | 2016-06-10 | 2021-10-12 | OneTrust, LLC | Data processing systems for fulfilling data subject access requests and related methods |
US11222309B2 (en) | 2016-06-10 | 2022-01-11 | OneTrust, LLC | Data processing systems for generating and populating a data inventory |
US10572686B2 (en) | 2016-06-10 | 2020-02-25 | OneTrust, LLC | Consent receipt management systems and related methods |
US11138242B2 (en) | 2016-06-10 | 2021-10-05 | OneTrust, LLC | Data processing systems and methods for automatically detecting and documenting privacy-related aspects of computer software |
US10783256B2 (en) | 2016-06-10 | 2020-09-22 | OneTrust, LLC | Data processing systems for data transfer risk identification and related methods |
US11200341B2 (en) | 2016-06-10 | 2021-12-14 | OneTrust, LLC | Consent receipt management systems and related methods |
US11294939B2 (en) | 2016-06-10 | 2022-04-05 | OneTrust, LLC | Data processing systems and methods for automatically detecting and documenting privacy-related aspects of computer software |
US11023842B2 (en) | 2016-06-10 | 2021-06-01 | OneTrust, LLC | Data processing systems and methods for bundled privacy policies |
US11188615B2 (en) | 2016-06-10 | 2021-11-30 | OneTrust, LLC | Data processing consent capture systems and related methods |
US10467432B2 (en) | 2016-06-10 | 2019-11-05 | OneTrust, LLC | Data processing systems for use in automatically generating, populating, and submitting data subject access requests |
US10706174B2 (en) | 2016-06-10 | 2020-07-07 | OneTrust, LLC | Data processing systems for prioritizing data subject access requests for fulfillment and related methods |
US10452866B2 (en) | 2016-06-10 | 2019-10-22 | OneTrust, LLC | Data processing systems for fulfilling data subject access requests and related methods |
US10585968B2 (en) | 2016-06-10 | 2020-03-10 | OneTrust, LLC | Data processing systems for fulfilling data subject access requests and related methods |
US11366909B2 (en) | 2016-06-10 | 2022-06-21 | OneTrust, LLC | Data processing and scanning systems for assessing vendor risk |
US11366786B2 (en) | 2016-06-10 | 2022-06-21 | OneTrust, LLC | Data processing systems for processing data subject access requests |
US11520928B2 (en) | 2016-06-10 | 2022-12-06 | OneTrust, LLC | Data processing systems for generating personal data receipts and related methods |
US10740487B2 (en) | 2016-06-10 | 2020-08-11 | OneTrust, LLC | Data processing systems and methods for populating and maintaining a centralized database of personal data |
US11222139B2 (en) | 2016-06-10 | 2022-01-11 | OneTrust, LLC | Data processing systems and methods for automatic discovery and assessment of mobile software development kits |
US10509894B2 (en) | 2016-06-10 | 2019-12-17 | OneTrust, LLC | Data processing and scanning systems for assessing vendor risk |
US10565397B1 (en) | 2016-06-10 | 2020-02-18 | OneTrust, LLC | Data processing systems for fulfilling data subject access requests and related methods |
US10284604B2 (en) | 2016-06-10 | 2019-05-07 | OneTrust, LLC | Data processing and scanning systems for generating and populating a data inventory |
US10496803B2 (en) | 2016-06-10 | 2019-12-03 | OneTrust, LLC | Data processing systems and methods for efficiently assessing the risk of privacy campaigns |
US11675929B2 (en) | 2016-06-10 | 2023-06-13 | OneTrust, LLC | Data processing consent sharing systems and related methods |
US10592692B2 (en) | 2016-06-10 | 2020-03-17 | OneTrust, LLC | Data processing systems for central consent repository and related methods |
US10642870B2 (en) | 2016-06-10 | 2020-05-05 | OneTrust, LLC | Data processing systems and methods for automatically detecting and documenting privacy-related aspects of computer software |
US10606916B2 (en) | 2016-06-10 | 2020-03-31 | OneTrust, LLC | Data processing user interface monitoring systems and related methods |
US10776517B2 (en) | 2016-06-10 | 2020-09-15 | OneTrust, LLC | Data processing systems for calculating and communicating cost of fulfilling data subject access requests and related methods |
US11134086B2 (en) | 2016-06-10 | 2021-09-28 | OneTrust, LLC | Consent conversion optimization systems and related methods |
US11461500B2 (en) | 2016-06-10 | 2022-10-04 | OneTrust, LLC | Data processing systems for cookie compliance testing with website scanning and related methods |
US11727141B2 (en) | 2016-06-10 | 2023-08-15 | OneTrust, LLC | Data processing systems and methods for synching privacy-related user consent across multiple computing devices |
US10416966B2 (en) | 2016-06-10 | 2019-09-17 | OneTrust, LLC | Data processing systems for identity validation of data subject access requests and related methods |
US10510031B2 (en) | 2016-06-10 | 2019-12-17 | OneTrust, LLC | Data processing systems for identifying, assessing, and remediating data processing risks using data modeling techniques |
US10607028B2 (en) | 2016-06-10 | 2020-03-31 | OneTrust, LLC | Data processing systems for data testing to confirm data deletion and related methods |
US10438017B2 (en) | 2016-06-10 | 2019-10-08 | OneTrust, LLC | Data processing systems for processing data subject access requests |
US10848523B2 (en) | 2016-06-10 | 2020-11-24 | OneTrust, LLC | Data processing systems for data-transfer risk identification, cross-border visualization generation, and related methods |
US10454973B2 (en) | 2016-06-10 | 2019-10-22 | OneTrust, LLC | Data processing systems for data-transfer risk identification, cross-border visualization generation, and related methods |
US11228620B2 (en) | 2016-06-10 | 2022-01-18 | OneTrust, LLC | Data processing systems for data-transfer risk identification, cross-border visualization generation, and related methods |
US10318761B2 (en) | 2016-06-10 | 2019-06-11 | OneTrust, LLC | Data processing systems and methods for auditing data request compliance |
US10909265B2 (en) | 2016-06-10 | 2021-02-02 | OneTrust, LLC | Application privacy scanning systems and related methods |
US11586700B2 (en) | 2016-06-10 | 2023-02-21 | OneTrust, LLC | Data processing systems and methods for automatically blocking the use of tracking tools |
US10846433B2 (en) | 2016-06-10 | 2020-11-24 | OneTrust, LLC | Data processing consent management systems and related methods |
US10565214B2 (en) | 2017-03-22 | 2020-02-18 | Bank Of America Corporation | Intelligent database control systems with automated request assessments |
US10013577B1 (en) | 2017-06-16 | 2018-07-03 | OneTrust, LLC | Data processing systems for identifying whether cookies contain personally identifying information |
WO2019023509A1 (en) * | 2017-07-27 | 2019-01-31 | OneTrust, LLC | Data processing and scanning systems for generating and populating a data inventory |
US11544409B2 (en) | 2018-09-07 | 2023-01-03 | OneTrust, LLC | Data processing systems and methods for automatically protecting sensitive data within privacy management systems |
US11144675B2 (en) | 2018-09-07 | 2021-10-12 | OneTrust, LLC | Data processing systems and methods for automatically protecting sensitive data within privacy management systems |
US10803202B2 (en) | 2018-09-07 | 2020-10-13 | OneTrust, LLC | Data processing systems for orphaned data identification and deletion and related methods |
CN111461644A (en) * | 2020-03-17 | 2020-07-28 | 国网辽宁省电力有限公司 | Audit information management and control platform |
WO2022011142A1 (en) | 2020-07-08 | 2022-01-13 | OneTrust, LLC | Systems and methods for targeted data discovery |
EP4189569A1 (en) | 2020-07-28 | 2023-06-07 | OneTrust LLC | Systems and methods for automatically blocking the use of tracking tools |
US11475165B2 (en) | 2020-08-06 | 2022-10-18 | OneTrust, LLC | Data processing systems and methods for automatically redacting unstructured data from a data subject access request |
US11436373B2 (en) | 2020-09-15 | 2022-09-06 | OneTrust, LLC | Data processing systems and methods for detecting tools for the automatic blocking of consent requests |
WO2022061270A1 (en) | 2020-09-21 | 2022-03-24 | OneTrust, LLC | Data processing systems and methods for automatically detecting target data transfers and target data processing |
WO2022099023A1 (en) | 2020-11-06 | 2022-05-12 | OneTrust, LLC | Systems and methods for identifying data processing activities based on data discovery results |
US11687528B2 (en) | 2021-01-25 | 2023-06-27 | OneTrust, LLC | Systems and methods for discovery, classification, and indexing of data in a native computing system |
WO2022170047A1 (en) | 2021-02-04 | 2022-08-11 | OneTrust, LLC | Managing custom attributes for domain objects defined within microservices |
US20240111899A1 (en) | 2021-02-08 | 2024-04-04 | OneTrust, LLC | Data processing systems and methods for anonymizing data samples in classification analysis |
WO2022173912A1 (en) | 2021-02-10 | 2022-08-18 | OneTrust, LLC | Systems and methods for mitigating risks of third-party computing system functionality integration into a first-party computing system |
WO2022178089A1 (en) | 2021-02-17 | 2022-08-25 | OneTrust, LLC | Managing custom workflows for domain objects defined within microservices |
US11546661B2 (en) | 2021-02-18 | 2023-01-03 | OneTrust, LLC | Selective redaction of media content |
WO2022192269A1 (en) | 2021-03-08 | 2022-09-15 | OneTrust, LLC | Data transfer discovery and analysis systems and related methods |
US11562078B2 (en) | 2021-04-16 | 2023-01-24 | OneTrust, LLC | Assessing and managing computational risk involved with integrating third party computing functionality within a computing system |
US11620142B1 (en) | 2022-06-03 | 2023-04-04 | OneTrust, LLC | Generating and customizing user interfaces for demonstrating functions of interactive user environments |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040024775A1 (en) * | 2002-06-25 | 2004-02-05 | Bloomberg Lp | Electronic management and distribution of legal information |
US6738760B1 (en) * | 2000-03-23 | 2004-05-18 | Albert Krachman | Method and system for providing electronic discovery on computer databases and archives using artificial intelligence to recover legally relevant data |
US20040243556A1 (en) * | 2003-05-30 | 2004-12-02 | International Business Machines Corporation | System, method and computer program product for performing unstructured information management and automatic text analysis, and including a document common analysis system (CAS) |
US6839707B2 (en) * | 2001-01-17 | 2005-01-04 | General Electric Company | Web-based system and method for managing legal information |
US20050216489A1 (en) * | 2004-03-26 | 2005-09-29 | Young Kendall G | Dynamic reference repository |
Family Cites Families (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6374336B1 (en) * | 1997-12-24 | 2002-04-16 | Avid Technology, Inc. | Computer system and process for transferring multiple high bandwidth streams of data between multiple storage units and multiple applications in a scalable and reliable manner |
JP2000112760A (en) * | 1998-10-02 | 2000-04-21 | Fujitsu Ltd | Object cooperative device |
US6922685B2 (en) * | 2000-05-22 | 2005-07-26 | Mci, Inc. | Method and system for managing partitioned data resources |
US7171379B2 (en) * | 2001-03-23 | 2007-01-30 | Restaurant Services, Inc. | System, method and computer program product for normalizing data in a supply chain management framework |
JP2002304476A (en) * | 2001-04-05 | 2002-10-18 | Hitachi Ltd | Role management type cooperative learning support system |
US6925499B1 (en) * | 2001-12-19 | 2005-08-02 | Info Value Computing, Inc. | Video distribution system using disk load balancing by file copying |
US7730063B2 (en) * | 2002-12-10 | 2010-06-01 | Asset Trust, Inc. | Personalized medicine service |
US7266645B2 (en) * | 2003-02-18 | 2007-09-04 | Intel Corporation | Reducing communication for reads and updates in distributed object systems |
US7478096B2 (en) * | 2003-02-26 | 2009-01-13 | Burnside Acquisition, Llc | History preservation in a computer storage system |
US7797381B2 (en) * | 2003-09-19 | 2010-09-14 | International Business Machines Corporation | Methods and apparatus for information hyperchain management for on-demand business collaboration |
US7904487B2 (en) * | 2003-10-09 | 2011-03-08 | Oracle International Corporation | Translating data access requests |
US7908663B2 (en) * | 2004-04-20 | 2011-03-15 | Microsoft Corporation | Abstractions and automation for enhanced sharing and collaboration |
US20050254440A1 (en) * | 2004-05-05 | 2005-11-17 | Sorrell John D | Private multimedia network |
US7734643B1 (en) * | 2004-06-30 | 2010-06-08 | Oracle America, Inc. | Method for distributed storage of data |
US7996462B2 (en) * | 2004-07-30 | 2011-08-09 | Sap Ag | Collaborative agent for a work environment |
US7549054B2 (en) * | 2004-08-17 | 2009-06-16 | International Business Machines Corporation | System, method, service method, and program product for managing entitlement with identity and privacy applications for electronic commerce |
US7814075B2 (en) * | 2004-12-30 | 2010-10-12 | Oracle International Corporation | Dynamic auditing |
US20060167838A1 (en) * | 2005-01-21 | 2006-07-27 | Z-Force Communications, Inc. | File-based hybrid file storage scheme supporting multiple file switches |
US7346734B2 (en) * | 2005-05-25 | 2008-03-18 | Microsoft Corporation | Cluster storage collection based data management |
WO2007002412A2 (en) | 2005-06-22 | 2007-01-04 | Affiniti, Inc. | Systems and methods for retrieving data |
US7739227B2 (en) * | 2007-03-01 | 2010-06-15 | At&T Intellectual Property I, L.P. | Enterprise confidential electronic data inventory systems, methods and computer program products |
US7827201B1 (en) * | 2007-04-27 | 2010-11-02 | Network Appliance, Inc. | Merging containers in a multi-container system |
-
2006
- 2006-06-22 WO PCT/US2006/024496 patent/WO2007002412A2/en active Application Filing
- 2006-06-22 US US11/473,542 patent/US20070011147A1/en not_active Abandoned
-
2009
- 2009-02-03 US US12/322,534 patent/US8112453B2/en active Active
-
2011
- 2011-11-08 US US13/291,778 patent/US20120117152A1/en not_active Abandoned
-
2014
- 2014-04-07 US US14/246,730 patent/US9602585B2/en active Active
-
2017
- 2017-03-21 US US15/464,558 patent/US20170193068A1/en not_active Abandoned
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6738760B1 (en) * | 2000-03-23 | 2004-05-18 | Albert Krachman | Method and system for providing electronic discovery on computer databases and archives using artificial intelligence to recover legally relevant data |
US6839707B2 (en) * | 2001-01-17 | 2005-01-04 | General Electric Company | Web-based system and method for managing legal information |
US20040024775A1 (en) * | 2002-06-25 | 2004-02-05 | Bloomberg Lp | Electronic management and distribution of legal information |
US20040243556A1 (en) * | 2003-05-30 | 2004-12-02 | International Business Machines Corporation | System, method and computer program product for performing unstructured information management and automatic text analysis, and including a document common analysis system (CAS) |
US20050216489A1 (en) * | 2004-03-26 | 2005-09-29 | Young Kendall G | Dynamic reference repository |
Cited By (57)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070032887A1 (en) * | 2005-07-26 | 2007-02-08 | Brother Kogyo Kabushiki Kaisha | Information management system, information processing device, and program |
US20070106653A1 (en) * | 2005-10-12 | 2007-05-10 | Yu Sun | Search engine |
US20070233708A1 (en) * | 2006-03-28 | 2007-10-04 | Andrew Baio | Accessing an events repository |
US20070260636A1 (en) * | 2006-03-28 | 2007-11-08 | Andrew Baio | Creating and viewing private events in an envents repository |
US7676449B2 (en) * | 2006-03-28 | 2010-03-09 | Yahoo! Inc. | Creating and viewing private events in an events repository |
US7668838B2 (en) * | 2006-03-28 | 2010-02-23 | Yahoo! Inc. | Providing event information to third party event applications |
US20080065740A1 (en) * | 2006-09-08 | 2008-03-13 | Andrew Baio | Republishing group event data |
US20080065599A1 (en) * | 2006-09-08 | 2008-03-13 | Andrew Baio | Generating event data display code |
US8290980B2 (en) | 2006-09-08 | 2012-10-16 | Yahoo! Inc. | Generating event data display code |
US20080301471A1 (en) * | 2007-05-31 | 2008-12-04 | Marc Demarest | Systems and methods in electronic evidence management for creating and maintaining a chain of custody |
US20080306645A1 (en) * | 2007-06-05 | 2008-12-11 | Snap-On, Incorporated | Methods and Systems for Providing Open Access to Vehicle Data |
US8918245B2 (en) * | 2007-06-05 | 2014-12-23 | Snap-On Incorporated | Methods and systems for providing open access to vehicle data |
US20140007261A1 (en) * | 2007-06-29 | 2014-01-02 | Microsoft Corporation | Business application search |
US10417586B2 (en) * | 2007-08-31 | 2019-09-17 | Red Hat, Inc. | Attaching ownership to data |
US20110040726A1 (en) * | 2007-09-17 | 2011-02-17 | Nicholas Daryl Crosbie | Layout Manager |
US20090094020A1 (en) * | 2007-10-05 | 2009-04-09 | Fujitsu Limited | Recommending Terms To Specify Ontology Space |
US9081852B2 (en) * | 2007-10-05 | 2015-07-14 | Fujitsu Limited | Recommending terms to specify ontology space |
US8280892B2 (en) | 2007-10-05 | 2012-10-02 | Fujitsu Limited | Selecting tags for a document by analyzing paragraphs of the document |
US20090150333A1 (en) * | 2007-12-10 | 2009-06-11 | International Business Machines Corporation | In-context collaboration tool for resource management systems |
US20100023997A1 (en) * | 2008-07-25 | 2010-01-28 | International Business Machines Corporation | Method of using xpath and ontology engine in authorization control of assets and resources |
US8561100B2 (en) * | 2008-07-25 | 2013-10-15 | International Business Machines Corporation | Using xpath and ontology engine in authorization control of assets and resources |
US20110099233A1 (en) * | 2009-10-26 | 2011-04-28 | Microsoft Corporation | Scalable queues on a scalable structured storage system |
US8266290B2 (en) | 2009-10-26 | 2012-09-11 | Microsoft Corporation | Scalable queues on a scalable structured storage system |
US20110112974A1 (en) * | 2009-11-11 | 2011-05-12 | International Business Machines Corporation | Distributed Policy Distribution For Compliance Functionality |
US10169723B2 (en) | 2009-11-11 | 2019-01-01 | International Business Machines Corporation | Distributed policy distribution for compliance functionality |
US20110119668A1 (en) * | 2009-11-16 | 2011-05-19 | Microsoft Corporation | Managing virtual hard drives as blobs |
US10628086B2 (en) | 2009-11-16 | 2020-04-21 | Microsoft Technology Licensing, Llc | Methods and systems for facilitating communications with storage |
US8516137B2 (en) | 2009-11-16 | 2013-08-20 | Microsoft Corporation | Managing virtual hard drives as blobs |
US10169385B2 (en) | 2012-11-21 | 2019-01-01 | International Business Machines Corporation | Managing replicated data |
US8898113B2 (en) | 2012-11-21 | 2014-11-25 | International Business Machines Corporation | Managing replicated data |
US9489412B2 (en) | 2012-11-21 | 2016-11-08 | International Business Machines Corporation | Managing replicated data |
US9110966B2 (en) | 2012-11-21 | 2015-08-18 | International Business Machines Corporation | Managing replicated data |
US20140258468A1 (en) * | 2013-03-05 | 2014-09-11 | Fuji Xerox Co., Ltd. | Relay apparatus, client apparatus, and computer-readable medium |
US20180219939A1 (en) * | 2013-03-05 | 2018-08-02 | Fuji Xerox Co., Ltd. | Relay apparatus, client apparatus, and computer-readable medium |
US10574738B2 (en) * | 2013-03-05 | 2020-02-25 | Fuji Xerox Co., Ltd. | Relay apparatus, client apparatus, and computer-readable medium |
US10958715B2 (en) * | 2013-03-05 | 2021-03-23 | Fuji Xerox Co., Ltd. | Relay apparatus, client apparatus, and computer-readable medium |
US10372442B2 (en) | 2013-03-14 | 2019-08-06 | Thoughtwire Holdings Corp. | Method and system for generating a view incorporating semantically resolved data values |
US9742843B2 (en) | 2013-03-14 | 2017-08-22 | Thoughtwire Holdings Corp. | Method and system for enabling data sharing between software systems |
US20140280496A1 (en) * | 2013-03-14 | 2014-09-18 | Thoughtwire Holdings Corp. | Method and system for managing data-sharing sessions |
US10313433B2 (en) | 2013-03-14 | 2019-06-04 | Thoughtwire Holdings Corp. | Method and system for registering software systems and data-sharing sessions |
US11921751B2 (en) * | 2013-03-15 | 2024-03-05 | BeulahWorks, LLC | Technologies for data capture and data analysis |
US20210117445A1 (en) * | 2013-03-15 | 2021-04-22 | BeulahWorks, LLC | Technologies for data capture and data analysis |
US20140351232A1 (en) * | 2013-05-21 | 2014-11-27 | Sap Ag | Accessing enterprise data using a natural language-based search |
US20140372481A1 (en) * | 2013-06-17 | 2014-12-18 | Microsoft Corporation | Cross-model filtering |
US9720972B2 (en) * | 2013-06-17 | 2017-08-01 | Microsoft Technology Licensing, Llc | Cross-model filtering |
US20170322978A1 (en) * | 2013-06-17 | 2017-11-09 | Microsoft Technology Licensing, Llc | Cross-model filtering |
US10606842B2 (en) * | 2013-06-17 | 2020-03-31 | Microsoft Technology Licensing, Llc | Cross-model filtering |
CN105431841A (en) * | 2013-06-17 | 2016-03-23 | 微软技术许可有限责任公司 | Cross-model filtering |
US20150066865A1 (en) * | 2013-08-27 | 2015-03-05 | Bank Of America Corporation | Archive information management |
US20180158734A1 (en) * | 2014-03-21 | 2018-06-07 | Semiconductor Components Industries, Llc | Method of separating electronic devices having a back layer and apparatus |
US11729118B2 (en) * | 2014-05-27 | 2023-08-15 | Magnet Forensics Investco Inc. | Systems and methods for locating application-specific data on a remote endpoint computer |
US20220060433A1 (en) * | 2014-05-27 | 2022-02-24 | Magnet Forensics Investco Inc. | Systems and methods for locating application-specific data on a remote endpoint computer |
US10282380B2 (en) * | 2014-07-16 | 2019-05-07 | Hitachi, Ltd. | Data retrieval/distribution method and apparatus, and storage medium |
US11823130B2 (en) * | 2015-01-21 | 2023-11-21 | Palantir Technologies Inc. | Systems and methods for accessing and storing snapshots of a remote application in a document |
US20190303868A1 (en) * | 2015-01-21 | 2019-10-03 | Palantir Technologies Inc. | Systems and methods for accessing and storing snapshots of a remote application in a document |
US20190278570A1 (en) * | 2018-03-08 | 2019-09-12 | Microsoft Technology Licensing, Llc | Annotating Features of a Resource to Facilitate Consumption in Different Computing Environments |
US11983189B2 (en) * | 2020-03-27 | 2024-05-14 | At&T Intellectual Property I, L.P. | Data pipeline controller |
Also Published As
Publication number | Publication date |
---|---|
US9602585B2 (en) | 2017-03-21 |
US20170193068A1 (en) | 2017-07-06 |
US20120117152A1 (en) | 2012-05-10 |
WO2007002412A3 (en) | 2007-09-20 |
US20090182715A1 (en) | 2009-07-16 |
WO2007002412A2 (en) | 2007-01-04 |
US8112453B2 (en) | 2012-02-07 |
US20140330883A1 (en) | 2014-11-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9602585B2 (en) | Systems and methods for retrieving data | |
US8914414B2 (en) | Integrated repository of structured and unstructured data | |
US9009201B2 (en) | Extended database search | |
US7610285B1 (en) | System and method for classifying objects | |
US8453159B2 (en) | Workspace system and method for monitoring information events | |
US7698303B2 (en) | System for categorizing and normalizing knowledge data based on user's affinity to knowledge | |
US8423565B2 (en) | Information life cycle search engine and method | |
US20170060856A1 (en) | Efficient search and analysis based on a range index | |
US20040267746A1 (en) | User interface for controlling access to computer objects | |
US20030065663A1 (en) | Computer-implemented knowledge repository interface system and method | |
US20040254938A1 (en) | Computer searching with associations | |
US11030054B2 (en) | Methods and systems for data backup based on data classification | |
US20100325101A1 (en) | Marketing asset exchange | |
US11914869B2 (en) | Methods and systems for encryption based on intelligent data classification | |
US20200242080A1 (en) | Methods and Systems for Natural Language Processing of Metadata | |
US11113238B2 (en) | Methods and systems for metadata tag inheritance between multiple storage systems | |
US20200242159A1 (en) | Methods and systems for event based tagging of metadata | |
US11176000B2 (en) | Methods and systems for custom metadata driven data protection and identification of data | |
Wagner | A virtualization approach to scalable enterprise content management | |
Maluf et al. | Achieving Scalability with Schema-Less Databases |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: AFFINITI, INC., NEW YORK Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:FALKENBERG, DAVID J.;REEL/FRAME:018031/0976 Effective date: 20060622 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: CYBERVORE, INC., NEW JERSEY Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:AFFINITI, INC.;REEL/FRAME:025968/0211 Effective date: 20110223 |