SAS^® Enterprise Content Categorization Add-On Modules

Features

SAS^® Industry Taxonomy Rules

• Provides an extensive suite of terms, entities and their hierarchical relationships to quick-start categorization efforts.
• More comprehensive than other prebuilt taxonomies with attributes, attribute values and SAS predefined rules.
• Available to virtually every industry and can be readily translated into more than 30 languages and dialects.
• Updates are included as part of the licensing agreement.

SAS^® Document Duplication Detection

• Designed to recognize, within a large set, which documents are similar up to a threshold of similarity.
• Configurable similarity threshold allows the system to detect versions of documents that have been substantially re-edited or to focus on documents that are only small variations of others.
• Abstract the documents from their actual format and focus on the content of the document.

SAS^® Search and Indexing

• Apply linguistic intelligence to search queries and documents at the preprocessing level to provide a more accurate and relevant search.
• Use advanced linguistics technologies such as stemming and automatic spelling correction to provide richer processing at the level of words, linguistic relations and word meanings.
• Organize information into an intuitive hierarchical directory, which encapsulates specific categories into more general categories, allowing for greater flexibility.
• Narrow down search within a category, or browse documents in the category of interest.

SAS^® Text Summarization

• Automatically summarize documents from defined specification of key concepts (e.g., anchor words or word strings).
• Natural order of key sentences describes the essence of text so it is meaningful to readers.
• Leverage existing concepts and concept taxonomies to define single concepts or relationships that are sought in the identification of key sentences, including Classifier concepts (authority lists), Regex concepts (regular expressions) and Grammar concepts (syntactic patterns).
• Documents written in different languages can be summarized while retaining the inherent meaning within the natural language of the source content. Word tokenization is dependent on the language of the materials being summarized.

SAS^® Crawler

• For web crawling, starting at a user-specified URL, the crawler follows the hyperlinks in the Web while repeatedly sending HTTP requests to simultaneously obtain corresponding HTML content and any URLs existent within that content.
• Markup matcher facility simplifies the definition of fielded content from HTML or XML documents, permitting Xpath and regular expression rules, and enabling user-defined template reuse.
• Crawl the highest-quality pages first, when the quantity of object pages is very large. Duplicates of URLs or page contents are automatically removed. Incremental crawler for updated content can also be set.
• Specify the minimum access interval for continuous downloads from each site, maximum parallel connections to each site or domain, or the maximum number of times to retry each failed HTTP request.
• Logon for cookie-supported and password-protected websites.

SAS^® Content Categorization Information Workbench

• Workflow definition tool incorporating automatic abstracting, categorization and entity extraction that is designed for indexers or editors.
• Combines human editorial review with automatic abstracting, categorization and metadata tagging.
• Provides a feedback loop to the taxonomy tool for editing the taxonomy based on the use of nodes in the taxonomy.

SAS^® Content Alerts

• Specify HTML, text or XML email alerts; use email, SMS or other means of alerts.
• Multiple alerts to the same user can be combined into a single alert.
• All alerts are encoded in an intermediate XML format for delivery processing.
• Users can specify the time when alerts are sent (time of day or as soon as possible).
• Communicate directly through the SMTP protocol to a send mail server. Automatically check for returned emails by accessing a POP server.
• Highly scalable to millions of users with a constant flow of documents.

SAS^® Text Data Language Pack

• SAS Enterprise Content Categorization ships with English and a native language if not English.
• Asian, Eastern and Western European, and Middle Eastern languages are available.