Features List

SAS Data Preparation List

SAS Data Preparation

Data & metadata access

  • Use any authorized internal source, accessible external data sources and data held in-memory in SAS Viya.
    • View a sample of a table or file loaded in the in-memory engine of SAS Viya, or from data sources registered with SAS/ACCESS, to visualize the data you want to work with.
    • Quickly create connections to and between external data sources.
    • Access physical metadata information like column names, data types, encoding, column count and row count to gain further insight into the data.
  • Data sources and types include:
    • Amazon S3.
    • Amazon Redshift.
    • DNFS, HDFS, PATH-based files (CSV, SAS, Excel, delimited).
    • DB2.
    • Hive.
    • Impala.
    • SAS LASR.
    • ODBC.
    • Oracle.
    • Postgres.
    • Teradata.
    • Feeds from Twitter, YouTube, Facebook, Google Analytics, Google Drive, Esri and local files.
    • SAS® Cloud Analytic Services (CAS).

Data provisioning

  • Parallel load data from desired data sources into memory simply by selecting them – no need to write code or have experience with an ETL tool. (Data cannot be sent back to the following data sources: Twitter, YouTube, Facebook, Google Analytics, Esri; it can only be sourced from these sites).
    • Reduce the amount of data being copied by performing row filtering or column filtering before the data is provisioned.
    • Retain big data in situ, and push processing to the source system by including SAS In-Database optional add-ons.

    Guided, interactive data preparation

    • Transform, blend, shape, cleanse and standardize data in an interactive, visual environment that guides you through data preparation processes.
    • Easily understand how a transformation affected results, getting visual feedback in near-real-time through the distributed, in-memory processing of SAS Viya.

    Machine learning & AI suggestions

    • Take advantage of AI and machine learning to scan data and make intelligent transformation suggestions.
    • Accept suggestions and complete transformations at the click of a button. No advanced or complex coding required.
    • Automated suggestions include:
      • Casing.
      • Gender analysis.
      • Match code.
      • Parse.
      • Standardization.
      • Missing value imputation for numeric variables.
      • One hot encoding.
      • Remove column.
      • Whitespace trimming.
      • Convert column data type.
      • Center and scale.
      • Dedupe.
      • Unique ID creation.
      • Column removal for sparse data.

    Column-based transformations

    • Use column-based transformations to standardize, remediate and shape data without doing configurations. You can:
      • Change case.
      • Convert column.
      • Rename.
      • Remove.
      • Split.
      • Trim whitespace.
      • Custom calculation.
    • Support for wide tables allows for the saving of data plans for quick data preparation jobs.

    Row-based transformations

    • Use row-based transformations to filter and shape data.
    • Create analytical-based tables using the transpose transformation to prepare the data for analytics and reporting tasks.
    • Create simple or complex filters to remove unnecessary data.

    Code-based transformations

    • Write custom code to transform, shape, blend, remediate and standardize data.
    • Write simple expressions to create calculated columns, write advanced code or reuse code snippets for greater transformational flexibility.
    • Import custom code defined by others, sharing best practices and collaborative productivity.

    Multiple-input-based transformations

    • Use multiple-input-based transformations to blend and shape data.
    • Blend or shape one or more sets of data together using the guided interface – there’s no requirement to know SQL or SAS. You can:
      • Append data.
      • Join data.
      • Transpose data.

    Data profiling

    • Profile data to generate column-based and table-based basic and advanced profile metrics.
    • Use the table-level profile metrics to uncover data quality issues and get further insight into the data itself.
    • Drill into each column for column-level profile metrics and to see visual graphs of pattern distribution and frequency distribution results that help uncover hidden insights.
    • Use a variety of data types/sources (listed previously). To profile data from Twitter, Facebook, Google Analytics or YouTube, you must first explicitly import the data into the SAS Viya in-memory environment.

    Data quality processing

    (SAS Data Quality in SAS Viya is included in SAS Data Preparation)

    Data cleansing

    • Use locale- and context-specific parsing and field extraction definitions to reshape data and uncover additional insights.
    • Use the extraction transformation to identify and extract contact information (e.g., name, gender, field, pattern, identify, email and phone number) in a specified column.
    • Use parsing when data in a specified column needs to be tokenized into substrings (e.g., a full name tokenized into prefix, given name, middle name and family name).
    • Derive unique identifiers from match codes that link disparate data sources.
    • Standardize data with locale- and context-specific definitions to transform data into a common format, like casing.

    Identity definition

    • Analyze column data using locale-specific rules to determine gender or context.
    • Use identification analysis to analyze the data and determine its context, which is particularly valuable if the data or source of data is unfamiliar.
    • Use gender analysis to determine the gender of a name using locale-specific rules so the data can be easily filtered or segmented.
    • Create a unique ID for each row with unique ID generator.
    • Identify the subject data in each column with identification analysis.
    • Identify, find and sort data by tagging data with columns and tables.

    Data matching

    • Determine matching records based upon locale- and context-specific definitions.
    • Easily identify matching records using more than 25 context-specific rules such as date, address, name, email, etc.
    • Use the results of the match code transformation to remove duplicates, perform a fuzzy search or a fuzzy join.
    • Find like records and logically group together.

    System & job monitoring

    • Use integrated monitoring capabilities for system- and job-level processes.
    • Gain insight into how many processes are running, how long they’re taking and who is running them.
    • Easily filter through all system jobs based on job status (running, successful, failed, pending and canceled).
    • Access job error logs to help with root-cause analysis and troubleshooting. (Note: Monitoring is available using SAS Environment Manager and the job monitor application.)

    Data import & data preparation job scheduling

    • Create a data import job from automatically generated code to perform a data refresh using the integrated scheduler.
    • Schedule data explorer imports as jobs so they will become an automatic, repeatable process.
    • Specify a time, date, frequency and/or interval for the jobs.

    Data lineage

    • Explore relationships between accessible data sources, data objects and jobs.
    • Use the relationship graph to visually show the relationships that exist between objects, making it easier to understand the origin of data and trace its processing.
    • Create multiple views with different tabs, and save the organization of those views.

    Plan templates & project collaboration

    • Use data preparation plans (templates), which consist of a set of transformation rules that get applied to one or more sources of data, to improve productivity (spend less time preparing data).
    • Reuse the templates by applying them to different sets of data to ensure that data is transformed consistently to adhere to enterprise data standards and policies.
    • Rely on team-based collaboration through a project hub used with SAS Viya projects. The project’s activity feed shows who did what and when, and can be used to communicate with other team members.

    Batch text analysis

    • Quickly extract contents of documents, and perform text identification and extraction.