SAS® Event Stream Processing Features

In-memory, distributed & optimized processing that scales

  • SAS Event Stream Processing server for processing millions of events per second and low-latency response times (millisecond, submillisecond). 
  • Built-in metering server for monitoring and recording event consumption for each SAS Event Stream Processing project, input window and production SAS Event Stream Processing server – speeding the collection of event consumption data.
  • Retained and aggregated data kept in memory for maximum performance.
  • Ability to take advantage of distributed grid architectures architectures, SAS Cloud Analytic Services (CAS), or on public and private cloud providers ..
  • Processing speeds can be customized with flexible thread-pool sizing, caching stores and more.
  • Includes patented, instantaneous 1+N way failover, native failover, guaranteed delivery without the use of persistence and other fault-tolerance functions to ensure successful event stream processing activity.
  • Full and open access to all event metadata.

Cloud ready

  • Containerized deployments for the cloud and the edge.
  • SAS Event Stream Processing operator framework, based on Kubernetes operator framework, delivers automation for deployment, upgrades and scalability in the cloud, public or private.
  • Amazon Kinesis adapter for streaming data source connections.
  • Prebuilt Docker containers that are used with the optional SAS Event Stream Processing operator framework for cloud-ready deployments.
  • Multitenancy-ready deployments that can be integrated with your multitenant and multiuser environments

In-stream learning model windows

  • Allows you to combine different window types to specify data stream input sources, patterns of interest and derived output actions. Streaming model windows include:
    • Train – Develop an advanced analytical model in stream and pass the resulting model updates to a score window.
    • Score – Apply the trained model to current events in stream to produce score output, as well as support for learning models that use both training and scoring together.
    • Calculate – Use with offline ASTORE models, Python code, data normalization and transformation methods, as well as learning models that bundle training and scoring together.
    • Model supervisor – Control what model to deploy, and when and where to deploy it (for example, to the score window).
    • Model reader – Integrate offline ASTORE models and publish a model to another streaming analytics window, such as the score window.

Ability to consume & connect streaming data

  • Extensive suite of data adapters and connectors for publishing and subscribing to live data streams of both structured and unstructured data, including videos and images.
  • Predefined adapters include read and write (i.e., publish and subscribe):
    • Adapter connector makes it easier to manage adapters from within a SAS Event Stream Processing project, simplifying adapter orchestration (similar to connector orchestration).
    • Amazon Kinesis
    • Apache Camel
    • Axeda
    • BACNET
    • Cassandra
    • Database ODBC: supports a variety of databases such as IBM DB2, IBM Netezza, Sybase ASE, and others. See Data Driver support for complete list.
    • Event Stream Processor
    • File/socket
    • HDAT Reader
    • HDFS
    • IBM WebSphere MQ
    • JMS
    • Kafka
    • MapR
    • Modbus
    • MQTT
    • Nurego
    • OPC-UA
    • OPC-DA
    • OSIsoft PI
    • Project Publish
    • Pylon
    • RabbitMQ
    • Rendezvous
    • REST
    • SAS Cloud Analytic Services
    • SAS data sets
    • Customized publish/subscribe APIs can also be written in C or Java.
    • SAS® LASR Analytic Server
    • Solace
    • Teradata
    • Teradata Listener
    • Tervela Data Fabric
    • TIBC
    • Timer
    • URL
    • UVC camera
    • WebSocket
    • XML/JSON file socket adapter.
  • Publish only to SAS Event Stream Processing from the following:
    • BoardReader
    • HTTP RESTful interfaces
    • Log sniffers (Oracle, Greenplum)
    • Network sniffer
    • SYSLOG
    • Twitter
    • Twitter GNIP
  • Subscribe only from SAS Event Stream Processing to:
    • SOAP
    • SMTP
  • Connectors and adapters for IoT:
    • Twitter
    • MQTT
    • OPC-UA
    • UVC
    • Pylon
    • Modbus
    • OSI PI Historian
    • BACnet gateway devices
    • Kafka
    • Cassandra (adapter only)
    • BoardReader
  • Data stream support: 
    • Kinesis
    • Hortonworks DataFlow (HDF) NiFi integration – SAS and HDF can provide immediate, streaming and deep intelligence.
  • Static data joins – Integrate static data from databases or files to enrich streaming data using ODBC database adaptor and connector support in conjunction with database drivers.
  • Teradata integration – Teradata Listener connector sends data from SAS Event Stream Processing to the Teradata Listener application. Teradata Listener ingests high-volume, real-time data streams and persists the data from those streams to Teradata, Aster or Hadoop.

Adaptable, in-stream analytics & data manipulation

  • Machine learning streaming-algorithm support lets you create scoring and learning procedural windows for various continuous learning algorithms. A combination of train and score windows are used to periodically update the model. These include:
    • Streaming linear regression.
    • Streaming logistic regression.
    • Support vector machine.
    • Density-based clustering (DBSCAN).
    • K-means clustering.
    • Recommender.
    • t-Distributed Stochastic Neighbor Embedding (t-SNE)
  • In-stream analytics packaged with SAS Event Stream Processing includes:
    • Image processing.
    • Video encoding.
    • Change detection.
    • Kalman filter.
    • Smoothing.
    • Slice operations.
    • Lag monitoring.
    • Subspace tracking.
    • Histogram
    • Fit statistics.
    • Moving relative range (MRR).
    • Pearson’s correlation.
    • Receiver operating characteristic (ROC).
    • Streaming summary (univariate statistics).
    • Segmented correlation.
    • Short-time Fourier transform.
    • Streaming speech transcription.
    • Streaming audio feature computation.
    • Text tokenization.
    • Text vectorization.
    • Weibull distribution fitting.
  • Algorithms for offline training packaged with SAS Event Stream Processing include:
    • Robust principal components analysis.
    • Bayesian network.
    • Recurrent neural networks.
    • Convolutional neural networks.
    • Deep neural networks.
  • Flexible, modular, window-driven architecture to define complex continuous queries:
    • Based on an extensive suite of interchangeable window types and operators to detect an unlimited number of patterns, correlations, computations and aggregations.
    • Prebuilt, common data quality routines are used to cleanse, standardize and filter livestream data before it’s stored, reducing downstream processing.
    • Patterns of interest can include nearly unlimited advanced analytics calculations with in-stream, machine learning k-means clusters and livestream analytical scoring.
  • Event stream windows to transform event state and data, and manipulate inbound streaming images:
    • The geofence window type allows you to track the location of objects relative to borders of a geofence. Alert when an object approaches, enters or leaves the defined geofence boundaries – and track entities within the geofence boundaries.
    • Transpose: Enables you to interchange an event’s rows as columns, or columns as rows.
    • Remove state: Facilitates the transition of a stateful part of a model to a stateless part of a model.
    • Multiple object tracker (MOT): Enables you to perform multiobject tracking (MOT) in real time.
    • Train: Model training on historical data (for accurate model development) complements high-performance analytics for at-rest data.
    • Define and customize notifications by SMS, email and other alerts as part of event stream model workflow.
  • Analytical models include SAS ASTORE, DATAStep2, DATAStep, Python and C code snippets.
  • Embeddable on gateways, edge devices, compute sticks and any existing C++ application (with dedicated thread-pool processing).

Graphical design time environment

  • Integration with SAS Model Manager provides faster, automated integration and monitoring of analytical models. Browse SAS Model Manager repository to easily locate and integrate advanced analytics to embed in SAS Event Stream Processing projects.
  • Added SAS Viya enabled mode.
    • Authentication with SAS Logon.
    • Uses Postgres as persisted data store.
  • Multitenancy-enabled using new Kubernetes operator framework and Docker containers delivered for ESP server in the cloud.
    • Database per tenant.
    • Schema per tenant.
  • Test mode enhancements.
    • Updated UI.
    • Access to ESP server log.
  • Run projects outside test mode.
    • Testing for long-running projects.
  • Usability and user experience improvements.
    • Application layout updated.
    • Expression validation within Compute, Join, Filter and Aggregation window.
    • Configurable ESP cloud server instance for test mode
    • More responsive UI.
    • Diagram improvements.
    • New test server management page.
    • Improved output schema panel.
    • Enhanced validation of ESP syntax and properties.
  • Integration with SAS Event Stream Manager.
    • Published project surfaced in SAS Event Stream Manager automatically.
    • Minor version updates in SAS Event Stream Manager pushed back to SAS Studio.

ESP operations & monitoring

  • Multitenancy support.
    • Database per tenant.
    • Schema per tenant.
  • ESP server monitoring.
    • Dynamic ESP cloud server management and configuration.
    • ESP heartbeat monitoring.
    • ESP server status reporting.
    • ESP server performance stats.
  • Enhanced metering.
    • More options for usage reporting.
    • Breakdown by license/server/type.
    • Breakdown by month/year.
  • Usability and user experience improvements.
    • More visual indicators of issues.
    • More responsive UI.
    • Improved deployment management with alerts from SAS Model Manager for new champion models.
    • Enhanced server config reporting.
  • Deployment controls.
    • Load and start projects without templates.
    • Stop and Unload projects without templates.
  • Integration with ESP Studio.
    • Published project surfaced in ESM automatically.
  • With unified project and server management via SAS Event Stream Manager, you can:
    • Construct and manage repeatable deployment plans with an easy-to-use interface, for projects executing on SAS Event Stream Processing servers on bare metal or in the cloud.
    • Quickly create deployments to monitor collections of servers and to simplify management.
    • Identify deployment errors and retry operations only on servers that need attention.
    • Create filtered lists of SAS Event Stream Processing servers to apply deployment operations.
    • Create repeatable deployment scripts for rapid automation and user prompts, simplifying SAS Event Stream Processing project activation.
    • Monitor events consumed with metering server displays to identify event usage patterns per license.
    • Easily add new SAS Event Stream Processing servers for improved monitoring.

Expanded deployments & open source support

  • Deploy SAS Event Stream Processing at the edge for IoT applications:
    • SAS Event Stream Processing for Edge Computing provides a smaller, configurable disk footprint for simplified deployment to smaller edge devices.
    • Support for Docker container deployment for SAS Event Stream Processing for Edge Computing for standardized deployment.
  • SAS Event Stream Processing Python development interface:
    • Speed development time with a familiar, open and flexible Python interface for developing, publishing, testing and streaming events through SAS Event Stream Processing projects.
    • Python publish/subscribe API – Publish events and subscribe to SAS Event Stream Processing using Python.

Use SAS & open source languages

  • C/C++.
  • SAS DATAStep2.
  • MapR streams support – The Kafka adapter is certified to work with the MapR converged data platform for publishing and subscribing.
  • Python publish/subscribe API.
  • SAS Event Stream Processing for SAS® Viya® and CAS – Deploy SAS Viya data mining and machine learning models to SAS Event Stream Processing for streaming analytics.

Visual monitoring of event streams

  • Support for visualizing streaming data and ESP project insights includes:
    • Secure access with log-on support for authorized access to SAS ESP Streamviewer application.
    • Streamviewer support for SAS graphs in real-time streaming dashboards.
    • Streamviewer components support embedding within applications to deliver Streamviewer real-time insights.
    • WebSocket support for reliable and fast SAS Event Stream Processing server connectivity.
    • User configurable dashboard for customized testing of streaming activity.
    • Interactively filter and query livestream activity to examine specific behavior of elements.
    • Faster response with new web socket support to monitor events from SAS Event Stream Processing server.
    • Compare historical activity with current processing using graphical representations.
    • Monitor stream processing detail by subscribing to events of interest.

Back to Top