Manage ETL settings
Learn how to manage ETL settings for normalizing and transferring data to SAS Customer Experience Targeting.
To manage the ETL settings for normalizing and transferring data to SAS Customer Experience Targeting:
- Navigate to Data Collection > Data Collection Settings > ETL Settings.
- Review and modify the settings in the Reprocessing Settings section:
Remove data when reprocessing is detected.
Select Yes to enable the ETL code to remove the reprocessed data automatically.
Select No if you do not want the ETL code to remove the reprocessed data. This option is useful when you want to delete data during maintenance windows or slow periods. You might also want to roll back the database and save the time it takes to run the delete statements. When you select No, SAS Customer Intelligence 360 creates the SQL delete statements in the location that you specify. You must then run the Resume Reprocessing job when you want to alert the system that the data has been removed.
Select Conditional when you want to specify an amount of data (in days) as a threshold value. If the system has to remove less than that amount of data during reprocessing, then it does so automatically. If the system has to remove more than that amount of data, then the system does not remove it automatically and behaves as if you selected the No option.
Allow reload during initial reprocessing ETL
Select Yes to reload the data during the first ETL process. If you select No, then the first ETL process that detects data reprocessing deletes the data, and the second ETL process starts reloading data. Separating these processes gives you an opportunity to perform database maintenance after the data is deleted. Note: This option is visible only when you select Yes or Conditional for the Remove data when reprocessing is detected setting, and it disappears if you select No.
Select Use ETL date range to have reprocessing load the same data volume during each ETL process, as it would during a typical ETL process. If you want to speed up reprocessing, and you have the resources available, select Override ETL range and choose a higher volume (number of hours per single load) than would normally be loaded during an ETL process.
- Review and modify the settings in the General ETL Settings section.
Maximum hours to process during ETL
Specify the maximum number of hours of data that can be reprocessed during an ETL process.
The ETL process automatically determines the date range of data that needs to be loaded. This range covers all data from the current time to the last time data was loaded into the CXT database. If ETL processes have not been run for some time, or a project is just starting, there could be several days’ or weeks’ worth of data to process. This setting enables you to preserve resources and temporary disk space by limiting how much data is processed in a single ETL run.
CAUTION: Ensure that this setting is larger than the frequency that you are running your ETL processes. If the setting is not larger than the ETL frequency, then ETL processing falls behind and is never able to catch up and process current data.
Hour processing offset
Specify the amount of time to allow between a user’s session and when the session data can be reprocessed. Most data can be reprocessed up to the last minute, but some data items can be processed only at the end of a user’s session. This offset specifies how much time the system waits to account for users to complete their session, so that their final data values can be calculated and processed without conflicts.
- Publish your changes.
For more information about how SAS Customer Experience Targeting works with this data, see the documentation for that product.