Overview

Rev.Up allows you to track all the data manipulations you do on the tenant on the Data Processing and Analysis (P&A) tab in the Jobs page. Every time you submit a task, you will see it listed on this page. Every task is an Action and sometimes logically grouped together into one Action. All the Actions are grouped together into a job. This job is called the Process and Analyze job. Hence, you will need to keep two levels in mind, Actions and Job.  

Each Job kicks off a workflow behind the scenes that is divided into four separate steps. Only after all the steps in the workflow complete successfully, all the Actions you took come into effect before your data is refreshed. You may access this page by clicking on Jobs under the Admin tab.  

Job and Actions

Let us see the different components of the Data Processing and Analysis Job. We will reference the picture below to understand the different parts of the Job.

Let us look at each component

1. Job Actions - the number of actions will be processed in a single job. This block also indicates if any Actions were not successful. In the case of a Partial Success, the Job will process the successful Actions.
2. Job Status - The picture is showing a Job in a Ready state. There are five different states:
• Ready - job is ready to be kicked off, either by on an automatic schedule or by clicking the Schedule Now button
• Running - the data processing is in-progress
• Blocked - the job cannot run because there is a job already running. You will have to wait until the running job completes
• Successful - the data was successfully updated
• Failed - one of the workflow steps has failed and hence the job has failed. When a job fails, all the actions on the job also failed. The support team will be notified, investigate the cause of failure, and help resolve it. Actions in a failed job will be carried over to the next job.

3. Schedule Now Button  - manually schedule the P&A job for the next available time. These jobs are entered into a queue and will start based on server bandwidth.
4. Actions in a Job - the Actions included in the Job and the time they occurred. Examples include:
   • Data Import - data ingested from Import Connections. The icon indicates object type. The name of the source will be shown followed by the CSV file imported under it (click to download)
   • Attribute Activation
   • Model Scoring
   • Product Spend Profile Attribute Configuration
   • Segment Edits
5. Action Status and Record Counts- A column showing if the Action was successful, partially successful, or failed. Counts related to the records found in a file and how many of those records failed or successfully uploaded. Click on the number under the "Record Failed" column to download a CSV of the records that failed to upload and an explanation of the error.
6. User Submitter - indicates what user took the action

Workflow

The work done in the job is divided into four steps. The below picture shows you an example of a Completed job (expanded view by clicking on the chevron icon for a Job). You can see the four steps in the sequence they are run.

Merging, De-duping & Matching

Here, all the Import and Delete actions are merged together. Delete actions will always run before any Import actions even if they are a part of the same job, no matter in what sequence you provide them 

If there are multiple Import Actions on the same entity, such as Accounts, the list is first de-duplicated using the Unique ID provided. Imported data is then matched to the Data Cloud to assign a D-U-N-S Number if a match is found. Learn more about Identity Resolution and the matching process here.

Analysis

In this step, all the Actions are used to bring into effect the changes to all relevant objects on your tenant. For example, if the Action was Attribute Activation, this step would analyze and generate buckets and make the new attribute visible on the My Data page. Another example, if you had loaded new Accounts, all these Accounts will be distributed into respective segments, segment counts will be re-calculated and Campaigns will be updated.    

Loading

In this step, all the data generated from the previous steps are loaded into the database on the cloud.

Scoring

In this step, all the new Accounts get Ratings and Scores. If the job changes the state of the data by more than 30%, then all the historical data is also re-scored.

Keep in mind that there is Score Segment feature that can be used to manually trigger a scoring job which will happen regardless of this Scoring step.

FAQ

What are some best practices to consider to speed up loading data in to the platform?

• Incremental updates to accounts and contacts are faster than full refresh (full replacement of records). Full refresh should be avoided as much as possible. If you have 100,000 accounts and you have 100 new accounts that needs to be added only providing the 100 new accounts is much faster than uploading the entire data set.
• Combine as many actions as possible in a single job. The platform is designed to processing large volumes of data and grouping these actions in a single job is faster than splitting these actions across multiple jobs.
• Combine smaller files in to larger files - If you have multiple small files containing account and contact data to be loaded, group them in to larger files. This reduces the pre-processing and hence improves the overall time taken to load the data
• Adding new columns takes more time than updating existing data. Try to group the actions of adding new attributes in a single job rather than spreading it over time.
• If you are adding new columns to capture transient information (e.g. account ratings from models managed outside of the platform), try to pre-create new columns ahead and use them for storing the actual values as and when the need arises. Pre-creating new columns ahead of time groups expensive process in a single job thus improving the performance of future updates

Are there any limits on executing Jobs?

• Only 1 P&A job can be manually scheduled per day. Otherwise, a P&A will automatically be scheduled every evening in the CDP.