Salesforce Data Cloud Insights

You’ve configured Data Cloud, ingested and modelled data. You’ve also performed identity resolution to unify customer profiles.

Now, before you start activating this data for AI, marketing segments, etc… it’s time for leveraging Salesforce Data Cloud Insights.

That is: analysing, calculating and defining insights from all that data, visualizing it using the business intelligence tools available in Data Cloud.

Salesforce Data Cloud Insights Overview

Data Cloud comes with a lot of Insights features which allow you to define and calculate multidimensional metrics.

These intelligent metrics can be, for example:

  • Customer Lifetime Value (LTV)
  • Most Purchased Products
  • Customer Satisfaction Scores (CSAT)
  • RFM Model insights.
  • Real-time metrics

These metrics are multidimensional.

In other words, they cross check and relate data from multiple objects and fields. That is why you need Insights to work with them. A simple Formula Field in a field will not allow you to perform the powerful calculations you can do with Data Cloud Insights.

What’s more. These insights can be leveraged in different applications of the tool, use real-time data actions and much more.

Let’s look at all the types of Insights you can calculate and define, one by one.

Salesforce Data Cloud Insights Terms

Here is a table with the most common terminology that you will use when performing Insights in Data Cloud:

TermDescriptionExample
Primary KeyThe unique identifier of a record in an object. A product SKU in a product catalogue, or an Email Address in a subscriber profile.
Foreign KeyA field in a relational database table that serves as a link shared between that table and another one. A subscriber ID can be present in multiple tables, being the link to relate the info in them.
AttributeA field or information in a data stream in Data Cloud.Attributes can be, for example, a customer Email Address or Last Name.
MeasuresNumerical, quantitative fields that store aggregated number data points.Total Purchase Value, Average Order Value.
DimensionsQualitative fields or values that can be used to sort or filter a measure. You can have a Measure with all Total Purchase Values but you want to filter them by Customer ID, so each measure value corresponds to its customer.
Visual Insight BuilderA user-friendly UX builder tool that allows you to create insights with limited knowledge of SQL language.The Visual Insight Builder lets you create a complex insight (RFM model values per customer) without having to use SQL, writing queries, etc.
Aggregate FunctionAn Aggregate Function is a function in data analysis that makes a calculation on a set of values and returns a single value.A simple Aggregate Function is the AVERAGE. For example, calculate the Average value of all the orders of a given customer.

Data Cloud Calculated Insights

You can use the Calculated Insights feature of Data Cloud to define and calculate multidimensional metrics based on your data.

Calculated Insights use either (ANSI) SQL or a Visual Declarative Builder and they are specially useful for these two phases of the Data Cloud use flow:

  1. Segmentation
  2. Activation

Some characteristics of Calculated Insights:

  • They are multi-dimensional. You can leverage all the data stored in Data Cloud, cross check related objects, use any category, etc.
  • They can enhance segmentation. You should use Calculated Insights to derive insights from your customer data and improve your segmentations.
  • Available via API with external systems.
  • You can use Calculated Insights to personalize segment activations.
  • Calculated Insights are automatically refreshed so incremental data changes are added without manual action from the user.
  • Insights are turned into actions when they’re integrated in Segmentation and Activation phases of Data Cloud.

Create Calculated Insights using SQL

If you are familiar with SQL, you can write SQL Queries to create your Calculated Insights in Data Cloud.

Follow these steps to create them:

  1. Go to Calculated Insights tab
  2. Click New and select a Data Space
  3. Click on Create with SQL
  4. Enter Calculated Insight name (API is auto-filled)
  5. Write the SQL statement (character limit of 131,021)
  6. Use the Check Syntax button to validate your SQL (the tool won’t let you continue if there are errors)
  7. Click Activate
  8. Select a Schedule and click on Enable

ANSI SQL uses the following statement format to write SQL Queries in Calculated Insights:

SELECT Attributes, Aggregation(Measures)
FROM [DMO 1]
JOIN [DMO 2] [alias]
WHERE [conditions]
GROUP BY Columns(Dimensions)

Create Calculated Insights using the Visual Builder

If you don’t know SQL, you can still create Calculated Insights using a user-friendly builder available in the UI.

To create calculated insights with this declarative tool, follow these steps:

  1. Go to Calculated Insights tab
  2. Click New and select Data Space
  3. Select Create with Builder and click on Calculated Insight
  4. Use the visual interface drag and drop tools to define your calculated insight
  5. Save
  6. Enter the Calculated Insight name (API name is auto filled)
  7. Select a Schedule
  8. Click Enable

Let’s take a look at how the Visual Editor works separately, as it has a lot of features.

The basic order of operations goes like this:

1. Object –>
2. Measures and Dimensions (Aggregate node) –>
3. Save Aggregate node –>
4. Add data (join, case, filter, transform, arithmetic)

You first select an Object from which you’re calculating the insight. For example, Orders.

Then you need to add at least 1 Aggregate Node, using at least 1 Measure and 1 Dimension to group by. E.g: average (aggregate function) purchase value (measure) + grouped by Customer ID (dimension)

Orders: Average Order Value organised by Customer ID.

You can later add different data to the insight: perform a JOIN with another table, create a Case to account for multiple conditions, perform arithmetic expressions, filter by a specific set of conditions, etc.

Create Calculated Insights using a Package

You can create a Calculated Insight using a Salesforce Installed Package in Data Cloud.

Follow these steps:

  1. Go to Calculated Insights tab
  2. Click New and select a Data Space
  3. Select Create from a Package
  4. Choose the Installed Package (Calculated Insight name, API name and SQL Query fields are auto filled).
  5. Save

Calculated Insights Details

Once a Calculated Insight is created, its record page will be visible in the Calculated Insights tab of Data Cloud.

These are the settings you can use for a given Calculated Insight:

SettingDescription
Enable / DisableActivate or deactivate the processing of a calculated insight.
DeleteRemove a calculated insight and all its components.
CloneDuplicate the SQL function of the insight and edit it.
EditChange the name and field names of a calculated insight.
Publish NowRun the calculated insight on demand, instantly, instead of processing it on a schedule.
ScheduleEdit the processing schedule of a calculated insight.
Update StatsUpdate the status, last modified date, last run time, etc of an insight.
Show in BuilderOnly available if you created the Calculated Insight with the Visual Builder. Opens the declarative UX builder to view the insight.

Calculated Insights Processing Limits

You can modify the frequency, time and date of processing of a Calculated Insight in Data Cloud.

The timings will vary depending on the Data Model Objects you’re creating the insight on and the time dimensions you are calculating with.

These are the Calculated Insights Limits that exist in Data Cloud:

FunctionDescriptionLimit
Calculated Insight execution TimeoutThis is the maximum time that the Calculated Insight can be processing before it's terminated by the system.2 hours
Total number of Calculated Insights per tenant or OrgTotal number of Calculated Insights (in any status) that you can have in a single tenant.300
Maximum number of times that a Calculated Insight can be processed manually in 24 hoursCalculated Insights can be manually processed. This is the limit on how many times you can do so within a 24-hour period.3 per calculated insight
Maximum number of nested calculated insightsAn existing calculated insight can be saved and used while creating another insight, as a nested function.

This is the total maximum number of existing insights you can use when creating a new one.
3
Maximum number of dimensions per calculated insightDimension examples: customer ID, Email Address, Product SKU, etc.10
Maximum number of measures per calculated insightMeasure examples: average order value, total purchase value, etc.50
Total number of calculated insight batch runs per yearThis is the number of times a calculated insight has been refreshed in the last year.100,000

Below, a table with the different processing Frequency options for Calculated Insights:

Frequency OptionDescriptionLimit
System Managed: several times per dayCalculated Insights that are system managed are automatically turned into Scheduled Calculated Insights once they are edited.N/A
ScheduledThe user can define a Start date, time and then a preferred frequency for the processing of the Calculated Insight.

If there are no changes in the source data, mappings, etc, the Calculated Insight will not run.
Every 6, 12, or 24 hours
Using Data Cloud in a Developer OrgSchedules are ignored in a Data Cloud Developer org. The user must click on Publish Now to process the Calculated Insight manually.0 (only manual processing available)

Calculated Insights Retention Limits

The data retention in a Calculated Insight depends on the date and time dimensions that it leverages in the Group By statement.

Depending on the time window of those dimensions, the system will retain data for a shorter or longer period of time. Think of it as a lookback window limit.

Here’s a table to show the different limits:

Date/Time DimensionRetention LimitExample
Yearly20 yearsTotal yearly spend by customer.
Quarterly5 yearsQuarterly total revenue
Monthly5 yearsTotal monthly purchases by customer.
Daily365 daysDaily website visits
Hourly48 hoursHourly abandoned cart events per ecommerce session.

Calculated Insights Rollup Setting

As the time dimension becomes more granular (higher, that is, from years to months to days to hours), the data retention period is lower.

E.g:
Hourly data is only retained for 48 hours
Daily data is only retained for 365 days

That means that if you make a Calculated Insight that tracks hourly ecommerce transactions, after the 48 hours, the data is no longer retained.

What Salesforce Data Cloud does is to roll up this data into the next tier time dimension. Here’s a table:

Data TimeInterval for groupingGrouped into next time
Hourly24 hoursDaily
Daily30 daysMonthly
Monthly3 monthsQuarterly
Quarterly4 quartersYearly

So for example, after 24 hours, the hourly ecommerce transactions are grouped into daily (the next tier time dimension), so in your SQL Queries, etc, you should take this rollup into account.

The system will only roll up the data in Calculated Insights using the following aggregate functions:

  • Sum
  • Min
  • Max
  • Count
  • Avg
  • Mean

Editing Calculated Insights

You can edit Calculated Insights that are created using either SQL or the Visual Builder.

  1. Go to the Calculated Insights tab
  2. Edit
  3. Make changes and click Update
  4. Save

Some important considerations when editing Calculated Insights:

  • The API name, data type or rollup behaviour cannot be changed for Measures.
  • For Dimensions, you cannot change the name and data type either.
  • You can’t update dimensions that are non-transformed (that are taken directly from the DMO) either. If you update transformed dimensions, you can do so as long as you don’t change its name or data type.
  • You can’t add dimensions to an existing calculated insight unless they are a key qualifier dimension.
  • You can’t remove existing measures and dimensions.
  • You can edit a calculated insight that is used in Segmentation and Activation.
  • You can edit the filers or update the JOIN conditions.

Data Cloud Streaming Insights

You can use the Streaming Insights feature of Data Cloud to define and calculate metrics based on streaming data for real-time triggers.

These insights are created based on data that changes continually (streaming). For example, data coming from Mobile or Web SDKs.

Streaming Insights use either SQL or an Insights Builder, and they are specially useful for creating orchestrations or data actions.

They cannot be used in the Segmentation or Activation phases of Data Cloud.

Some characteristics of Calculated Insights:

  • They are used for streaming, continually changing data (e.g: patterns, nuanced changes, real-time triggers, alerts, etc).
  • They leverage Data Actions to orchestrate triggers and be shared with other apps.
  • They are available with the JDBC API.
  • The available sources are: Web SDK, Mobile SDK, MC Personalization
  • They can aggregate data every 5 minutes, up to 24 hours.

Data Actions and Streaming Insights

The biggest use of Streaming Insights is in their data flow with Salesforce Data Cloud Data Actions.

What are Data Actions?

Triggers that send data change events to the Salesforce Platform Event, Marketing Cloud Engagement or a Webhook.

Example of a Data Action Use Case:
If a customer enters a clothing store, real-time streaming geo-location data triggers sending an SMS to that customer with a special in-store discount.

Salesforce recommends the following process order to ensure that you can leverage Data Actions together with Streaming Insights:

  1. Data Sources
    Map Web and Mobile SDK source streams.
  2. Data mapping & Identity Resolution
    Create unified individuals after data harmonization.
  3. Insights
    Create your insights in Data Cloud.
  4. Enhancement
    Enrich your insights with unified individual data and profile-related attributes.
  5. Data Actions
    Configure your data action triggers on the streaming insights.

Data Actions and Streaming Insights Limits

Please find the below table for a detailed breakdown of current Data Action and Streaming Insights limits as per Salesforce Documentation:

FeatureData Cloud Limit
Total N of streaming insights20
Total N of dimensions10
Total N of measures5
Aggregations available (both SQL and Insights Builder)SUM
AVG
MIN
MAX
COUNT
MEAN
APPROX_COUNT_DISTINCT
LAST
FIRST
PERCENTILE
STDDEV
Aggregation time frame limitsMinimum: 1 min
Maximum: 24 hrs
JoinInner

- Engagement Join Individual
- Engagement Join Unified Individual Profile
Data Sources- Marketing Cloud Personalization
- Mobile SDK
- Web SDK
Total N of Data Actions definitions per tenant20
Total N of Data Actions executions generated per tenant, per minute100
Total N of Data Action Targets per tenant10
Total N of Data Action Rules per Data Action10
Maximum N of data attributes in each payload20
Latency for the first Data Action Target to receive the data trigger event15 min (only when first created and configured)
Type of Streaming Insights that can be configured with a Data ActionOnly SQL streaming insights

Creating a Streaming Insight with SQL

  1. Go to the Calculated Insights tab in Data Cloud
  2. Click New and Select a Data Space
  3. Create Streaming Insights > Next
  4. Enter the Name for the insight (API is auto populated)
  5. Write your SQL Query (max 131,021 characters)
  6. Use the Check Syntax to validate your SQL expression
  7. Save and Run

ANSI SQL uses the following statement format to write SQL Queries in Streaming Insights:

SELECT Attributes, Aggregation(Measures)
WINDOW.START as start__c
WINDOW.END as end__c
FROM [DMO 1]
JOIN [DMO 2] [alias]
WHERE [conditions]
GROUP BY Columns(Dimensions)
WINDOW(TimeAttribute, X MINUTE/HOUR)

Creating a Streaming Insight with the Insights Builder

  1. Go to the Calculated Insights tab in Data Cloud
  2. Click New and Select a Data Space
  3. Create With Builder > Next > Streaming Insight
  4. Select an Object created from streaming data sources
  5. Use the visual interface drag and drop tools to define your streaming insight
  6. Save and Run

Streaming vs Calculated Insights

Here is a table with the main differences between Calculated Insights and Streaming Insights.

As you can see, they serve different purposes and will greatly depend on what you want to do with your data, use cases and the type of data you want to analyse:

FeatureStreaming InsightsCalculated Insights
PurposeUsed for triggering real-time, time-sensitive or rule based actions.

E.g:
alerts, notifications based on data, Salesforce CRM orchestrations, etc
Refining your Segmentations and enriching personalization

E.g:
RFM models, ranked affinity scores, aggregations, etc
Creation MethodsSQL or BuilderSQL or Builder
Available in Data Cloud components- Data Action Targets
- JDBC Driver
- Segmentation
- Activation
- APIs
- Analytics
Data Sources it leveragesOnly Engagement data from real-time sources.

MC Personalization
Web SDK
Mobile SDK
Both Profile and Engagement data
Processing LatencyReal-timeBatch
Data VolumeMicro batches of few recordsLarge, historical data for complex calculations

Data Cloud Real-time Insights

You can also use the Real-Time Insights feature of Data Cloud to create highly complex personalization logic for your website.

For example, track user behaviour on your site to drive:
Personalized communications based on browsing behaviour
Send them a promotion based on total purchase value
Build ranked affinity scores based on cart add events, PDP clicks, etc

Some characteristics of Real-Time Insights:

  • They must be created based on a real-time data graph.
  • They have to be created using the Visual Builder.
  • They can be used in both Segmentation and Activation.

Creating a Real-time Insight with the Insights Builder

  1. Go to the Calculated Insights tab in Data Cloud
  2. Click New and Select a Data Space
  3. Create Real-Time Insights > Next
  4. Select an Object
  5. Select a real-time data graph as input
  6. Use the visual interface drag and drop tools to define your real-time insight
  7. Name your Real-Time Insight (API is auto populated)
  8. Save and Enable

Validating Insights

Finally, after you’ve created your Insights (whether it’s a Calculated or Streaming one), you can use Data Explorer to validate that the data shows and returns values as expected.

Follow these steps to validate your insights in Data Explorer:

  1. Go to the Data Explorer tab
  2. Select a Data Space
  3. From the dropdown menu, choose Calculated Insights
  4. Select your Calculated Insight or Streaming Insight
  5. Review the data output

You may also personalize the columns shown and/or use filters to sort your data, order it by ascending or descending order and so on.