You’ve configured Data Cloud, ingested and modelled data. You’ve also performed identity resolution to unify customer profiles.
Now, before you start activating this data for AI, marketing segments, etc… it’s time for leveraging Salesforce Data Cloud Insights.
That is: analysing, calculating and defining insights from all that data, visualizing it using the business intelligence tools available in Data Cloud.
Salesforce Data Cloud Insights Overview
Data Cloud comes with a lot of Insights features which allow you to define and calculate multidimensional metrics.
These intelligent metrics can be, for example:
- Customer Lifetime Value (LTV)
- Most Purchased Products
- Customer Satisfaction Scores (CSAT)
- RFM Model insights.
- Real-time metrics
These metrics are multidimensional.
In other words, they cross check and relate data from multiple objects and fields. That is why you need Insights to work with them. A simple Formula Field in a field will not allow you to perform the powerful calculations you can do with Data Cloud Insights.
What’s more. These insights can be leveraged in different applications of the tool, use real-time data actions and much more.
Let’s look at all the types of Insights you can calculate and define, one by one.
Salesforce Data Cloud Insights Terms
Here is a table with the most common terminology that you will use when performing Insights in Data Cloud:
Term | Description | Example |
---|---|---|
Primary Key | The unique identifier of a record in an object. | A product SKU in a product catalogue, or an Email Address in a subscriber profile. |
Foreign Key | A field in a relational database table that serves as a link shared between that table and another one. | A subscriber ID can be present in multiple tables, being the link to relate the info in them. |
Attribute | A field or information in a data stream in Data Cloud. | Attributes can be, for example, a customer Email Address or Last Name. |
Measures | Numerical, quantitative fields that store aggregated number data points. | Total Purchase Value, Average Order Value. |
Dimensions | Qualitative fields or values that can be used to sort or filter a measure. | You can have a Measure with all Total Purchase Values but you want to filter them by Customer ID, so each measure value corresponds to its customer. |
Visual Insight Builder | A user-friendly UX builder tool that allows you to create insights with limited knowledge of SQL language. | The Visual Insight Builder lets you create a complex insight (RFM model values per customer) without having to use SQL, writing queries, etc. |
Aggregate Function | An Aggregate Function is a function in data analysis that makes a calculation on a set of values and returns a single value. | A simple Aggregate Function is the AVERAGE. For example, calculate the Average value of all the orders of a given customer. |
Please check the following link to see the official Salesforce documentation on Data Cloud Insights:
https://help.salesforce.com/s/articleView?id=sf.c360_a_insights.htm&type=5
Data Cloud Calculated Insights
You can use the Calculated Insights feature of Data Cloud to define and calculate multidimensional metrics based on your data.
Calculated Insights use either (ANSI) SQL or a Visual Declarative Builder and they are specially useful for these two phases of the Data Cloud use flow:
- Segmentation
- Activation
Some characteristics of Calculated Insights:
- They are multi-dimensional. You can leverage all the data stored in Data Cloud, cross check related objects, use any category, etc.
- They can enhance segmentation. You should use Calculated Insights to derive insights from your customer data and improve your segmentations.
- Available via API with external systems.
- You can use Calculated Insights to personalize segment activations.
- Calculated Insights are automatically refreshed so incremental data changes are added without manual action from the user.
- Insights are turned into actions when they’re integrated in Segmentation and Activation phases of Data Cloud.
Please check the following link to see the official Salesforce documentation on Data Cloud Calculated Insights:
https://help.salesforce.com/s/articleView?id=sf.c360_a_calculated_insights.htm&type=5
Create Calculated Insights using SQL
If you are familiar with SQL, you can write SQL Queries to create your Calculated Insights in Data Cloud.
Follow these steps to create them:
- Go to Calculated Insights tab
- Click New and select a Data Space
- Click on Create with SQL
- Enter Calculated Insight name (API is auto-filled)
- Write the SQL statement (character limit of 131,021)
- Use the Check Syntax button to validate your SQL (the tool won’t let you continue if there are errors)
- Click Activate
- Select a Schedule and click on Enable
ANSI SQL uses the following statement format to write SQL Queries in Calculated Insights:
SELECT Attributes, Aggregation(Measures) FROM [DMO 1] JOIN [DMO 2] [alias] WHERE [conditions] GROUP BY Columns(Dimensions)
For a full list of the ANSI SQL Function Library available in Data Cloud, check the following link:
https://help.salesforce.com/s/articleView?id=sf.c360_a_create_a_calculated_insights_sql_function.htm&type=5
Create Calculated Insights using the Visual Builder
If you don’t know SQL, you can still create Calculated Insights using a user-friendly builder available in the UI.
To create calculated insights with this declarative tool, follow these steps:
- Go to Calculated Insights tab
- Click New and select Data Space
- Select Create with Builder and click on Calculated Insight
- Use the visual interface drag and drop tools to define your calculated insight
- Save
- Enter the Calculated Insight name (API name is auto filled)
- Select a Schedule
- Click Enable
Let’s take a look at how the Visual Editor works separately, as it has a lot of features.
The basic order of operations goes like this:
1. Object –>
2. Measures and Dimensions (Aggregate node) –>
3. Save Aggregate node –>
4. Add data (join, case, filter, transform, arithmetic)
You first select an Object from which you’re calculating the insight. For example, Orders.
Then you need to add at least 1 Aggregate Node, using at least 1 Measure and 1 Dimension to group by. E.g: average (aggregate function) purchase value (measure) + grouped by Customer ID (dimension)
Orders: Average Order Value organised by Customer ID.
You can later add different data to the insight: perform a JOIN with another table, create a Case to account for multiple conditions, perform arithmetic expressions, filter by a specific set of conditions, etc.
Check this link for the documentation on using the Visual Builder to create Calculated Insights:
https://help.salesforce.com/s/articleView?id=sf.c360_a_insight_builder.htm&type=5
Visual Builder Supported Functions link:
https://help.salesforce.com/s/articleView?id=sf.c360_a_insight_builder_supported_functions_limitations.htm&type=5
Create Calculated Insights using a Package
You can create a Calculated Insight using a Salesforce Installed Package in Data Cloud.
Follow these steps:
- Go to Calculated Insights tab
- Click New and select a Data Space
- Select Create from a Package
- Choose the Installed Package (Calculated Insight name, API name and SQL Query fields are auto filled).
- Save
Check this link for the documentation on using Installed Packages to create Calculated Insights:
https://help.salesforce.com/s/articleView?id=sf.c360_a_create_calculated_insight_from_package.htm&type=5
Calculated Insights Details
Once a Calculated Insight is created, its record page will be visible in the Calculated Insights tab of Data Cloud.
These are the settings you can use for a given Calculated Insight:
Setting | Description |
---|---|
Enable / Disable | Activate or deactivate the processing of a calculated insight. |
Delete | Remove a calculated insight and all its components. |
Clone | Duplicate the SQL function of the insight and edit it. |
Edit | Change the name and field names of a calculated insight. |
Publish Now | Run the calculated insight on demand, instantly, instead of processing it on a schedule. |
Schedule | Edit the processing schedule of a calculated insight. |
Update Stats | Update the status, last modified date, last run time, etc of an insight. |
Show in Builder | Only available if you created the Calculated Insight with the Visual Builder. Opens the declarative UX builder to view the insight. |
Calculated Insights Processing Limits
You can modify the frequency, time and date of processing of a Calculated Insight in Data Cloud.
The timings will vary depending on the Data Model Objects you’re creating the insight on and the time dimensions you are calculating with.
These are the Calculated Insights Limits that exist in Data Cloud:
Function | Description | Limit |
---|---|---|
Calculated Insight execution Timeout | This is the maximum time that the Calculated Insight can be processing before it's terminated by the system. | 2 hours |
Total number of Calculated Insights per tenant or Org | Total number of Calculated Insights (in any status) that you can have in a single tenant. | 300 |
Maximum number of times that a Calculated Insight can be processed manually in 24 hours | Calculated Insights can be manually processed. This is the limit on how many times you can do so within a 24-hour period. | 3 per calculated insight |
Maximum number of nested calculated insights | An existing calculated insight can be saved and used while creating another insight, as a nested function. This is the total maximum number of existing insights you can use when creating a new one. | 3 |
Maximum number of dimensions per calculated insight | Dimension examples: customer ID, Email Address, Product SKU, etc. | 10 |
Maximum number of measures per calculated insight | Measure examples: average order value, total purchase value, etc. | 50 |
Total number of calculated insight batch runs per year | This is the number of times a calculated insight has been refreshed in the last year. | 100,000 |
Below, a table with the different processing Frequency options for Calculated Insights:
Frequency Option | Description | Limit |
---|---|---|
System Managed: several times per day | Calculated Insights that are system managed are automatically turned into Scheduled Calculated Insights once they are edited. | N/A |
Scheduled | The user can define a Start date, time and then a preferred frequency for the processing of the Calculated Insight. If there are no changes in the source data, mappings, etc, the Calculated Insight will not run. | Every 6, 12, or 24 hours |
Using Data Cloud in a Developer Org | Schedules are ignored in a Data Cloud Developer org. The user must click on Publish Now to process the Calculated Insight manually. | 0 (only manual processing available) |
Calculated Insights Retention Limits
The data retention in a Calculated Insight depends on the date and time dimensions that it leverages in the Group By statement.
Depending on the time window of those dimensions, the system will retain data for a shorter or longer period of time. Think of it as a lookback window limit.
Here’s a table to show the different limits:
Date/Time Dimension | Retention Limit | Example |
---|---|---|
Yearly | 20 years | Total yearly spend by customer. |
Quarterly | 5 years | Quarterly total revenue |
Monthly | 5 years | Total monthly purchases by customer. |
Daily | 365 days | Daily website visits |
Hourly | 48 hours | Hourly abandoned cart events per ecommerce session. |
Calculated Insights Rollup Setting
As the time dimension becomes more granular (higher, that is, from years to months to days to hours), the data retention period is lower.
E.g:
Hourly data is only retained for 48 hours
Daily data is only retained for 365 days
That means that if you make a Calculated Insight that tracks hourly ecommerce transactions, after the 48 hours, the data is no longer retained.
What Salesforce Data Cloud does is to roll up this data into the next tier time dimension. Here’s a table:
Data Time | Interval for grouping | Grouped into next time |
---|---|---|
Hourly | 24 hours | Daily |
Daily | 30 days | Monthly |
Monthly | 3 months | Quarterly |
Quarterly | 4 quarters | Yearly |
So for example, after 24 hours, the hourly ecommerce transactions are grouped into daily (the next tier time dimension), so in your SQL Queries, etc, you should take this rollup into account.
The system will only roll up the data in Calculated Insights using the following aggregate functions:
- Sum
- Min
- Max
- Count
- Avg
- Mean
Editing Calculated Insights
You can edit Calculated Insights that are created using either SQL or the Visual Builder.
- Go to the Calculated Insights tab
- Edit
- Make changes and click Update
- Save
Some important considerations when editing Calculated Insights:
- The API name, data type or rollup behaviour cannot be changed for Measures.
- For Dimensions, you cannot change the name and data type either.
- You can’t update dimensions that are non-transformed (that are taken directly from the DMO) either. If you update transformed dimensions, you can do so as long as you don’t change its name or data type.
- You can’t add dimensions to an existing calculated insight unless they are a key qualifier dimension.
- You can’t remove existing measures and dimensions.
- You can edit a calculated insight that is used in Segmentation and Activation.
- You can edit the filers or update the JOIN conditions.
Data Cloud Streaming Insights
You can use the Streaming Insights feature of Data Cloud to define and calculate metrics based on streaming data for real-time triggers.
These insights are created based on data that changes continually (streaming). For example, data coming from Mobile or Web SDKs.
Streaming Insights use either SQL or an Insights Builder, and they are specially useful for creating orchestrations or data actions.
They cannot be used in the Segmentation or Activation phases of Data Cloud.
Some characteristics of Calculated Insights:
- They are used for streaming, continually changing data (e.g: patterns, nuanced changes, real-time triggers, alerts, etc).
- They leverage Data Actions to orchestrate triggers and be shared with other apps.
- They are available with the JDBC API.
- The available sources are: Web SDK, Mobile SDK, MC Personalization
- They can aggregate data every 5 minutes, up to 24 hours.
Please check the following link to see the official Salesforce documentation on Data Cloud Streaming Insights:
https://help.salesforce.com/s/articleView?id=sf.c360_a_streaming_insights.htm&type=5
Data Actions and Streaming Insights
The biggest use of Streaming Insights is in their data flow with Salesforce Data Cloud Data Actions.
What are Data Actions?
Triggers that send data change events to the Salesforce Platform Event, Marketing Cloud Engagement or a Webhook.
Example of a Data Action Use Case:
If a customer enters a clothing store, real-time streaming geo-location data triggers sending an SMS to that customer with a special in-store discount.
Salesforce recommends the following process order to ensure that you can leverage Data Actions together with Streaming Insights:
- Data Sources
Map Web and Mobile SDK source streams. - Data mapping & Identity Resolution
Create unified individuals after data harmonization. - Insights
Create your insights in Data Cloud. - Enhancement
Enrich your insights with unified individual data and profile-related attributes. - Data Actions
Configure your data action triggers on the streaming insights.
Data Actions and Streaming Insights Limits
Please find the below table for a detailed breakdown of current Data Action and Streaming Insights limits as per Salesforce Documentation:
Feature | Data Cloud Limit |
---|---|
Total N of streaming insights | 20 |
Total N of dimensions | 10 |
Total N of measures | 5 |
Aggregations available (both SQL and Insights Builder) | SUM AVG MIN MAX COUNT MEAN APPROX_COUNT_DISTINCT LAST FIRST PERCENTILE STDDEV |
Aggregation time frame limits | Minimum: 1 min Maximum: 24 hrs |
Join | Inner - Engagement Join Individual - Engagement Join Unified Individual Profile |
Data Sources | - Marketing Cloud Personalization - Mobile SDK - Web SDK |
Total N of Data Actions definitions per tenant | 20 |
Total N of Data Actions executions generated per tenant, per minute | 100 |
Total N of Data Action Targets per tenant | 10 |
Total N of Data Action Rules per Data Action | 10 |
Maximum N of data attributes in each payload | 20 |
Latency for the first Data Action Target to receive the data trigger event | 15 min (only when first created and configured) |
Type of Streaming Insights that can be configured with a Data Action | Only SQL streaming insights |
Creating a Streaming Insight with SQL
- Go to the Calculated Insights tab in Data Cloud
- Click New and Select a Data Space
- Create Streaming Insights > Next
- Enter the Name for the insight (API is auto populated)
- Write your SQL Query (max 131,021 characters)
- Use the Check Syntax to validate your SQL expression
- Save and Run
ANSI SQL uses the following statement format to write SQL Queries in Streaming Insights:
SELECT Attributes, Aggregation(Measures) WINDOW.START as start__c WINDOW.END as end__c FROM [DMO 1] JOIN [DMO 2] [alias] WHERE [conditions] GROUP BY Columns(Dimensions) WINDOW(TimeAttribute, X MINUTE/HOUR)
Please take into account that the SQL Expression for a Streaming Insight requires the time window definition. The last line, WINDOW, is where we define the window (from 5 minutes to 24 hours).
Creating a Streaming Insight with the Insights Builder
- Go to the Calculated Insights tab in Data Cloud
- Click New and Select a Data Space
- Create With Builder > Next > Streaming Insight
- Select an Object created from streaming data sources
- Use the visual interface drag and drop tools to define your streaming insight
- Save and Run
Streaming vs Calculated Insights
Here is a table with the main differences between Calculated Insights and Streaming Insights.
As you can see, they serve different purposes and will greatly depend on what you want to do with your data, use cases and the type of data you want to analyse:
Feature | Streaming Insights | Calculated Insights |
---|---|---|
Purpose | Used for triggering real-time, time-sensitive or rule based actions. E.g: alerts, notifications based on data, Salesforce CRM orchestrations, etc | Refining your Segmentations and enriching personalization E.g: RFM models, ranked affinity scores, aggregations, etc |
Creation Methods | SQL or Builder | SQL or Builder |
Available in Data Cloud components | - Data Action Targets - JDBC Driver | - Segmentation - Activation - APIs - Analytics |
Data Sources it leverages | Only Engagement data from real-time sources. MC Personalization Web SDK Mobile SDK | Both Profile and Engagement data |
Processing Latency | Real-time | Batch |
Data Volume | Micro batches of few records | Large, historical data for complex calculations |
Data Cloud Real-time Insights
You can also use the Real-Time Insights feature of Data Cloud to create highly complex personalization logic for your website.
For example, track user behaviour on your site to drive:
Personalized communications based on browsing behaviour
Send them a promotion based on total purchase value
Build ranked affinity scores based on cart add events, PDP clicks, etc
Some characteristics of Real-Time Insights:
- They must be created based on a real-time data graph.
- They have to be created using the Visual Builder.
- They can be used in both Segmentation and Activation.
Please check the following link to see the official Salesforce documentation on Data Cloud Real-Time Insights:
https://help.salesforce.com/s/articleView?id=sf.c360_a_real_time_insights.htm&type=5
Creating a Real-time Insight with the Insights Builder
- Go to the Calculated Insights tab in Data Cloud
- Click New and Select a Data Space
- Create Real-Time Insights > Next
- Select an Object
- Select a real-time data graph as input
- Use the visual interface drag and drop tools to define your real-time insight
- Name your Real-Time Insight (API is auto populated)
- Save and Enable
Validating Insights
Finally, after you’ve created your Insights (whether it’s a Calculated or Streaming one), you can use Data Explorer to validate that the data shows and returns values as expected.
Follow these steps to validate your insights in Data Explorer:
- Go to the Data Explorer tab
- Select a Data Space
- From the dropdown menu, choose Calculated Insights
- Select your Calculated Insight or Streaming Insight
- Review the data output
You may also personalize the columns shown and/or use filters to sort your data, order it by ascending or descending order and so on.