Flows | Remove Duplicates from Collection Variables in a Flow

In this article, I am going to show how can we remove duplicate records from collection variables or de-duplicate collection variables in Salesforce Flows.This is a pure declarative approach as now we have some Apex Actions also available.

Main takeaways from this article:

1. How to de-duplicate records in collection variables of a Flow in Salesforce?

2. How to use "Remove All" and "Remove Uncommon" operators in Assignment Element of a Salesforce Flow.

De-Duplicate Records in a Flow Collection Variable

Problem: 

In flows, we sometimes use Multiple Get Record Elements to fetch Records from an object based on different conditions. It's a way to do OR query in Flows as that is not possible in a single GET Element.

Now, if we need to perform any DML on these two collection variables after combining their data, like update or delete, Flow throws an error message like " Duplicate Record Ids Found". This is because there are duplicate records within the collection variable which we are trying to update/delete.

Solution:

This problem can be solved by using operators within the Assignment Element in Flow called "Remove Uncommon" and "Remove All". We will use assignment Element to de-duplicate the collection variable so that it can be used to Update/Delete Records.

Important Consideration:
The strategy or way that I am going to showcase in this article below will only work if the Collections or Collection Variables that we are going to compare or de-duplicate are identical in nature. It means that they should have same properties and have same schema. For Example, If I am using one Get Records Element for Cases and fetching all the fields and I am using another Get Element for Cases but fetching only certain fields, this below strategy won't work if we try to de-duplicate these two Collections from Get Record Elements. All/both Collections that are compared should have same object and field properties (in short same structure and type). This is an important consideration. 

Sample Use Case:

Consider we have two Get Elements in the Flow. Both are querying Contact Records but based on different criteria.

Now we need to combine these two collection variables of two Get Elements in order to perform some delete/update operation.So we need to something like:

Final Result Needed : Get Element 1 + Get Element 2

We will normally use an Assignment Element to do that. 

We can simply use the "Add" operator in Assign element to add these two. Seems easy right? 

But there's one catch, if both the Get Elements returned some same/duplicate Contacts (when a single contact record met the criteria of both Get Elements ),then we will face the issue as I described in starting about duplicate records. 

Actual Result Needed: Get Element 1 + Get Element 2 minus the Duplicates in both.

Solution:

We will use the Assign Element not only to combine/join both the collection variables(variable 1 and 2) but also to Remove Duplicates from them.This problem can be solved by using a property within the Assignment Element called "Remove Uncommon" and "Remove All". 

Let's Dive into action to see how to use this.

Please Note: In order to try and showcase a Specific Use Case, I may have overlooked some of the best practises related to Flow Development in this article. Please make sure to follow these in real world scenarios and projects. Check some really important ones below.

Overall Flow for our Sample Use Case:

1. Get Elements:

As I mentioned above, we are just using two Get Elements to get records from Contacts for two different criteria. Consider this as an OR Statement for fetching data for two criteria.   

Get Element 1({!Contacts_Based_on_Name}) is querying Contacts based on Name. Let's call it Variable 1. 

Get Element 2 ({!Get_Contacts_Based_On_Email}) is querying the Contacts based on Email. Let's call it Variable 2.

Assume Variable 1 returns/has 3 records:  abc, def, ghi (Consider these as dummy contacts)

Assume Variable 2 returns/has 3 records:  abc, jkl, nmo (Consider these as dummy contacts)

So here, record abc is the common one in two variables. When we are done de-duplicating, final update list/variable should have only one abc and it should look like abc,def,ghi,jkl,nmo

2. Assignment Element: 

This is where the Magic will happen. We will join/combine these two variables and also to de-duplicate and remove the common records from these two variables.

Here is how the final Assignment Element will look like:

1. Create a new Variable of type Record for Contact Object(Multiple Records/Collection Variable).

I have named this variable as Merge Contact Records as shown Below. This will be used in our Assignment Element.Check steps below.

Step wise details for Assignment Element:

1. Assign Value to the new Variable:
We will assign this new variable (MergeContactRecords) equal to Get Element 1.

So after this step, MergeContactRecords =  abc, def, ghi (equal to Get Element 1 values).

2. Remove Unique or uncommon values from the two variables

Now, we are using an operator called "Remove Uncommon". 
This will basically remove all the Uncommon or Unique records from the two variables and only keep the duplicate values in both the variables. In our case, we are comparing MergeContactRecords and Get Element 2 variables.

MergeContactRecords is equal to abc,def,ghi (after step 1) and Variable 2 is equal to abc, jkl, nmo. So, this operator will remove all unique values and only return "abc" which is common in both the variables. This variable will be used in later steps again.

After this step, MergeContactRecords = abc (Removed all unique values from it)

3. Joining Two Get Elements.

Now, we are just combining the two Get Element variables 1 and 2 and assigning their values into Get Element 1. Get Element 1= Get Element 1 + Get Element 2

After this step, Get Element 1 = abc,def,ghi,abc,jkl,nmo 

Remember, this still has the duplicate value "abc" which we will need to remove.

4. Remove all records which are duplicates from the joined variable:

In last step, we made Get Element 1/ Variable 1 = abc,def,ghi,abc,jkl,nmo.

Important: Now we will use "Remove All" operator to remove all the values which are duplicates from the Variable 1. 

Basically, "Remove All" operator is used to delete/remove all values(which we provide on right side of the operator) from a collection variable.
Here we are removing the values in variable  "MergeContactRecords" (abc) from Variable 1.
Remember, MergeContactRecords is equal to "abc" so it will remove all "abc" values from Variable 1.

After this step, Variable 1 or Collection variable of Get Element 1 = def,ghi,jkl,nmo (Removed two abc instances from this)

Please note previously this variable(Var 1) had two same records "abc" which are now both removed.

5. Adding the Common Record back to Variable :

In the last step, we removed both the duplicate values("abc") from the final list or collection variable 1. But, we need to add one of them back to the list. 

If you see above in step 2, MergeContactRecords is holding that unique/common value ("abc"). So now we will just append this value in our list from pervious step using "Add Operator". 

Since it will be added only once, we no longer will have duplicates.

After this step, Variable 1 / Get Element 1 = def,ghi,jkl,nmo,abc

Variable 1 or Get Element 1 Collection Variable is our de-duplicated list/collection with data from both the Get Elements and it can be used further to perform update/delete without any issues.

And we are Done!

There is an alternative way to De-Duplicate collections: Check that out here:https://accidentalcoder.blogspot.com/2020/07/remove-duplicates-from-flow-collection-variables-using-loops.html

Checkout all my Flow Posts here: Interesting Flow Scenarios and Use Cases

If you like this please do Share! #Learn and Share!

Enter your email address:

Delivered by FeedBurner