The Do’s and Don’ts of Duplicate Management in Salesforce Part I

All organizations can benefit from a good Salesforce Duplicate Management Strategy, as my colleague Pete wrote in a previous post. Today we are going to take a deeper dive into some of the best practices and common mistakes to avoid when setting up and maintaining duplicate management in Salesforce.

DO: Create and Implement a Duplicate Management Plan

Define What a Duplicate Record Actually Is

The specific criteria for what constitutes a duplicate record in your system will vary depending on what type of data your organization collects, as well as your specific needs and processes. Ask yourself — what fields do you always check to determine if a record is a duplicate? Keep in mind it may be different for each type of record. For example, two organization accounts with the same name may be a duplicate but two contacts with the same name may not be (and vice versa). 

Determine the Sources of Duplicate Records

Most likely duplicates are entering your system from both internal and external sources. External sources may include online forms such as subscribing to a newsletter or volunteer sign up form, or an externally integrated system for donations, events, marketing etc. Internal sources would primarily consist of manual data entry and data importing.

Set Realistic Goals for What Success Will Look Like

It’s important to remember that as much as we’d all love a duplicate-free world, that simply does not exist. Determine an acceptable number of duplicates that can be reviewed and merged by a staff member on a regular basis. Create a plan to reduce the number of duplicates that will need to be managed by focusing on solutions for your most common sources of duplicate generation. 

DON’T: Neglect User Experience or Training 

Don’t overlook the effectiveness of user experience and training as your first line of defense to prevent duplicates from coming into the system in the first place.

Don't Overlook Salesforce Search

Get familiar with Salesforce search and use it often. Simply training users to always search before creating a new record (and reminding them regularly so they don’t forget!) can go a long way. The Salesforce Global Search feature has Salesforce’s AI-powered product Einstein built in which will return records with similar names and even nicknames, such as Bill for William. In addition to searching for contacts by name, you can search by company, mailing address, and phone number, just to name a few. Take your search queries to the next level by combining search terms (e.g. Carol Smith NY) or using wildcards and operators (e.g. C* Smith).

Search for names similar to Lisa or Bob with last names beginning with the letter B in New York

Don’t Skip Data Cleansing Before Importing 

If you have several users regularly importing data in the system, consider centralizing data loading as much as possible and ensure users responsible for the imports have received adequate training. Clean data imports are crucial to keeping data integrity intact so you’ll want to spend some time to ensure your import file is as clean as possible.

You can use Excel to find and remove duplicate data from spreadsheets before importing. Moreover, by standardizing the formatting of addresses and phone numbers, you’ll not only make your data look prettier but also catch duplicates that may be missed otherwise.

Last but not least, it is an important best practice to always perform a system backup before import. Remember, while Salesforce has the ability to delete records - there is no undo option!

Don't Neglect or Forget to Regularly Review and Update Page Layouts

Streamlining page layouts is another way to see immediate benefits by making data entry easier and providing an enhanced user experience. Page layouts and Lightning page layouts should have a consistent structure across objects and record types with variations only where relevant and helpful. Record pages that are difficult to navigate lead to poor data entry practices which can quickly result in inconsistent data that ends up being duplicated elsewhere in the system.

DO: Take Advantage of Native Duplicate Management Tools

Add the Potential Duplicates Component to Page Layouts

This helpful tool can be added to the lightning page layout for Account, Contact and Lead records. You also have the ability to display a temporary toast message. 

Contact record with potential duplicates component and toast message

Customize and Activate Matching Rules and Duplicate Rules

Used properly, these tools will be your best friends for managing duplicate records on an ongoing basis and they already exist in your Salesforce org right now! Let’s look at the difference between the two and how they work together to find and prevent duplicate records.

• Matching Rules define the criteria for what is considered a duplicate record. This is where you identify the individual fields that you want to check against to find a match. You have the ability to choose an exact or fuzzy matching method which will dictate how strictly the matching criteria will be used when comparing fields, as well as the option to match on blank fields.
• Duplicate Rules define the action that occurs when a user attempts to create a duplicate record. Each duplicate rule requires at least one matching rule to identify which existing records are possible duplicates. This is where you can choose to allow or block duplicate records from being created, alert the user when they are potentially creating a duplicate, and create a record of that duplicate in a duplicate record set.

You need both Matching Rules and Duplicate Rules in order to utilize Salesforce Duplicate Management. Once you have your rules set up, don’t stop with just Accounts and Contacts, you create custom Matching Rules and Duplicate Rules for almost any object including custom objects! You can also use cross-object matching such as checking Lead records with Contacts.

Important Note to Nonprofits: The Standard Contact and Account Matching Rules that come with Salesforce are not recommended to use with NPSP so you’ll want to use Custom Matching Rules instead. Check to see if your org already has the “NPSP Contact Personal Email Match” Matching Rule otherwise create one by matching on First Name, Last Name, and Personal Email. You can create additional rules around Work Email, Home Phone, Mailing Address, etc. 

For Accounts, it’s safe to use the Standard Account Matching for Organizations but you’ll want to create a Custom Matching Rule for Households and add a condition to each Duplicate Rule to specify which Account Record Type each rule should be associated with.

Custom Household Account Matching Rule with matching criteria: Formal Greeting AND Billing Street AND (City OR ZIP OR Household Phone)

Custom Household Account Duplicate Rule with condition that Account Record Type Equals Household Account

Track Down Potential Duplicates with Duplicate Jobs (If You Have It)

Matching and Duplicate Rules are great for finding and preventing future duplicates entering the system, which you can monitor with duplicate record reports, but they won’t help you find any existing duplicates that were created before you activated the rules. This is where Duplicate Jobs come in! Released in Spring 18, Duplicate Jobs can scan your entire org to find potential duplicates in mass based on a Standard or Custom Matching Rule.

Unfortunately, Duplicate Jobs are only available in Performance and Unlimited Editions, or by purchasing Insights Platform Data Integrity, an add-on product specifically for nonprofits. If you believe, like we do, that more orgs should have access to this powerful duplicate management tool, join us in voting for Lightning Duplicate Jobs for Enterprise Edition on the Salesforce IdeaExchange. This functionality is too good not to share with everyone, especially nonprofits!

DON’T: Set Up Rules that Will Result in Data Loss 

When configuring Salesforce Duplicate Management it’s important to understand how it will impact data coming into your system from all sources, especially 3rd party applications. This is particularly relevant when it comes to the Block and Alert settings on Duplicate Rules as both of these options could completely block records with match criteria attempting to enter through the API such as web-to-lead submissions, data imports, and integrations. Even worse - you could potentially lose that data completely if it is not being captured in another system!

Luckily, there are a few solutions you could use to mitigate this issue. The simplest solution is to set all your Duplicate Rules to Allow and Report only. This will allow data to enter the system without restrictions and still keep a record of all the duplicates that were identified for you to review and merge after the fact. Alternatively, if you would like to continue to use the Block or Alert settings for internal users performing manual data entry, you can set up two of each Duplicate Rule and set the conditions for each based on the User Profile. This way you’ll still be able to report on all potential duplicates coming into the system and avoid potential data loss. Duplicate Rule with condition to exclude Users with System Administrator and Integration User Profiles

It's important to note that some apps, like Classy, have recommended settings for Matching and Duplicate Rules (e.g. alert is required for enhanced matching). Other apps, like Veevart, have requirements for things you should NOT set in Matching Rules (e.g. Matching Rules cannot have alerts turned on). To sum up, it's critical that you take your 3rd party applications into consideration as you create rules for your particular use cases.

Conclusion

While it’s impossible to eliminate duplicates completely, following these tips will help set you up for success and save a headache down the road. Stay tuned for our next article The Do’s and Don’ts of Duplicate Management in Salesforce Part II where we’ll discuss important considerations for managing and merging existing duplicates in your system.

What are some of your top do’s and don’ts to follow when managing duplicate records in Salesforce? Want to share your experiences or opinions? Tell me about them on the Salesforce Trailblazer Community, or tweet directly at me on Twitter @SalesforceAF.