Creating & Editing Filters
  • 09 Nov 2022
  • 19 Minutes to read
  • PDF

Creating & Editing Filters

  • PDF

Cloudingo walks you through four steps to create or edit filters. Before you begin, be sure to keep in mind that the fields you match on will determine the level of confidence you have in automating the merges later. 

From the dashboard, start by clicking either Create New Filter on the Actions drop down menu or Edit from one of your filter menus.



Tab 1: The Basics


A filter is defined based on the object(s) that will be scanned for duplicates. Single-table filters compare like object to like object, for example, existing Leads to existing Leads, or existing Contacts to existing Contacts. Multi-table (or cross object) filters compare records across two different objects, for example, existing Leads to either existing Accounts or existing Contacts.

Filter Operation

Choose the filter type you are creating.

  • Merge: Create a filter across the same object (ex: find Leads with a matching Lead record, Contacts with a matching Contact, or Accounts with a matching Account).
  • Convert: Create a filter across multiple objects (ex: Find Leads with a matching Contact record or Leads with a matching Account.
  • Cross-Matching Merge: Create a merge filter that checks two different fields against each other to find duplicate matches (Ex: if you have two email fields in Salesforce such as Email and Alternate Email, you can compare them against each other using this filter operation).

    When building a cross-match field filter, specify the fields you want to cross examine in Step 1 of the filter configuration.
Cross-field matching
For cross-matching field filters, you must add at least one additional matching field in Tab 2. The matching field on Tab 2 can be any other field match you wish. You will not need to configure either of your cross-matched fields since you already specified them in Step 1.

Video: Cross-field Matching

Object Type

On single object merge filters, select the object (lead, contact, account, person accounts) you want to examine for duplicates.

  • Source Object Type: For cross-object filters, select the primary object you want to scan against to detect duplicates (ex: if you are scanning leads against contacts, you would choose ‘Lead’ as your source object).
  • Target Object Type: For cross-object filters, select the secondary object that you will scan your source object against to detect duplicates (ex: if you are scanning leads against contacts, choose ‘Contact’ as your target object).

Filter Name

Enter a name for the filter (displayed on the dashboard). The name is the easiest way to identify what duplicates are being found within that filter. It should reference the object and an indication of the filter criteria. (Example: "Leads (email+ last name)".)

Description Name

Description name is optional, but can be used to help describe how your filter is configured (similar to the Filter Name).

Enable

The Enabled checkbox is selected by default. This checkbox must be selected in order for the filter to scan Salesforce data for duplicates.

Exclude Import Records 

By default, records from import files that remain in the Cloudingo import wizard will be included in scans performed during all future import uploads as well as your dedupe dashboard filter results. We highly recommend completing your import processes within Cloudingo entirely or deleting the incomplete imports out of the wizard.

If you have files uploaded that you are still needing to manage, but want your dedupe dashboard filters to exclude those records from pulling into the results, you can edit your current filter(s) and select the ‘Exclude Import Records’ option in Tab 1 of the filter configuration.

Note

Choosing this option will also disabling the filter from appearing as a filter option to scan against in Step 5 of the import process. If you regularly upload files and are working on dashboard filters simultaneously, consider creating separate filters used specifically for the import process. Keep dedupe dashboard filters separate and enabling this option to exclude the import records.

Rule to Merge Records

Cloudingo uses the default automation rule on all filters unless specified otherwise. You can configure custom automation rules and associate those rules to the filter by choosing the custom rule in this drop box. After saving this change, the filter results will always use the specified rule. See configuring single object merge automation rules and configuring cross-object convert automation rules for more info.


Click continue at the bottom of tab 1 to move to Tab 2.



Tab 2: What Fields do you Want to Match on?


Fields to Dedupe Against (Left Panel)

This panel lists your Salesforce standard and custom fields that can be used to identify duplicates. Select fields by dragging any from the left panel to the right. (Use the search option to quickly find fields.) 

Clicking 'Refresh Fields' will refresh the field schema. Cloudingo syncs with Salesforce, so if you add any new fields in Salesforce that you need to see right away, clicking this option will force the schema to sync for the object you are working in. The field should then show up in your field list.

Selected Fields (Right Panel)

Field selections work on an AND logic (meaning all criteria specified must be met in order to pull into the filter results).

When using ‘Equal’ or ‘Not Equal’, 'Contains', 'Does Not Contain', 'Starts With', or 'Ends With', you can specify multiple values (which will use 'or' logic), separated by commas and no spaces. For example, to examine records in the state of Texas only, drag the State field and set it to (Equal) ‘TX,Texas’ to find all possible matches. (When using 'Equal' or 'Not Equal', the values must be entered exactly as they appear in Salesforce in order to be detected.) This applies to all field types except multi-picklists.

Group Name Selection 

The Group Name can be changed in the ‘Group results by this field‘ drop box. The Group Name is the field value that will be displayed in the Merge Preview page when listing each unique group’s name.

Match



Select a matching style for Cloudingo to use in assessing field value matches.

Matching style definitions

Alpha Numeric OnlyStrips out special characters, symbols, and spaces.
Company Name CleanedStrips out standard corporate suffixes and articles, like Inc., Incorporated, Co., Company, The, A, etc. This list is customizable (we do not advise removing clean terms from the existing list without careful consideration). Words can be added to the list to better accommodate your business industry by going to the Admin menu, and choosing 'Synonyms' to search/edit the Company Clean list. See more details here within this article. If you want Cloudingo to also strip out all spaces from your values before making the comparison against other records, select the ‘Remove Spaces’ checkbox. (Example: BestBuy, Inc. vs Best Buy. Typically “company clean” would not catch this scenario because it only strips out the ‘, Inc.’ for comparison.)
Email (Username)Matches the username portion of a string of text (text preceding an @ symbol)
Email/Web (Domain)Matches the domain portion of a string of text (text that follows after an @ symbol).
ExactOne-to-one match of field data.
First N CharactersN stands for “number.” This number is user-defined. (Example: if 4 were selected as the variable, Cloudingo would only look at the first 4 letters of the field value.)
First N WordsN stands for “number.” This number is user-defined. (Example: if 2 were selected as the variable, Cloudingo would only look at the first 2 words of the field value.)
FuzzyUses an algorithm that calculates likely matches based on a threshold. (Note: it is recommended that you use this option last after several passes on the data as it returns the widest set of possible matches). See more details on Fuzzy logic here.
Last N CharactersN stands for “number.” This number is user-defined. (Example: if 4 were selected as the variable, Cloudingo would only look at the last 4 letters of the field value.)
Last N WordN stands for “number.” This number is user-defined. (Example: if 2 were selected as the variable, Cloudingo would only look at the last 2 words of the field value.)
Letters OnlyStrips out any non-letter characters, symbols, and spaces. (Example: ABC, A B C, A.B.C. would all be matches.)
Numeric OnlyStrips out any non-numeric characters, symbols, and spaces. (Example: 414, 4 1 4, 4.1.4 would all be matches.)
Phone InternationalA fuzzy matching algorithm specific to international phone numbers that accounts for differences in formatting and spacing.
Phone North AmericaA fuzzy matching algorithm specific to North American phone numbers that accounts for differences in formatting and spacing.
URLMatches a web site URL address.

Video: Matching Styles

Ignore Case

If selected, Cloudingo will ignore differences in upper and lower case letters. A=a, B=b
When enabled, it will still find matches if one record is in all caps vs. one that is all lower case (or title case).

Match on Empty Field

If selected, Cloudingo will include records where the selected field has no value. You can include up to two Match on Empty fields per filter configuration.

Cloudingo will first look for matches between records that contain values in the selected field. Cloudingo will then add to one of the groups those records that have no values for that field (records with empty fields).

When Match on Empty is selected, Cloudingo returns the following matches:

1. One or more records match for the field value value.
2. One or more records have a field value (that are the same) and one or more records have a blank field value.
3. All records are blank for the field value.

Important
Best practices determine that “Match on Empty Field” should only be used for one or two fields per filter. You will also get the best results by cleansing the data first without selecting “Match on Empty Field,” and then selecting it for a later scan and cleanse of the data.

Video: Match on Empty


Synonym Matching

One way Cloudingo recognizes that records are duplicates are through synonyms. Synonym matching can be used on any of these (standard, but not on custom) fields: 

  • First Names: When enabled, matches are identified between familiar and formal versions of names using a standard lexicon. For example, when selected, Robert and Bob will be matched. To use Name synonyms for your matching style in filter configurations, you must select Exact in the Match drop down menu first, then enable the 'Enable Synonym Matching' box.
  • State/Province:When enabled, matches are identified by comparing spelled out (full names) of states or provinces with their standard abbreviations. For example, Texas is matched with TX. To use State/Province synonyms for your matching style in filter configurations, you must select Exact in the Match drop down menu first, then enable the 'Enable Synonym Matching' box.
  • Country: When enabled, matches are identified by comparing spelled out (full names) of nations with their standard abbreviations. For example, United Sates, USA, and US are all matched. To use Country synonyms for your matching style in filter configurations, you must select Exact in the Match drop down menu first, then enable the 'Enable Synonym Matching' box.
  • Street: When using 'Fuzzy' matching style on any standard Street field, Cloudingo will automatically use a preset synonym list. Please see more details here within this help article.
  • Company Clean: When using 'Company Clean' matching style on Account Name or Company Name (standard fields), Cloudingo will automatically use a preset synonym list. Please see more details here within this help article.

You can manage the pre-set list of synonyms for each of these categories. To access the Synonym Lists, click Admin Synonyms and choose the list you want to view or edit from the drop down list.



Click Continue at the bottom of tab 2 to move to Tab 3.



Tab 3: Limit Your Records



You can define the "scope" of a filter’s scan of your Salesforce data. By default Cloudingo will scan your entire database (all leads, contacts, and accounts) for duplicates. But on Tab 3, you can define subsets of data for filtering and deduplication. For example, in orgs that break out Accounts into various Record Types, you may want to dedupe within a record type, but not across the whole object so that an Account can exist once in each Record Type.

  1. Drag the field defining the data point from the left to the right (from 'Fields to Filter Results' list to the 'Selected Criteria' area).
  2. Set the desired operator (options will vary depending on field type chosen). Examples include 'Equal', 'Not Equal', 'Greater Than', 'Contains', 'Within X Number of Days'.
  3. Enter required values.

When the scope is set in Tab 3, then ALL records within the group of matches must contain the criteria you specify.


Tips

  • When using a Salesforce ID number, set the operator to “Starts With.” Internally, Salesforce uses 18 digit IDs. Cloudingo reads the 18 digit IDs, however, Salesforce typically only exposes a 15-character version of those IDs to users.

  • When using ‘Equal’ or ‘Not Equal’, 'Contains', 'Does Not Contain', 'Starts With', or 'Ends With', you can specify multiple values, separated by commas and no spaces. For example, to examine records in the state of Texas only, drag the State field and set it to (Equal) ‘TX,Texas’ to find all possible matches. (When using 'Equal' or 'Not Equal', the values must be entered exactly as they appear in Salesforce in order to be detected.) This applies to all field types except multi-picklists.

  • Multi-picklist field options include ‘Excludes Value(s)’ & ‘Includes Value(s)’. When using ‘Equal’ using multi-picklist values, all values must be present on the records for the match to be identified. When using ‘Includes Value(s)’, ‘or’ logic is used and will identify record matches where any record has a value set to one of the specified picklist values. 



Click Continue at the bottom of tab 3 to move to tab 4, or hit Save to save the filter without recalculating, or Save & Run to save the filter and recalculate the filter results (this will not start the job, only query for new results).



Tab 4: Limit Your Groups



Tab 4 is helpful in very specific use-cases where you cannot identify or eliminate a group of matches any other way. Remember, a group is the set of duplicate records that match the fields set on Tab 2 and that fall into the scope set on Tab 3.

Limit Group Size

To further limit your filter by only displaying groups where the number of matching records is greater than, less than, or within a certain number of records, choose the ‘Limit group size’ option and specify the record count limit. For example, you can choose ‘Less than or equal to’ 5, which will limit your results to only the groups that have less than 5 matching records.

Add Group Rule

You can further narrow your group results by clicking on the Add Group Rule option. The Field drop box will allow you to choose any field within Salesforce.

An example for using a group limitation would be if you have a loose filter that requires manual review, and you only want to see records where at LEAST one record out in the group was created or modified after a set date. You would choose ‘Last Modified Date’ as the field, and ‘One or more records have a date greater than or equal to’ as the rule, and a specific date. Then update this date each time you need to manually review and manage the results.

Multi-picklist field options include ‘One or more records excludes a value’ and ‘One or more records includes a value’. When using ‘One or more records has an exact value’ using multi-picklist values, all values must be present on the records for the match to be identified. When using one of the other options, ‘or’ logic is used and will identify all record matches where any record has a value set to one of the specified picklist values.

Add as many group rules as necessary by clicking the ‘Add Group Rule’ option and selecting the field specifications.



Click Save to save the filter without recalculating, or Save & Run to save the filter and recalculate the filter results (this will not start the job, only query for new results).



Special Considerations

Fuzzy Matching

Use fuzzy matching as a last matching option to find any stray duplicates (on any non-street field).

“Fuzzy” uses a Soundex-Sounds like propriety algorithm. The algorithm estimates how something would sound if pronounced out loud. It is most useful for instances when you would be manually reviewing results to find deeply hidden matches (for example records that may just have a simple misspelling in the value). The results can be very broad, and can frequently return false positives. 

For this reason, Cloudingo will not allow the Fuzzy matching style to be chosen on a field when there is not more than one field chosen for matching within the filter configuration. You must have at least two field matches specified (ex: you cannot do a Name Fuzzy match only without email, phone, or some other field match included). Fuzzy also cannot be designated as your first field's matching style in the filter configuration. It can only be chosen as a matching style one of the filter’s secondary field selections. If you need to use Fuzzy on a field, just be sure to add another field to the 'selected' fields within your criteria first in the list. 

Using fuzzy matching on street addresses



Cloudingo offers a more unique and configurable type of fuzzy logic on Street address fields. For more information on configuring fuzzy for street fields, click here.

An example of a match that fuzzy (on Street) would find is 123 Main St. vs. 123 Main Street. Cloudingo will ignore St. & Street to find matches based on the ‘123 Main’ portion of the field value. The fuzzy street logic ignores the portions of the field value that are listed in your synonym matches BEFORE it tries to make the comparison on the field values. 

When using ‘Fuzzy’ as your matching option within your filter configuration on a Street field, Cloudingo allows you some flexibility on how to determine the logic.

Cloudingo uses a pre-set list of synonyms (like Avenue vs. Ave., Street vs St., etc.) The list is configurable. To add additional words to the matching algorithm, manage the list by clicking Admin → Synonyms. Select ‘Address Synonyms’ in the drop down menu below ‘Manage Synonyms’.


Video: Using Fuzzy Matching on Street

Video: Using Synonyms to Find Dupes


Company Clean

When using Company Clean, Cloudingo uses a pre-defined set of words (like 'Inc., Co., LLC, Group, etc.) to find additional matches. One example would be Nike vs. Nike, Inc. would be detected using Company Name Clean, but not using Exact match. The list is configurable. To add additional words to the matching algorithm, manage the list by clicking Admin → Synonyms. Select ‘Company Clean’ in the drop down menu below ‘Manage Synonyms’.

Warning!
We do not advise removing keywords from the pre-defined list without very careful consideration. We also recommend carefully thinking through every possible scenario when adding new words to this list. This customization should generally be used for special one-off situations. It would not be typical to make a large number of modifications to this list. The Company Clean matching algorithm is intended to be used for Company/Account Names only.


 
Our pre-defined list will appear. Expand any section by clicking the arrow to the left of the keyword placement options. Consider all scenarios where this might be a problem before adding new words here. 

  • Starts with: Click in the box over the 'add synonym' text to add text that appears in the beginning of your company/account string that you want to stripped out for comparison.
  • Replace if not first or last word: Click in the box over the 'add synonym' text to add a word that appears in the middle (anywhere that is not in the beginning or ending portion) of your company/account string that you want to stripped out for comparison. Cloudingo determines a string is a word if a space exists before and after the string specified, and will also strip out the spaces with the string for comparison.
  • Ends with: Click in the box over the 'add synonym' text to add text that appears in the end of your company/account string that you want to stripped out for comparison. 
  • Contains: Click in the box over the 'add synonym' text to add text that appears anywhere in your company/account string that you want to stripped out for comparison. Cloudingo will strip out the string specified (even if the string is not considered a word, by definition: separated by spaces before and after the string).


Indexing

When creating or editing a filter using new fields that have not previously been configured in another filter (or set to be indexed) before, Cloudingo automatically initiates a re-indexing process on the object when the filter is saved. This process may take some time. To prevent this automatic re-indexing every time you edit/create a new filter, you can pre-set a list of fields that you will be using in your filters. 

Go to Admin → Objects. Click on the object you will be creating/editing filters for, then click ‘Edit Fields’ to add additional fields that will be used in your filter configurations. This will trigger a one time re-index upon saving, but will prevent further re-indexes as you work on your configurations. For more details, please visit our help article here.

Warning
Use caution when setting up filters to use formula, calculated, or related object field values. If the field is a reference field rather than a physical field on the object, changes to the field value will NOT trigger syncs to Cloudingo. Using calculated fields on an ongoing basis may require re-indexing the object by Cloudingo support.


Specific Scenarios

Excluding Email Domains in Filters

If you would like to find duplicates based on matching email domains, you can exclude multiple domains (such as gmail.com, yahoo.com, hotmail.com, etc.) to narrow down your search.

Step 1

In Salesforce you will need to create a new custom field on the object called Email Domain with the following settings: (Note: If your formula fails in Step 3 below, try re-typing the formula from below instead of copying/pasting, OR use single instead of double quotes around the '@' sign.)

  1. Type = Formula
  2. Return Type = Text
  3. Advanced Formula: RIGHT(Email, LEN(Email) - FIND("@", Email))

Step 2

When creating a filter in Cloudingo, on Step 3 drag and drop Email Domain to the Selected Criteria column. Select "Not Equal" in the dropdown and type the domains you'd like to exclude. Separate the domain values by commas with no spaces (example: gmail.com,yahoo.com,hotmail.com).

 

Using Email Username to Find Duplicates

A common method to find Salesforce duplicates is to use the username (also called the Email ID) portion of an email address (the part before the @ sign) to find matches that have different email domains (the part after the @ sign). 

For example, jwilliams@company.com and jwilliams@company.co.uk. Both email addresses have the same username but different variations in the domain portion.

Step 1

In Salesforce you will need to create a new custom field on the object called Email Username with the following settings: (Note: If your formula fails in Step 3 below, try re-typing the formula from below instead of copying/pasting, OR use single instead of double quotes around the '@' sign.)

  1. Type = Formula
  2. Return Type = Text
  3. Advanced Formula: LEFT(Email,FIND("@",Email) - 1)

Step 2

When creating a filter in Cloudingo, on Step 2 drag and drop Email Username to the Selected Criteria column. (Note: It is recommended that you use additional filter criteria such as name, company name, or some other identifier in addition to this field.)




Videos


Video: Filter Best Practices


Video: Cross-field Matching



Video: Matching Styles


Video: Match on Empty


Video: Using Fuzzy Matching on Street


Video: Using Synonyms to Find Dupes


Video: Refining Filters - Subsets & Advanced Group Rules


What is Next?

Filter configuration is the most important feature that you need to understand. Now that you have mastered Filters, you are ready to move on and learn about Merge Rules (the second most important feature to understand within Cloudingo)!

To learn more about Merge Rules, please visit our help article here.

Was this article helpful?


What's Next