Marketo

Purpose

The Cloudingo Marketo Integration Web Service API provides Enterprise customers the ability to use the Cloudingo platform from their own Marketo instances. The power to find duplicates, merge data into records, and manage data does not have to be done from the Cloudingo website alone.

System Design

The Cloudingo platform runs in its own cloud, separate from Salesforce. The setup and configuration involves connecting Cloudingo to your Salesforce cloud and creating endpoints in Cloudingo for your Marketo instance to connect to.

Proprietary Indexing

Cloudingo leverages its own indexes for finding duplicates. These proprietary indexes allow for more complex duplicate scenarios than can be supported in Salesforce Apex and SOQL. The Cloudingo system is also unrestricted by Salesforce governance (CPU limits, query result sizes, looping maximums) because indexes on the Cloudingo cloud servers are used to perform searches.

Newly created or modified records in Salesforce result in index updates in the Cloudingo platform. The speed at which these updates occur is dependent on the system load for both the Salesforce and Cloudingo servers, as well as the Cloudingo synchronization mode being used for your account. An understanding of index updates is fundamental to understanding results. Usually, the timing of index updates is not an issue, but when data is flowing into Salesforce from multiple sources simultaneously, duplicate detection will depend on the indexes being up-to-date.

Web Service Technologies

The Cloudingo Web Service API is provided to customers via standard web services technologies. Cloudingo supports both REST and SOAP technologies. However, for Marketo Integration, REST is recommended and examples below will be provided using REST as a result.

Dual Support for REST/JSON and SOAP/XML

The dual support for REST and SOAP allows customers to program against our platform in the most comfortable way for that customer. The REST endpoints use JSON as the data format, as is the most common convention in modern web programming. The SOAP endpoints use XML as the data format. SOAP provides WSDL definitions of the web service interface which can be used by tools to generate proxy code. These proxies are convenient for interacting with the web service in a way that is natural to the programming language being used by our customers.

Navigation

1. The Cloudingo Integration dashboard can be accessed by logging into https://app.cloudingo.com. Once logged in, the user is presented with the Dedupe dashboard by default. To access the Cloudingo integration dashboard, choose the 'Integrations' → 'Manage' option from the left-hand-side navigation window.
   
   
   
   
2. At the top of the page, choose the 'Marketo' tab.
   
   
   
3. If no integrations have previously been created, the screen will automatically display an option to 'Create New'. The option is also available under the 'Actions' menu.
   
   
   
4. After choosing the option to create a new Marketo Integration, you will be taken to the Marketo Integration Create screen.

Step 1: Marketo Settings

1. In Marketo, create a new custom webservice and capture details.
   1. Navigate to the 'Admin' → 'Integration' → 'LaunchPoint' section in the left-hand-side navigation window and create a new custom webservice. 
      1. Keep this page open to gather details for Cloudingo.
         
   2. In Cloudingo:
      1. Marketo Client ID: Populate the Client ID from Marketo.
      2. Marketo Client Secret: Populate Client Secret from Marketo.
      3. Marketo Authorized User: Populate the Authorized User from Marketo.
2. In Marketo, gather webservice details.
   1. Navigate to 'Web Services' from the Integration section in the left-hand-side navigation window.
      
   2. In Cloudingo: 
      1. Marketo API Endpoint: Populate the REST API Endpoint as provided in Marketo under the Web Services Integration. 
      2. Marketo Identity Endpoint: Populate the REST API Identity Endpoint as provided in Marketo under the Web Services Integration.
         
         
         Note:
         The remaining steps below are set in Cloudingo.

         Creating and testing the webhook is explained later in this article where the Cloudingo Integration Key and Password will be needed to plug into Marketo.

         
         

3. Marketo Token: This value will automatically populate with the Marketo Identity Token from the last successful Marketo connection established for the Cloudingo Integration.
   1. Value is read-only and will get populated when the connection is tested.
4. Marketo Token Expiration: This value will automatically populate the current expiration for the latest Marketo Identity Token.
   1. Value is read-only and will get populated when the connection is tested.
5. Confirm Client Connection: Choose this option to test the Cloudingo connection to Marketo with the identity endpoint and credentials provided. 
   1. If the connection is successful, the Marketo Token and Marketo Token Expiration will be updated automatically.
6. Marketo daily API call limit (optional): Value can be adjusted downward to limit the number of API calls Marketo can use within a 24-hour period. 
   1. The limit cannot exceed the default maximum established for your Account.
7. Merge all dupes detected: Choose this option to have Cloudingo merge all duplicates detected with the incoming record.
   1. The standard/default behavior if deselected will merge the incoming record with only the Master record duplicate detected. 
8. Use Master Lead Score in Marketo: Choose this option to update the merged record with the lead score of the Master record.
   1. The standard/default behavior if deselected will sum the lead score value from all records merged. 

Step 2: Cloudingo Settings

1. Integration Name in Cloudingo: Give the integration a descriptive name.
2. Salesforce Account: Choose the desired Account that the integration will apply.
   1. If only one org is connected to Cloudingo, it will default automatically.
3. Cloudingo Integration Key: This is the unique identifier for the integration. 
   1. A key will be populated by default but a new key can be regenerated if desired.
   2. This key will be sent as part of the headers when calling the Cloudingo Integration API.
4. Cloudingo Integration Password: Enter any password.
   1. Password entries will be automatically encrypted. 
      1. Plain text version of the password is never exposed but can be viewed from the integration configuration using the 'show password' icon.
      2. In conjunction with the Cloudingo Integration Key, the (encrypted) Integration Password will authenticate any incoming API calls.
5. Enable Integration in Cloudingo: Choose this option to enable the integration. 
   1. Deselect to disable the integration.
6. Cloudingo daily API call limit: Value can be adjusted downward to limit the number of API calls that Cloudingo can use within a 24-hour period. 
   1. API call limit cannot exceed the default maximum established for the Account.
7. Allowed API Actions: 
   1. Currently, there is only one option for Marketo Integrations and the 'Check for Duplicates' option must be selected.
   2. The action listed below is enabled by default when a new Cloudingo Marketo Integration has been created. 
   3. Available actions for a particular integration can be limited via the Cloudingo Integration Wizard.
   4. HTTP Post should be used for all actions.

Step 3: Duplicate search settings

1. Check for Lead duplicates: Choose this option to scan incoming records against existing Leads in Salesforce to identify a match.
2. Check for Account duplicates: Choose this option to scan incoming records against existing Accounts in Salesforce to identify a match.
3. Check for Contact duplicates: Choose this option to scan incoming records against existing Contacts in Salesforce to identify a match.
4. Validate Addresses: Choose this option to perform the validate addresses process on the incoming and/or billing address records.
   1. Validated address will be included as part of the API response so that it can be mapped to update the Marketo record.
   2. Validation credits/tokens must be purchased to use this feature. 
5. Append Account Info to Marketo Records: Choose this option to have the API add/append any available Account data from Salesforce based on the company name provided and Account search options selected. 

Step 4: Object settings

Account setup

1. Options only appear if 'Check for Account Duplicates' is enabled in Step 2.
2. Filter: Choose an existing filter or create custom filter logic to use to identify duplicate matches.
   1. Existing: Choose an existing dedupe filter. The filter name & user dashboard where the filter is located will be displayed for easy selection. The rule already assigned to the pre-configured filter will automatically get applied for merging logic (master and field selections).
      1. Filters using 'Match on Empty' in any of the matching criteria will not appear. API does not support Match on Empty. 
   2. Custom: Choose 'Custom' in the drop box to build new filter logic. Matching criteria and options are limited. If needing to use options other than 'Exact' and 'Ignore Case', pre-build a dedupe filter and choose the filter from the list. 
      1. Automation Rule: Choose a rule from the drop box to tell Cloudingo how to merge the records (master and field selections).
      2. Custom Account Filter - Matching Fields: Choose the 'Add Field' option to add matching criteria.
         1. Account Field: Choose a field for matching criteria in the drop box.
         2. Matching Style: Choose a matching style (Exact).
         3. Other Options: Choose desired options (Ignore Case).
         4. Repeat steps until all matching criteria are defined.
      3. Custom Account Filter - Limit Records (Optional): Choose the 'Add Field' option to add limit criteria.
         1. Account Field: Choose a field in the drop box.
         2. Condition: Choose a condition (Ex: Equals, Not Equals, Has a Value, Has no Value).
         3. Repeat steps until all limits are defined.

Lead setup

1. Options only appear if 'Check for Lead Duplicates' is enabled in Step 2.
2. Filter: Choose an existing filter or create custom filter logic to use to identify duplicate matches.
   1. Existing: Choose an existing dedupe filter. The filter name & user dashboard where the filter is located will be displayed for easy selection. The rule already assigned to the pre-configured filter will automatically get applied for merging logic (master and field selections).
      1. Filters using 'Match on Empty' in any of the matching criteria will not appear. API does not support Match on Empty. 
   2. Custom: Choose 'Custom' in the drop box to build new filter logic. Matching criteria and options are limited. If needing to use options other than 'Exact' and 'Ignore Case', pre-build a dedupe filter and choose the filter from the list. 
      1. Automation Rule: Choose a rule from the drop box to tell Cloudingo how to merge the records (master and field selections).
      2. Custom Lead Filter - Matching Fields: Choose the 'Add Field' option to add matching criteria.
         1. Lead Field: Choose a field for matching criteria in the drop box.
         2. Matching Style: Choose a matching style (Exact).
         3. Other Options: Choose desired options (Ignore Case).
         4. Repeat steps until all matching criteria are defined.
      3. Custom Lead Filter - Limit Records (Optional): Choose the 'Add Field' option to add limit criteria.
         1. Lead Field: Choose a field in the drop box.
         2. Condition: Choose a condition (Ex: Equals, Not Equals, Has a Value, Has no Value).
         3. Repeat steps until all limits are defined.

Contact setup

1. Options only appear if 'Check for Contact Duplicates' is enabled in Step 2.
2. Filter: Choose an existing filter or create custom filter logic to use to identify duplicate matches.
   1. Existing: Choose an existing dedupe filter. The filter name & user dashboard where the filter is located will be displayed for easy selection. The rule already assigned to the pre-configured filter will automatically get applied for merging logic (master and field selections).
      1. Filters using 'Match on Empty' in any of the matching criteria will not appear. API does not support Match on Empty. 
   2. Custom: Choose 'Custom' in the drop box to build new filter logic. Matching criteria and options are limited. If needing to use options other than 'Exact' and 'Ignore Case', pre-build a dedupe filter and choose the filter from the list. 
      1. Automation Rule: Choose a rule from the drop box to tell Cloudingo how to merge the records (master and field selections).
      2. Custom Contact Filter - Matching Fields: Choose the 'Add Field' option to add matching criteria.
         1. Contact Field: Choose a field for matching criteria in the drop box.
         2. Matching Style: Choose a matching style (Exact).
         3. Other Options: Choose desired options (Ignore Case).
         4. Repeat steps until all matching criteria are defined.
      3. Custom Contact Filter - Limit Records (Optional): Choose the 'Add Field' option to add limit criteria.
         1. Choose 'Add Field'.
         2. Field: Choose a field in the drop box.
         3. Condition: Choose a condition (Ex: Equals, Not Equals, Has a Value, Has no Value).
         4. Repeat steps until all limits are defined.

Show Marketo Payload Template

1. At the bottom of Step 4, an option is available to view a payload template. 
2. This will provide a JSON string which can be used when setting up the Marketo Webhook. 
3. The template is built on the basis of duplicate search rules and filters selected.

Integration Dashboard

After a Marketo Integration has been created, it will display on the Integration dashboard similarly to the following:

This will provide a brief summary of the configured integration including the following:

1. The name of the integration.
2. The actions that have been selected.
3. The current status of the integration.
4. The current API call limit for the integration.
5. The number of calls made for the integration in the last 24 hours.
6. The number of calls made to the connection system in the last 24 hours.

Step 5: Create Marketo Webhook

1.  Create a custom field in Marketo for the webhook to update the incoming record.
   1. In Marketo, navigate to the 'Admin' → 'Database Management' → 'Field Management' → 'New Custom Field' option. 
   2. Give the field a descriptive name and set it to be a Boolean field type. 
      1. Field value returned confirms if the record is a duplicate or not (true/false).
2. Create a Marketo webhook to initiate the call to the Cloudingo Integration.
   1. In Marketo, navigate to the 'Admin' → 'Integration' → 'Webhooks' section in the left-hand-side navigation window.
   2. Create a new webhook to call the Cloudingo MarketoCheckForDuplicates service.
      1. The webhook will provide a record-level check to allow for Cloudingo duplicate prevention as part of Marketo workflows.
         
      2. The template will be customized for the dedupe filters but should follow the structure of the input for the Marketo CheckForDuplicates method (shown below). 
      3. At minimum, field values chosen for filtering criteria should be sent to the webhook as well as the MarketoId. 
         1. Use the 'Show Marketo Payload Template' option in Step 4 within Cloudingo Integration configuration to obtain content for this template.
3. Add a response mapping to update the custom Boolean field in Marketo.
   
4. If using the Validate Addresses feature, include additional response mappings for the Lead address and/or Company billing address.
5. Set up a custom header to allow for authentication between Marketo and Cloudingo.
   1. Navigate to 'Admin' → 'Integration' → 'Webhooks' → 'Webhook Actions' → 'Set Custom Header'.
      
      
   2. In this screen the following 3 headers should be added:
      1. Cloudingo Integration Key: This will be displayed within the Cloudingo Integration configuration in Step 1.
      2. Cloudingo Integration Password: This will displayed within the Cloudingo Integration configuration in Step 1.
      3. Content-Type: this should be set to application/json assuming that JSON is the communication option chosen.

Step 6: Create Marketo Workflows

1. Create two Smart Campaigns. It is recommended to create a separate Program in Marketo for these Smart Campaigns. 
   1. The first will initiate the call to the webhook and perform the duplicate check and respond if an existing duplicate record was detected.
   2. The second will allow for the sync to Salesforce if no duplicate is detected.

Example (other workflows can be established as desired):

1. Send to Cloudingo:
   1. Smart List: Select the Lead is Created option to determine the Leads that qualify to be sent to Cloudingo. This will cause any newly created Lead to initiate the workflow.
   2. Flow: Select the option to Call Webhook and select the Webhook created to call Cloudingo in order to dedupe the new Lead record.
   3. Schedule: Set for each Lead to run through the workflow once in order to avoid unnecessary overhead on Leads that have already been deduped, and activate the campaign workflow.
2. Sync to Salesforce: Once the Cloudingo API call completes, the custom Boolean field will be updated with a True/False to indicate if a duplicate was found according to the response mapping for the Marketo Webhook.
   1. Smart List: A Data Value Changes trigger should be added to initiate the workflow when the custom Boolean value is changed. 
      1. If the value is True (No duplicate found), then the record has been deduped by Cloudingo and the record qualifies to be synced to Salesforce.
   2. Flow: The Sync Lead to SFDC option should be added to move the record to Salesforce since the record does not have a current duplicate in Salesforce.
   3. Schedule: Set for each Lead to run through the workflow once in order to avoid unnecessary overhead on Leads that have already been deduped, and activate the campaign workflow.

More Information:

Endpoints

The Cloudingo Integration API is currently available via SOAP (XML based) or REST (JSON based) service endpoints. The functionality and methods available for both are identical, however the end point as well as the manner for initiating the call and handling return values will vary based on the type of service. All endpoints are handled over SSL.

REST Endpoints

Production Environment: https://api.cloudingo.com/rest/MarketoCheckForDuplicates

SOAP Endpoints

Prod Environment: https://api.cloudingo.com/soap/

Prod Environment WSDL: https://api.cloudingo.com/wsdl

Security

Only SSL (https) communication is available for the Marketo Integration API. Security for the Marketo Integration API is handled via a combination of the Cloudingo Integration Key and the Cloudingo Integration Password which should be added to the headers of any REST or SOAP service call. The following headers should be applied as HTTP headers.

Service Call Headers

CloudingoIntegrationKey 

The auto-generated GUID provided by the Cloudingo Integration Dashboard within the Cloudingo web portal.

CloudingoIntegrationPassword

The supplied password to be used to authenticate the caller in combination with the integration key. The password can be entered on the Cloudingo Integration Dashboard and the value will be encrypted. The encrypted value should be passed in the header.

CloudingoApiVersion (optional) 

The version of the API to call. If you are planning on continuing with the version available during setup, this should be supplied with all calls. If not provided, this will default to the latest version of the API. Therefore, if not provided, a new API version release will result in all calls automatically upgrading to the latest API. If this is not desired, the version can be supplied so when to upgrade the API version is controlled by the caller.

Content-type

applicaton/json

Available Actions (Detailed Inputs, Outputs, Response Codes)

We previously described the available actions in this article. This section will detail the inputs, outputs, & method specific response codes for each action.

MarketoCheckForDuplicates

Will evaluate the current data in your organization based on the inputs to determine any possible duplicates using configured duplicate definition. If configured:

1. Inputs:
   1. Marketo RecordInfo: Name value pairs of fields and their corresponding values for the new record along with the current Marketold being evaluated.
   2. JSON Example:
      
2. Outputs:
   1. An Integration status code and message for the overall operation.
   2. The inputs provided for the duplicate check
   3. A list of potential duplicates
   4. Marketo Result Info object providing values from address validation if configured to validate addresses.
   5. Boolean flag to identify if a duplicate was found
   6. JSON Example:
      
3. Method Specific Response Codes(Detailed below):
   1. SuccessNoActionNeeded
   2. ActionCannotBeCompleted
   3. ErrorMergeFailed

Action Workflows

The following section provides details on the overall workflows for particular Cloudingo API actions. While separate actions are provided, in some scenarios the actions work together in order to provide overall functionality depending on each client’s usage pattern.

Marketo Duplicate Check and Merge Workflow

Limits and Conditions 

There are a number of checks that occur with each API call as well as Cloudingo API limits and Salesforce API limits. As a result, there is an overall process flow involved with performing actions through the Cloudingo Integration API when these checks come into play and a number of potential return messages if limits or conditions are met. The following is a list of the primary call limits:

1. Daily API Call Limit: based on a 24-hour rolling period. The exact value will depend on the level purchased.
2. 5-minute period: limited to 1000 API calls in any 5 minute period.
3. Concurrent API Calls: limited to 100 API calls processing at any given time.

Limits and Conditions Workflow

Return Statuses

The return details will be provided with each return method. Those listed below without a corresponding action apply to any action. Any method level error response will have the same structure as shown below. Non-method level errors will be listed in the corresponding status and status message as a part of the response object for the specific method. General Error Response JSON:

For SOAP, exceptions will all be WebFaultExceptions of type CloudingoApiError which will include a Message, ReferenceNumber, and Status.