Integration Patterns Overview

Typically, when you implement Salesforce, you need to integrate it with other applications. Although each integration scenario is unique, there are common requirements and issues that developers and architects must resolve.

This document describes strategies (in the form of patterns) for these common integration scenarios. Each pattern describes the design and approach for a particular scenario rather than detailing a specific implementation. In this document you’ll find:

• A number of patterns that address key “archetype” integration scenarios
• A selection matrix to help determine which pattern best fits your scenario
• Integration tips and best practices

Purpose and Scope

This document is for designers and architects who need to integrate the Salesforce Platform with other applications in their enterprise. This content is a distillation of many successful implementations by Salesforce architects and partners.

To become familiar with integration capabilities and options available for large-scale adoption of Salesforce-based applications, read Pattern Summary and Pattern Selection Guide. Architects and developers should consider these pattern details and best practices during the design and implementation phase of a Salesforce integration project.

If implemented properly, these patterns enable you to get to production as fast as possible and have the most stable, scalable, and maintenance-free set of applications possible. Salesforce’s own consulting architects use these patterns as reference points during architectural reviews and actively maintain and improve them.

As with all patterns, this content covers most integration scenarios, but not all. While Salesforce allows for user interface (UI) integration, mashups, for example,such integration is outside the scope of this document.

Pattern Template

Each integration pattern follows a consistent structure. This provides consistency in the information provided in each pattern and also makes it easier to compare patterns.

Name

The pattern identifier that also indicates the type of integration contained in the pattern.

Context

The overall integration scenario that the pattern addresses. Context provides information about what users are trying to accomplish and how the application will behave to support the requirements.

Problem

The scenario or problem (expressed as a question) that the pattern is designed to solve. When reviewing the patterns, read this section to quickly understand if the pattern is appropriate for your integration scenario.

Forces

The constraints and circumstances that make the stated scenario difficult to solve.

Solution

The recommended way to solve the integration scenario.

Sketch

A UML sequence diagram that shows you how the solution addresses the scenario.

Results

Explains the details of how to apply the solution to your integration scenario and how it resolves the forces associated with that scenario. This section also contains new challenges that can arise as a result of applying the pattern.

Sidebars

Additional sections related to the pattern that contain key technical issues, variations of the pattern, pattern-specific concerns, and so on.

Example

An end-to-end scenario that describes how the design pattern is used in a real-world Salesforce scenario. The example explains the integration goals and how to implement the pattern to achieve those goals.

Pattern Summary

The following table lists the integration patterns contained in this document.

List of Patterns

Pattern Approach

The integration patterns in this document are classified into three categories:

• Data Integration—These patterns address the requirement to synchronize data that resides in two or more systems so that both systems always contain timely and meaningful data. Data integration is often the simplest type of integration to implement, but requires proper information management techniques to make the solution sustainable and cost effective. Such techniques often include aspects of master data management (MDM), data governance, mastering, de-duplication, data flow design, and others.

• Process Integration—The patterns in this category address the need for a business process to leverage two or more applications to complete its task. When you implement a solution for this type of integration, the triggering application has to call across process boundaries to other applications. Usually, these patterns also include both orchestration (where one application is the central “controller”) and choreography (where applications are multi-participants and there is no central “controller”). These integrations often require complex design, testing, and exception handling requirements. Also, such composite applications are typically more demanding on the underlying systems because they often support long-running transactions, and the ability to report on or manage process state.

• Virtual Integration—The patterns in this category address the need for a user to view, search, and modify data that’s stored in an external system. When you implement a solution for this type of integration, the triggering application has to call out to other applications and interact with their data in real time. This type of integration removes the need for data replication across systems, and means that users always interact with the most current data.

Choosing the best integration strategy for your system isn’t trivial. There are many aspects to consider and many tools that can be used, with some tools being more appropriate than others for certain tasks. Each pattern addresses specific critical areas including the capabilities of each of the systems, volume of data, failure handling, and transactionality.

Pattern Selection Guide

The selection matrix tables list the patterns and their key aspects to help you determine which pattern best fits your integration requirements. The patterns are categorized using these dimensions.

Integrating Salesforce with Another System

This table lists the patterns and their key aspects to help you determine which pattern best fits your requirements when your integration is from Salesforce to another system.

Integrating Another System with Salesforce

This table lists the patterns and their key aspects to help you determine the pattern that best fits your requirements when your integration is from another system to Salesforce.

Middleware Terms and Definitions

This table lists some key terms related to middleware and their definitions with regards to these patterns.

Remote Process Invocation — Request and Reply

Context

You use Salesforce to track leads, manage your pipeline, create opportunities, and capture order details that convert leads to customers. But, the Salesforce system doesn’t contain or process orders. After the order details are captured in Salesforce, the order is created in the remote system, which manages the order to conclusion.

When you implement this pattern, Salesforce calls the remote system to create the order and then waits for successful completion. If successful, the remote system synchronously replies with the order status and order number. As part of the same transaction, Salesforce updates the order number and status internally. The order number is used as a foreign key for subsequent updates to the remote system.

Problem

When an event occurs in Salesforce, how do you initiate a process in a remote system, pass the required information to that process, receive a response from the remote system, and then use that response data to make updates within Salesforce?

Forces

Consider the following forces when applying solutions based on this pattern.

• Does the call to the remote system require Salesforce to wait for a response before continuing processing? Is the call to the remote system a synchronous request-reply or an asynchronous request?
• If the call to the remote system is synchronous, does Salesforce have to process the response as part of the same transaction as the initial call?
• Is the message size small or large?
• Is the integration based on the occurrence of a specific event, such as a button click in the Salesforce user interface, or DML-based events?
• Is the remote endpoint able to respond to the request with low latency? How many users are likely to be executing this transaction during a peak period?

Solution

This table contains solutions to this integration problem.

Sketch

This diagram illustrates a synchronous remote process invocation using Apex calls.

Salesforce Calling Out to a Remote System

Download Diagram

In this scenario:

• An action is initiated on the Lightning page (for example, a button click).
• The browser (via a client-side controller in the case of a Lightning component) performs an HTTP POST that in turn performs an action on the corresponding Apex controller.
• The controller invokes the actual call to the remote web service.
• The response from the remote system is returned to the Apex controller. The controller processes the response, updates data in Salesforce as required, and re-renders the page.

In cases where the subsequent state must be tracked, the remote system returns a unique identifier that’s stored in the Salesforce record.

Results

The application of the solutions related to this pattern allows for event-initiated remote process invocations in which Salesforce handles the processing.

Calling Mechanisms

The calling mechanism depends on the solution chosen to implement this pattern.

Error Handling and Recovery

It’s important to include an error handling and recovery strategy as part of the overall solution.

• Error handling—When an error occurs (exceptions or error codes are returned to the caller), the caller manages error handling. For example, an error message displayed on the end user’s page or logged to a table requiring further action.

• Recovery—Changes aren’t committed to Salesforce until the caller receives a successful response. For example, the order status isn’t updated in the database until a response that indicates success is received. If necessary, the caller can retry the operation.

Idempotent Design Considerations

Idempotent capabilities guarantee that repeated invocations are safe. If idempotency isn’t implemented, repeated invocations of the same message can have different results, potentially resulting in data integrity issues. Potential issues include the creation of duplicate records or duplicate processing of transactions.

It’s important to ensure that the remote procedure being called is idempotent. If the call is made by user interface, we need to take care of idempotency at the integration layer especially if there is no guarantee that Salesforce only calls one time.

The most typical method of building an idempotent receiver is for it to track duplicates based on unique message identifiers sent by the consumer. Apex web service or REST calls must be customized to send a unique message ID.

In addition, operations that create records in the remote system must check for duplicates before inserting. Check by passing a unique record ID from Salesforce. If the record exists in the remote system, update the record. In most systems, this operation is termed as an upsert operation.

Security Considerations

Any call to a remote system must maintain the confidentiality, integrity, and availability of the request. The following security considerations are specific to using Apex SOAP and HTTP calls in this pattern.

• One-way SSL is enabled by default, but two-way SSL is supported with both self-signed and CA-signed certificates to maintain authenticity of both the client and server.

• To streamline your Apex code and simplify the setup of authenticated callouts, specify a named credential in the callout endpoint.

• Consider the use of OAUTH 2.0 as the authentication mechanism for integration to external systems.

• Where necessary, consider using one-way hashes or digital signatures using the Apex Crypto class methods to ensure request integrity.

• The remote system must be protected by implementing the appropriate firewall mechanisms. See Security Considerations.

• Salesforce currently doesn’t support WS-Security. While it does not natively generate these complex WS-Security headers or automatically enforce them from an incoming WSDL, you can manually construct and implement them by creating custom Apex classes to handle the SOAP headers and security tokens for incoming requests.

Sidebars

Timeliness

Timeliness is of significant importance in this pattern. Usually:

• The request is typically invoked from the user interface, so the process must not keep the user waiting.
• Salesforce has a configurable timeout of up to 120 seconds for calls from Apex.
• Completion of the remote process is executed in a timely manner to conclude within the Salesforce timeout limit and within user expectations.
• External calls are subject to Apex synchronous transaction governor limits, so make sure to mitigate the risk of instantiating more than 50 transactions that run for more than five seconds each. In addition to ensuring the external endpoint is performant, options to mitigate the risk of a timeout include:
  • Setting the timeout of the external callout to five seconds.
  • Using a continuation in Lightning Components to handle long-running transactions

Data Volumes

This pattern is used primarily for small volume, real-time activities, due to the small timeout values and maximum size of the request or response for the Apex call solution. Don’t use this pattern in batch processing activities in which the data payload is contained in the message.

Endpoint Capability and Standards Support

The capability and standard support for the endpoint depends on the solution that you choose.

State Management

When integrating systems, keys are important for ongoing state tracking. There are two options.

• Salesforce stores the remote system’s primary or unique surrogate key for the remote record.
• The remote system stores the Salesforce unique record ID or some other unique surrogate key.

There are specific considerations for handling integration keys, depending on which system contains the master record, as shown in the following table.

Complex Integration Scenarios

In certain cases, the solution prescribed by this pattern can require the implementation of several complex integration scenarios. This is best served by using middleware or having Salesforce call a composite service. These scenarios include:

• Orchestration of business processes and rules involving complex flow logic
• Aggregation of calls and their results across calls to multiple systems
• Transformation of both inbound and outbound messages
• Maintaining transactional integrity across calls to multiple systems

Governor Limits

For information about Apex limits, see Execution Governors and Limits in the Apex Developer Guide.

Middleware Capabilities

The following table highlights the desirable properties of a middleware system that participates in this pattern.

Example

A utility company uses Salesforce and has a separate system that contains customer billing information. As a part of ordering process new billing accounts must be created in the billing system and Salesforce should reflect the Billing account number within the order activation process. They have an existing web service that allows the creation of a billing account and returns the billing account number as a response.

This requirement can be accomplished with the following approach.

1. Salesforce consumes the billing account service WSDL from an Apex proxy class.
2. Execute the Apex proxy class with the customer information passed to the external webservice from a custom apex controller or external service.
3. The custom controller then parses the return values from the Apex callout and stores that information on the salesforce object.

This example demonstrates that:

• The state of the customer is tracked with an account number stored on the Salesforce account object.
• Subsequent processing of the reply message by the caller

Remote Process Invocation — Fire and Forget

Context

You use Salesforce to track leads, manage your pipeline, create opportunities, and capture order details that convert leads to customers. However, as a part of the Order management processes, a billing account needs to be created in the billing system for the order.

When you implement this pattern, Salesforce calls the remote system to create the billing account, but doesn’t wait for the call’s successful completion. The remote system can optionally update Salesforce with the new billing account created in a separate transaction.

Problem

When an event occurs in Salesforce, how do you initiate a process in a remote system and pass the required information to that process without waiting for a response from the remote system?

Forces

Consider the following forces when applying solutions based on this pattern.

• Does the call to the remote system require Salesforce to wait for a response before continuing processing? Is the call to the remote system synchronous or asynchronous?
• If the call to the remote system is synchronous, does the response need to be processed by Salesforce as part of the same transaction as the call?
• Is the message size small?
• Is the integration based on the occurrence of a specific event, such as a button click in the Salesforce user interface, or DML-based events?
• Is guaranteed message delivery from Salesforce to the remote system a requirement?
• Is the remote system able to participate in a contract-first integration in which Salesforce specifies the contract? In some solution variants (for example, outbound messaging), Salesforce specifies a contract that the remote system endpoint implements.
• Does the endpoint or the Enterprise Service Bus (ESB) support long polling?
• Are declarative configuration methods preferred over custom Apex development? In this case, solutions such as platform events are preferred over Apex callouts.

Solution

The following table contains solutions to this integration problem.

Sketch

The following diagram illustrates a call from Salesforce to a remote system in which creating or updating operations on a record trigger the call.

Download Diagram

In this scenario:

1. A remote system subscribes to the platform event.
2. An update or insert occurs on a given set of records in Salesforce.
3. A Salesforce Flow triggers when a set of conditions is met.
4. This flow creates a platform event.
5. The remote listener receives the event message, and places the message on a local queue.
6. The queuing application forwards the message to the remote application for processing.

In the case where the remote system must perform operations against Salesforce, you can implement an optional callback operation.

Results

The application of the solutions related to this pattern allows for:

• User interface–initiated remote process invocations in which the result of the transaction can be displayed to the end user
• DML event-initiated remote process invocations in which the result of the transaction can be processed by the calling process

Calling Mechanisms

The calling mechanism depends on the solution chosen to implement this pattern.

Error Handling and Recovery

An error handling and recovery strategy must be considered as part of the overall solution. The best method depends on the solution you choose.

Idempotent Design Considerations

Platform events are only published to the bus once. There is no retry on the Salesforce side. It is up to the ESB to request that the events be replayed. In a replay, the platform event’s replay ID remains the same and the ESB can try duplicate messages based on the replay ID.

Idempotency is important for outbound messaging because it’s asynchronous and retries are initiated when no positive acknowledgment is received. Therefore, the remote service must be able to handle messages from Salesforce in an idempotent fashion.

Outbound messaging sends a unique ID per message and this ID remains the same for any retries. The remote system can track duplicate messages based on this unique ID. The unique record ID for each record being updated is also sent, and can be used to prevent duplicate record creation.

The idempotent design considerations in the Remote Process Invocation—Request and Reply pattern also apply to this pattern.

Security Considerations

Any call to a remote system must maintain the confidentiality, integrity, and availability of the request. Different security considerations apply, depending on the solution you choose.

See Security Considerations.

Sidebars

Timeliness

Timeliness is less of a factor with the fire-and-forget pattern. Control is handed back to the client either immediately or after positive acknowledgment of a successful hand-off to the remote system. With Salesforce outbound messaging, the acknowledgment must occur within 24 hours (this can be extended to seven days); otherwise, the message expires. For platform events, Salesforce sends the events to the event bus and doesn’t wait for a confirmation or acknowledgment from the subscriber. If the subscriber doesn’t pick up the message, the subscriber can request to replay the event using the event reply ID. High-volume event messages are stored for 72 hours (three days). To retrieve past event messages, use CometD clients to subscribe to a channel.

Data Volumes

Data volume considerations depend on which solution you choose. For the limits of each solution, see the Salesforce Limits Quick Reference.

Endpoint Capability and Standards Support

The capability and standard support for the endpoint depends on the solution that you choose.

State Management

When integrating systems, unique record identifiers are important for ongoing state tracking. For example, if a record is created in the remote system, you have two options.

• Salesforce stores the remote system’s primary or unique surrogate key for the remote record.
• The remote system stores the Salesforce unique record ID or some other unique surrogate key.

The following table lists considerations for state management in this pattern.

Complex Integration Scenarios

Each solution in this pattern has different considerations for complex integration scenarios such as transformation and process orchestration.

Governor Limits

Due to the multitenant nature of the Salesforce platform, there are limits to outbound callouts. Limits depend on the type of outbound call and the timing of the call.

For limits and allocations that apply to platform events, see the Platforms Events Developer Guide.

Reliable Messaging

Reliable messaging attempts to resolve the issue of guaranteeing the delivery of a message to a remote system in which the individual components are unreliable. The method of ensuring receipt of a message by the remote system depends on the solution you choose.

In case of Salesforce Change Data Capture types of change events are provided to handle special situations, such as capturing changes not caught in the Salesforce application servers, or handling high loads of changes.

In some cases, Salesforce sends Gap events instead of change events to inform subscribers about errors, or if it’s not possible to generate change events. A gap event contains information about the change in the header, such as the change type and record ID. It doesn’t include details about the change, such as record fields. To capture changes more efficiently, Overflow events are generated for single transactions that exceed a threshold. For more information please see here.

Middleware Capabilities

The following table highlights the desirable properties of a middleware system that participates in this pattern.

Solution Variant—Platform Events: Publishing Behavior and Transactions

When platform event messages are published immediately, event publishing doesn't respect transaction boundaries of the publishing process. Event messages can be published before the transaction completes or even if a transaction fails. This behavior can lead to issues when a subscriber expects to find data that the publishing transaction commits. The data might not be present when the subscriber receives the event message. To solve this issue, set the platform event publish behavior to Publish After Commit in the event definition. The publish behaviors you can set in a platform event definition are:

• Publish After Commit to have the event message published only after a transaction commits successfully. Select this option if subscribers rely on data that the publishing transaction commits. For example, a process publishes an event message and creates a task record. A second process that is subscribed to the event is fired and expects to find the task record. Another reason for choosing this behavior is when you don’t want the event message to be published if the transaction fails.
• Publish Immediately to have the event message published when the publish call executes. Select this option if you want the event message to be published regardless of whether the transaction succeeds. Also choose this option if the publisher and subscribers are independent, and subscribers don’t rely on data committed by the publisher. For example, the immediate publishing behavior is suitable for an event used for logging purposes. With this option, a subscriber might receive the event message before data is committed by the publisher transaction.

Example

A telecommunications company wants to use Salesforce as a front end for creating accounts using the lead-to-opportunity process. An order is created in Salesforce when the opportunity is closed and won, but the back-end ERP system is the data master. The order must be saved to the Salesforce opportunity record, and the opportunity status changed to indicate that the order was created.

The following constraints apply.

• Only declarative development can be implemented.
• You don’t require immediate notification of the order number after the opportunity converts to an order.
• The organization has an ESB that supports the CometD protocol and is able to subscribe to platform events.

This example is best implemented using Salesforce platform events, but does require that the ESB subscribes to the platform event.

On the Salesforce side:

• Create a Flow to initiate the platform event (for example, when the opportunity status changes to Close-Won).
• Create a new platform event which publishes the opportunity details.

On the remote system side:

• The ESB subscribes to the Salesforce platform event using the CometD protocol.
• The ESB receives one or more notifications indicating that the opportunity is to be converted to an order.
• The ESB forwards the message to the back-end ERP system so that the order can be created.
• After the order is created in the ERP system, a separate thread calls back to Salesforce using the SessionId as the authentication token. The callback updates the opportunity with the order number and status. You can do this callback using documented pattern solutions, such as Salesforce platform events, Salesforce SOAP API, REST API, or an Apex web service.

This example demonstrates the following.

• Implementation of a remote process invoked asynchronously
• End-to-end guaranteed delivery
• Subsequent callback to Salesforce to update the state of the record

Batch Data Synchronization

Context

You’re moving your CRM implementation to Salesforce and want to:

• Extract and transform accounts, contacts, and opportunities from the current CRM system and load the data into Salesforce (initial data import).
• Extract, transform, and load customer billing data into Salesforce from a remote system on a weekly basis (ongoing).
• Extract customer activity information from Salesforce and import it into an on-premises data warehouse on a weekly basis (ongoing).

Problem

How do you import data into Salesforce and export data out of Salesforce, taking into consideration that these imports and exports can interfere with end-user operations during business hours, and involve large amounts of data?

Forces

There are various forces to consider when applying solutions based on this pattern:

• Should the data be stored in Salesforce? If not, there are other integration options an architect can and should consider (mashups, for example).
• If the data should be stored in Salesforce, should the data be refreshed in response to an event in the remote system?
• Should the data be refreshed on a scheduled basis?
• Does the data support primary business processes?
• Are there analytics (reporting) requirements that are impacted by the availability of this data in Salesforce?

Solution

The following table contains various solutions to this integration problem.

Sketch

The following diagram illustrates the sequence of events in this pattern, where the Salesforce is the data master.

Download Diagram

The following diagram illustrates the subsequent synchronization of events in near real time, where Salesforce is the data master.

Results

You can integrate data that’s sourced externally with Salesforce under the following scenarios:

• External system is the data master—Salesforce is a consumer of data provided by a single source system or multiple systems. In this scenario, it’s common to have a data warehouse or data mart that aggregates the data before the data is imported into Salesforce.
• Salesforce is the data master—Salesforce is the system of record for certain entities and Salesforce Change Data Capture client applications can be informed of changes to Salesforce data.

In a typical Salesforce integration scenario, the implementation team does one of the following:

• Implement change data capture on the source data set.
• Implement a set of supporting database structures, known as control tables, in an intermediate, on-premises database.

The ETL tool is then used to create programs that will:

1. Read a control table to determine the last run time of the job and extract any other control values needed.
2. Use the above control values as filters and query the source data set.
3. Apply predefined processing rules, including validation, enrichment, and so on.
4. Use available connectors/transformation capabilities of the ETL tool to create the destination data set.
5. Write the data set to Salesforce objects.
6. If processing is successful, update the control values in the control table.
7. If processing fails, update the control tables with values that enable a restart and exit.

Note: We recommend that you create the control tables and associated data structures in an environment that the ETL tool has access to even if access to Salesforce isn’t available. This provides adequate levels of resilience. Salesforce should be treated as a spoke in this process and the ETL infrastructure is the hub.

For an ETL tool to gain maximum benefit from data synchronization capabilities, consider the following:

• Chain and sequence the ETL jobs to provide a cohesive process.
• Use primary keys from both systems to match incoming data.
• Use specific API methods to extract only updated data.
• If importing child records in a master-detail or lookup relationship, group the imported data using its parent key at the source to avoid locking. For example, if you’re importing contact data, be sure to group the contact data by the parent account key so the maximum number of contacts for a single account can be loaded in one API call. Failure to group the imported data usually results in the first contact record being loaded and subsequent contact records for that account to fail in the context of the API call.
• Any post-import processing, such as triggers, should only process data selectively.
• If your scenario involves large data volumes, follow the best practices from this Trailhead module Best Practices for Deployments with Large Data Volumes
.

Error Handling and Recovery

An error handling and recovery strategy must be considered as part of the overall solution. The best method depends on the solution you choose.

Security Considerations

Any call to a remote system must maintain the confidentiality, integrity, and availability of the request. Different security considerations apply, depending on the solution you choose.

• A Lightning Platform license with at least “API Only” user permissions is required to allow authenticated API access to the Salesforce API.
• We recommend that standard encryption be used to keep password access secure.
• Use the HTTPS protocol when making calls to the Salesforce APIs. You can also proxy traffic to the Salesforce APIs through an on-premises security solution, if necessary.

See Security Considerations.

Sidebars

Timeliness

Timeliness isn’t of significant importance in this pattern. However, care must be taken to design the interfaces so that all of the batch processes complete in a designated batch window.

As with all batch-oriented operations, we strongly recommend that you take care to insulate the source and target systems during batch processing windows. Loading batches during business hours might result in some contention, resulting in either a user's update failing, or more significantly, a batch load (or partial batch load) failing.

For organizations that have global operations, it might not be feasible to run all batch processes at the same time because the system might continually be in use. Data segmentation techniques using record types and other filtering criteria can be used to avoid data contention in these cases.

State Management

You can implement state management by using surrogate keys between the two systems. If you need any type of transaction management across Salesforce entities, we recommend that you use the Remote Call-In pattern using Apex.

Standard optimistic record locking occurs on the platform, and any updates made using the API require the user, who is editing the record, to refresh the record and initiate their transaction. In the context of the Salesforce API, optimistic locking refers to a process where:

• Salesforce doesn’t maintain the state of a record being edited by a specific user.
• Upon reading, it records the time when the data was extracted.
• If the user updates the record and saves it, Salesforce checks to see if another user has updated the record in the interim.
• If the record has been updated, the system notifies the user that an update was made and the user should retrieve the latest version of the record before proceeding with their updates.

Middleware Capabilities

The most effective external technologies used to implement this pattern are traditional ETL tools. It’s important that the middleware tools chosen support the Salesforce Bulk API.

It’s helpful, but not critical, that the middleware tools support the getUpdated() function. This function provides the closest implementation to standard change data capture capability on the Salesforce platform.

The following table highlights the desirable properties of a middleware system that participates in this pattern.

Example

A utility company uses a mainframe-based batch process that assigns prospects to individual sales reps and teams. This information needs to be imported into Salesforce on a nightly basis.

The customer has decided to implement change data capture on the source tables using a commercially available ETL tool. The solution works as follows:

• A cron-like scheduler executes a batch job that assigns prospects to users and teams.
• After the batch job runs and updates the data, the ETL tool recognizes these changes using change data capture. The ETL tool collates the changes from the data store.
• The ETL connector uses the Salesforce REST API to load the changes into Salesforce.

Remote Call-In

Context

You use Salesforce to track leads, manage your pipeline, create opportunities, and capture order details that convert leads to customers. The Salesforce system creates the orders internally and then pushes that data to the external billing system for provisioning and activation. Billing Orders are managed by an external (remote) system. That remote system needs to update the order status in Salesforce when the order or billing entity on the billing system status changes.

Problem

How does a remote system connect and authenticate with Salesforce to notify Salesforce about external events, create records, and update existing records?

Forces

There are various forces to consider when applying solutions based on this pattern:

• Is the purpose of the remote call to Salesforce to notify Salesforce of an event that occurred externally using an event-driven architecture? Or is the purpose to perform CRUD operations on specific records? If you use an event-driven architecture, the event producer (the remote process) is decoupled from the Salesforce event consumer.
• Does the call to Salesforce require the remote process to wait for a response before continuing processing? Remote calls to Salesforce are always synchronous request-reply, although the remote process can discard the response if it isn’t needed to simulate an asynchronous call.
• Does each transaction operate against a single Salesforce object or multiple, related objects?
• What is the format of the message (for example, SOAP or REST, or both over HTTP)?
• Is the message size relatively small or large?
• If the remote system is SOAP-capable, is the remote system able to participate in a contract-first approach, where Salesforce dictates the contract? This is required where our SOAP API is used, for which a predefined WSDL is supplied.
• Is transaction processing required?
• What is the extent to which you’re tolerant of customization in Salesforce?

Solution

This table contains various solutions to this integration problem.

Sketch

The following diagrams illustrate the sequence of events when you implement this pattern using either REST API for notifications from external events or SOAP API to query a Salesforce object. The sequence of events is the same when using the REST API.

Remote System Querying Salesforce Via SOAP API

Remote System Notifying Salesforce with Events Via REST API

Results

In an event-driven architecture, the remote system calls into Salesforce using SOAP API, REST API, or Bulk API 2.0 to publish an event to the Salesforce event bus. Publishing an event notifies all subscribers. Event subscribers can be on the Salesforce Platform such as Flows, or Lightning Components, Apex triggers. Event subscribers can also be external to the Salesforce Platform such as CometD subscribers.

When working directly with Salesforce objects, the solutions related to this pattern allow for:

• The remote system calls the Salesforce APIs to query the database and execute single-object operations (create, update, delete, and so on).
• The remote system to call the Salesforce REST API composite resources to perform a series of object operations.
• Remote system to call custom-built Salesforce APIs (services) that can support multi-object transactional operations and custom pre/post processing logic.

Calling Mechanisms

The calling mechanism depends on the solution chosen to implement this pattern.

Error Handling and Recovery

An error handling and recovery strategy must be considered as part of the overall solution.

• Error handling — All the remote call-in methods, standard or custom APIs, require the remote system to handle any subsequent errors, such as timeouts and the management of retries. Middleware can be used to provide the logic for error handling and recovery.
• Recovery — A custom retry mechanism needs to be created if quality-of-service requirements dictate it. In this case, it’s important to ensure idempotent design characteristics. Platform event enables subscribers to use the replay ID to fetch messages within a certain time period after those messages were published.

Idempotent Design Considerations

Idempotent capabilities guarantee that repeated invocations are safe and won’t have a negative effect. If idempotency isn’t implemented, then repeated invocations of the same message can have different results, potentially resulting in data integrity issues, for example, creation of duplicate records, duplicate processing of transactions, and so on.

The remote system must manage multiple (duplicate) calls, in the case of errors or timeouts, to avoid duplicate inserts and redundant updates (especially if downstream triggers and workflow rules fire). While it’s possible to manage some of these situations within Salesforce (particularly in the case of custom SOAP and REST services), we recommend that the remote system (or middleware) manages error handling and idempotent design.

Security Considerations

Different security considerations apply, depending on the pattern solution chosen. In all cases, the platform uses the logged-in user’s access rights (for example, profile settings, sharing rules, permission sets, and so on). Additionally, profile IP restrictions can be used to restrict access to the API for a specific IP address range.

See Security Considerations.

Sidebars

Timeliness

SOAP API and Apex web service API are synchronous. The following timeouts apply:

• Session timeout — The session times out if there’s no activity based on the Salesforce org’s session timeout setting.
• Query timeout — Each SOQL query has an individual timeout limit of 120 seconds.

Data Volumes

Data volume considerations depend on which solution and communication type you choose.

Endpoint Capability and Standards Support

The capability and standard support for the endpoint depends on the solution that you choose.

State Management

When integrating systems, keys are important for on-going state tracking, for example, if a record gets created in the remote system, to support ongoing updates to that record. There are two options:

• Salesforce stores the remote system’s primary or unique surrogate key for the remote record.
• The remote system stores the Salesforce unique record ID or some other unique surrogate key. There are specific considerations for handling integration keys in this synchronous pattern.

Complex Integration Scenarios

Each solution in this pattern has different considerations when handling complex integration scenarios such as transformation and process orchestration.

Governor Limits

Due to the multitenant nature of the Salesforce platform, there are limits when using the APIs.

Reliable Messaging

Reliable messaging attempts to resolve the issue of guaranteeing the delivery of a message to a remote system where the individual components themselves might be unreliable. The Salesforce SOAP API and REST API are synchronous and don’t provide explicit support for any reliable messaging protocols, per se (for example, WS-ReliableMessaging).

We recommend that the remote system implement a reliable messaging system to ensure that error and timeout scenarios are successfully managed. Publishing of platform events from external systems relies on Salesforce APIs, so the same considerations for SOAP API and REST API apply.

Middleware Capabilities

This table highlights the desirable properties of a middleware system that participates in this pattern:

Example

A printing supplies and services company uses Salesforce as a front end to create and manage printer supplies and orders. Salesforce asset records representing printers are periodically updated with printing usage statistics (ink status, paper level) from the on-premises Printer Management System (PMS), which regularly monitors printers on client sites. Upon reaching a set threshold value (for example, low ink status or low/empty paper level of less than 30%), multiple apps/processes (variable) interested in the event are notified, email or Chatter alerts are sent, and an Order record is created. The PMS stores the Salesforce ID (Salesforce is the asset record master).

The following constraints apply:

• The PMS can participate in a contract-first integration, where Salesforce provides the contract and the PMS acts as a client (consumer) of the Salesforce service (defined via the Enterprise or Partner WSDL).
• There should be no custom development in Salesforce.

This example is best implemented using the Salesforce SOAP API or REST API to publish events, and declarative automation (Flow) in Salesforce. The primary reason to use platform events is the variable and non-finite number of subscribers; however, for simple updates to a finite list of records such as orders, then use SOAP or REST API to update the records.

In Salesforce:

• Define a platform event in Salesforce to contain the notification data coming from the PMS.
• Create a Flow that’s triggered by the printer event notification. The process updates the printer asset and creates an order (using an auto-launched Flow).
• Download the Enterprise or Partner WSDL and provide it to the remote system.

In the remote system:

• Create a client stub from the Enterprise or Partner WSDL.
• Authenticate to Salesforce (via OAuth web server or bearer token flow) using the integration user’s credentials.
• Upon printer status event, the PMS calls the API to create the printer status platform event (with printer usage statistics). The Salesforce event bus notifies Flow subscriber and all other subscribers.

When you use platform events, the event bus lets you replay events for 72 hours for high-volume platform events. Publishing those events using a middleware solution can help incorporate error handling on the publishing side. However, you can implement error handling on the subscribing side if you need higher reliability.

This example demonstrates the following:

• Implementation of a Salesforce synchronous API client (consumer).
• A callback to Salesforce to publish a platform event (aligned with previously covered request/reply platform event patterns).

UI Update Based on Data Changes

Context

You use Salesforce to manage customer cases. A customer service rep is on the phone with a customer working on a case. The customer makes a payment, and the customer service rep needs to see a real-time update within the Salesforce application from the payment processing application, indicating that the customer has successfully paid the order’s outstanding amount.

Problem

When an event occurs in Salesforce, how can the user be notified in the Salesforce user interface without having to refresh their screen and potentially losing work?

Forces

There are various forces to consider when applying solutions based on this pattern:

• Does the data being acted on need to be stored in Salesforce?
• Can a custom user interface layer be built for viewing this data?
• Will the user have access for invoking the custom user interface?

Solution

The recommended solution to this integration problem is to use platform events which ensures near real time eventing in Salesforce. Platform Events provide a structured, flexible payload independent of any Salesforce object. It also ensures durable topics with a replay window of 72 hours. This ensures that events are available even if an offline consumer becomes available later. This solution is composed of the following components :

• Apex Trigger or Flows with a logic to publish a platform event on a topic
• A topic that allows you to publish an event from Apex Trigger or Flow
• A JavaScript-based implementation of the Bayeux protocol (currently CometD ) that can be used by the user interface
• A Lightning component
• A JavaScript library included as a static resource

Sketch

The following diagram illustrates how Streaming API can be implemented to stream notifications to the Salesforce user interface. These notifications are triggered by record changes in Salesforce.

UI Update in Salesforce Triggered by a Data Change

Results

Benefits

The application of the solution related to this pattern has the following benefits:

• Eliminates the need for writing custom polling mechanisms
• Eliminates the need for a user-initiated feedback loop

Unsupported Requirements

The solution has the following limitations:

• Delivery of notifications isn’t guaranteed.
• Order of notifications isn’t guaranteed.
• Notifications aren’t generated from record changes made by Bulk API.

Security Considerations

Standard Salesforce org-level security is adhered to. It’s recommended that you use the HTTPS protocol to connect to the Streaming API. See Security Considerations

Sidebars

The optimal solution involves creating a custom user interface in Salesforce. It’s imperative that you account for an appropriate user interface container that can be used for rendering the custom user interface. Supported browsers are listed in the Streaming API Developer Guide

Example

A telecommunications company uses Salesforce to manage customer cases. The customer service managers want to be notified when a case is successfully closed by one of their customer service reps.

Implementing the solution prescribed by this pattern, the customer should:

• Create an Apex Trigger PushTopic that sends a platform event when a case is saved with a Status of “Closed” and Resolution of “Successful.”
• Create a custom user interface available to customer service managers. This user interface subscribes to the event bus to consume platform events.
• Implement logic in the custom user interface that shows alerts generated by that manager’s customer service reps.

Data Virtualization

Context

You use Salesforce to track leads, manage your pipeline, create opportunities, and capture order details that convert leads to customers. However, Salesforce isn’t the system that contains or processes orders. Orders are managed by an external (remote) system. But sales reps want to view and update real-time order information in Salesforce without having to learn or use the external system.

Problem

In Salesforce, how do you view, search, and modify data that’s stored outside of Salesforce, without moving the data from the external system into Salesforce?

Forces

There are various forces to consider when applying solutions based on this pattern:

• Do you want to build a declarative/point-and-click outbound integration or UI mashup in Salesforce?
• Do you have a large amount of data that you don’t want to copy into your Salesforce org?
• Do you need to access small amounts of remote system data at any one time?
• Do you need real-time access to the latest data?
• Do you store your data in the cloud or in a back-office system, but want to display or process that data in your Salesforce org?
• Do you have data residency concerns for storing certain types of data in Salesforce?

Solution

The following table contains various solutions to this integration problem.

Sketch

The following diagram illustrates how you can use Salesforce Connect to pull data from an external system using an OData adapter.

In this scenario:

1. The browser performs an AJAX call that in turn performs an action on the corresponding external object adapter.
2. The adapter translates the action into an OData request and makes an HTTP GET request to the remote system via the Integration and Services layers.
3. The remote system returns a JSON response to Salesforce via the Integration and Services layers.
4. The response is translated from OData into an external object and presented back to the browser.

Results

The application of the solutions related to this pattern allows for user-interface initiated invocations in which the result of the transaction can be displayed to the end user.

Calling Mechanisms

The calling mechanism depends on the solution chosen to implement this pattern.

Error Handling

It’s important to include error handling as part of the overall solution. When an error occurs (exceptions or error codes are returned to the caller), the caller manages the error handling. The Salesforce Connect Validator is a free tool to run some common queries and notice error types and failure causes.

Benefits

Some of the benefits of using a Salesforce Connect solution are:

• This solution doesn't consume data storage in Salesforce.
• Users don't have to worry about regularly synchronizing data between the external system and Salesforce.
• A declarative setup that can be achieved quickly with OData, or a cross-org adapter, or using minimal code with a custom Apex adapter.
• Users can access external data with much of the same functionality as custom objects in the form of external objects.
• Ability to do a federated search in the connected external system using global search.
• Ability to run reports that access external data from cloud and on-premises sources. Refer to reporting considerations below.

Salesforce Connect Considerations

The Salesforce Connect solution has the following considerations:

• External objects behave like custom objects, but some features aren’t available for external objects. For more information, see Salesforce Compatibility Considerations for Salesforce Connect.
• External objects can impact report performance. For more information, see Report Considerations for Salesforce Connect.
• For additional considerations for using Salesforce Connect adapters see Considerations for Salesforce Connect—All Adapters.
• If you’re considering using a cross-org adapter, see Considerations for Salesforce Connect—Cross-Org Adapter.
• If you’re considering using a OData adapter, see Considerations for Salesforce Connect—OData 2.0 and 4.0 Adapters.
• If you’re considering using a custom Apex adapter, see Considerations for Salesforce Connect—Custom Adapter.

Security Considerations

Solutions for this pattern should adhere to standard Salesforce org-level security. It’s recommended you use the HTTPS protocol to connect to any remote system. For more details, see Security Considerations

If you’re using an OData connector, make sure that you understand the special behaviors, limitations, and recommendations for Cross-Site Request Forgery (CSRF) on OData external data sources. For more information, see CSRF Considerations for Salesforce Connect — OData 2.0 and 4.0 Adapters.

Sidebars

Timeliness

Timeliness is of significant importance in this pattern. Keep the following points in mind:

• The request is typically invoked from the user interface, so the process must not keep the user waiting.
• Depending on the availability of and the connection to the external system, it can take a long time to retrieve external data. Salesforce has a configurable 120-second maximum timeout value to wait for a response from the external system.
• Completion of the remote process should execute in a timely manner and complete within the Salesforce timeout limit and within user expectations.

Data Volumes

This pattern is used primarily for small volume, real-time activities, due to the small timeout values and maximum size of the request or response for the Apex call solution. Don’t use this pattern in batch processing activities in which the data payload is contained in the message.

Endpoint Capability and Standards Support

The capability and standards support for the endpoint depends on the solution that you choose.

State Management

When integrating systems, keys are important for on-going state tracking. For example, if a record gets created in the remote system, typically the record needs some sort of identifying key to support ongoing updates. There are two options for storing keys.

• Salesforce stores the primary or unique surrogate key for the remote record.
• The remote system stores the Salesforce unique record ID or some other unique surrogate key. There are specific considerations for handling integration keys in this synchronous pattern.

Complex Integrations

In certain cases, the solution prescribed by this pattern can require the implementation of a complex integration scenario. These scenarios are often solved using middleware.

• Aggregation of calls and their results across calls to multiple systems
• Transformation of both inbound and outbound messages
• Maintaining transactional integrity across calls to multiple systems
• Other process orchestration activities between Salesforce and the external system

Governing Limits

Different limits apply for different adapters. For more details, see General Limits for Salesforce Connect.

Middleware Capabilities

The following table highlights the desirable properties of a middleware system that participates in this pattern.

External Object Relationships

External objects support standard lookup relationships, which use the 18-character Salesforce record ID to associate related records. However, data that’s stored outside your Salesforce org often doesn’t contain those record IDs. Therefore, two special types of lookup relationships are available for external objects: external lookups and indirect lookups.

This table summarizes the types of relationships that are available to external objects.

High Data Volume Considerations for Salesforce Connect—OData 2.0 and 4.0 Adapters

If your org hits rate limits when accessing external objects, consider selecting the High Data Volume option on the associated external data sources. Doing so bypasses most rate limits, but some special behaviors and limitations apply. For more information, see High Data Volume Considerations for Salesforce Connect

Client-Driven and Server-Driven Paging for Salesforce Connect—OData 2.0 and 4.0 Adapters

It's common for Salesforce Connect queries of external data to have a large result set that's broken into smaller batches or pages. You decide whether to have the paging behavior controlled by the external system (server-driven) or by the OData 2.0 or 4.0 adapter for Salesforce Connect (client-driven). The Server Driven Pagination field on the external data source specifies whether to use client-driven or server-driven paging. If you enable server-driven paging on an external data source, Salesforce ignores the requested page sizes, including the default queryMore() batch size of 500 rows. The pages returned by the external system determine the batches, but each page can’t exceed 2,000 rows. However, the limits for the OData adapters for Salesforce Connect still apply.

Example

A manufacturing company uses Salesforce to manage customer cases. The customer service agents want to access the real-time order information from the back-office ERP system to get a 360 view of the customer without having to learn and manually run reports in ERP.

Implementing the solution prescribed by this pattern, you should:

• Configure your external data source with an OData endpoint. Your remote application may include native support for OData. For other applications, major integration vendors such as Dell Boomi, Informatica, Jitterbit, MuleSoft, and Progress Software have partnered with Salesforce on Salesforce Connect to build adapters.
• Point Salesforce Connect at the OData endpoint, either directly, or through a middleware solution.
• Sync your external database tables with external objects in Salesforce. When a user accesses a page with data from these external objects, Salesforce Connect makes real-time callouts to your back-end applications.

Appendix A: Resources

Developer Documentation

• Salesforce Help: Give Integration Users API Only Access

• SOAP API Developer Guide

• REST API Developer Guide

• Streaming API Developer Guide

• Bulk API 2.0 and Bulk API Developer Guide

• Apex Developer Guide

• SOQL and SOSL Reference

• Salesforce Developer Limits and Allocations Quick Reference

• Platform Events Developer Guide

• Apex Developer Guide: Salesforce Connect

Trailhead

• Large Data Volumes

• Platform Events Basics

• Change Data Capture Basics

• Quick Start: Salesforce Connect

Appendix B: Security Considerations

To be effective members of the enterprise portfolio, all applications must be created and integrated with relevant security mechanisms. Modern IT strategies employ a combination of on-premises and cloud-based services.

While integrating cloud-to-cloud services typically focuses on web services and associated authorization, connecting on-premises and cloud services often introduces increased complexity. This section describes security tools, techniques, and Salesforce-specific considerations.

Reverse Proxy Server

A reverse proxy is a server that sits in front of web servers and forwards client (e.g. web browser) requests to those web servers. Reverse proxies are typically implemented to help increase security, performance, and reliability.

It’s a type of proxy server that retrieves resources on behalf of a client from one or more servers. These resources are then returned to the client as though they originated from the proxy server itself. Unlike a forward proxy, which is an intermediary for its associated clients to contact any server, a reverse proxy is an intermediary for its associated servers to be contacted by any client.

In Salesforce implementations, such a service is typically provided via an external gateway product. For example, open source options such as Apache HTTP, lighttpd, and nginix can be used. Commercial products include IBM WebSeal and Computer Associates SiteMinder. These products can be easily configured to proxy and manage all outbound Salesforce requests on behalf of the internal requester.

Encryption

Some enterprises require selected transactions or data fields to be encrypted between a combination of on-premises and cloud-based applications. If your organization must adhere to additional compliance requirements, you can implement alternatives, including:

• On-premises commercial encryption gateway services, including Salesforce’s own, CipherCloud, IBM DataPower, Computer Associates. For each solution, the encryption engine or gateway is invoked at the transaction boundary by sending and receiving an encrypted payload or when encrypting or decrypting specific data fields before the HTTP(S) request executes.

• Cloud-based options, such as Salesforce Shield Platform Encryption. Shield Platform Encryption gives your data a whole new layer of security while preserving critical platform functionality. The data you select is encrypted at rest using an advanced key derivation system. You can protect your data more securely than ever before. Refer to the Salesforce online help for more information.

Specialized WS-* Protocol Support

To address the requirements of security protocols (such as WS-*), we recommend these alternatives.

• Security/XML gateway—Inject WS-Security credentials (IBM WebSeal or Datapower, Layer7, TIBCO, and so on) into the transaction stream itself. This approach requires no changes to application-level web services or web service invocations from Salesforce. You can also reuse this approach across the Salesforce installation. However, it requires additional design, configuration, testing, and maintenance to manage the appropriate WS-Security injection into the existing security gateway approach.

• Transport-level encryption—Encrypt the communication channel using two-way SSL and IP restrictions. While this approach doesn’t directly implement the WS-* protocol by itself, it secures the communication channel between the on-premises applications and Salesforce without passing a username and password. It also doesn’t require changes to Salesforce-generated classes. However, some on-premises web services modifications might be required (at either the application itself or at the middleware/ESB layer).

• Salesforce custom development—Add WS-Security headers to the outbound SOAP request via the WSDL2Apex utility. This generates a Java-like Apex class from the WSDL file used to invoke the internal service. While this approach requires no changes to back-end web services or additional components in the DMZ, it does require:

  • an increased build and test effort
  • a relatively complex and manual process to hand-code the WS-Security attributes (including XML serialization within the Apex code)
  • a higher long-term maintenance effort

Note: The last option isn’t recommended due to its complexity and the risk that such integrations need periodic reviews based on regular updates to Salesforce.

Other Security Considerations

• Named Credentials: Intended to secure and simplify authenticated API callouts to external systems, a named credential specifies the URL of a callout endpoint and its required authentication parameters in one definition. To streamline your Apex code and simplify the setup of authenticated callouts, specify a named credential as the callout endpoint. For more details see here.

• OAUTH 2.0: OAuth 2.0 is an industry-standard authorization framework that allows a user to grant limited access to their protected resources to a third-party application without sharing their credentials (like passwords). Instead of a direct password exchange, the user approves an access token request, which the application then uses to access the user's data from a resource server. This process separates the client application from the user's identity and password, enhancing security by providing a temporary, scope-specific "key" for accessing resources. For more information see here.

• Private Connect: When you integrate your Salesforce org with applications hosted on third-party cloud services, it’s essential to be able to send and receive HTTP/s traffic securely. With Private Connect, increase security on your Amazon Web Services (AWS) integrations by setting up a fully managed network connection between your Salesforce org and your AWS Virtual Private Cloud (VPC). Then, route your cross-cloud traffic through the connection instead of over the public internet to reduce exposure to outsider security threats. Private Connect is available in partnership with AWS via a feature called AWS PrivateLink. Private Connect is bi-directional: you can initiate both inbound and outbound traffic. With inbound connections, you can send traffic into Salesforce using the standard APIs. And with outbound connections, you can send traffic out of Salesforce via features like Apex callouts, External Services, and External Objects. For more information on VPC please here.

Appendix C: Event Driven Architecture

The Salesforce Event Bus with Platform Events, Change Data Capture (CDC), and Pub /Sub API enables enterprises to create event driven style architectures (EDAs).

An EDA decouples event message consumers (subscribers) from event message producers (publishers), allowing for greater scale and flexibility. Specific patterns are covered in this document as part of the existing pattern structure that compare and contrast related alternatives or options. However, getting a holistic understanding of the event driven architecture, and how patterns interplay, can help you select the right pattern for your needs.

About the Authors

Khalid Mohammed is a distinguished Architect leader within Salesforce's Professional Services. Since joining Salesforce in 2015, he has leveraged his extensive expertise in Enterprise Application and Architecture, honed through numerous customer transformations before his tenure at Salesforce. Khalid heads the Delivery Architecture Board and is also acknowledged as a Certified Technical Architect leader.

Gulal Kumar is a Software Engineering Architect at Salesforce, with a focus on data and integration architecture. With over 20 years of experience in integration and APIs, modernization programs, security, and AIML initiatives, he brings a wealth of expertise. Gulal has been committed to advancing business transformation initiatives, enhancing security and resiliency, promoting architecture excellence, and leading AIML initiatives across various domains.

Sushant Verma is a Technical Architect at Salesforce, working within Professional Services. He assists our customers with critical technology transformations.