Salesforce provides a Streaming API that allows for an event-driven approach for handling high-volume data synchronization. Rather than having systems synchronously request data from Salesforce, data can be pushed out from Salesforce to other systems.
Let's look at an example implmentation:
Account record is inserted into Salesforce. Upon insertion, a CDC event is created and subscribers to that event can react to it. In this case, an external subscriber gets the latest copy of the event, and inserts that event into an external database.
With this approach, other systems can query for the account data against the external database without needing to directly access Salesforce.
The Streaming API supports various event types: PushTopic, Change Data Capture (CDC), platform, and generic events which all provide slightly different capabilities for streaming data. The event type you choose will be dependent on your specific use case, but the overall architectural pattern is similar between event types.
For example, Changed Data Capture (CDC) events provide a way to notify external systems of data changes in Salesforce as changes occur. Because CDC events are fundamentally asynchronous, there’s no guarantee that any given change will be immediately available on the external system. However, by supplying data to external systems and using those systems to handle data requests, you can reduce the need to read large volumes of data directly from Salesforce.
The increased ability to scale application performance and data synchronization with evented architectures does come with some complexity. Message delivery isn’t always guaranteed, so in rare cases, a change in Salesforce could be lost, leading to synchronization issues with external systems.
It’s important to have a process to reconcile any such data synchronization issues that may develop over time. Even with a reconciliation process in place, streaming events cannot provide the same guarantees as a synchronous transactional approach. Nevertheless, if your business needs can be fulfilled given these limitations, platform events offer tremendous gains in scalability.
Steve Stearns is a Regional Success Architect Director at Salesforce on the Scale and Availability team. Scale and Availability architects apply the learnings from our large customer engagements to help influence product scale evolution and innovation.