Salesforce operates resilient infrastructure across multiple regions with automated failover and infrastructure-level resilience. The platform handles data center redundancy, network availability, and infrastructure patching, with real-time platform availability status visible at trust.salesforce.com.
You design the reliability of everything that runs on this infrastructure: the data models that scale within governor limits, the transactions that anticipate and recover from failures, the monitoring that detects when your solution deviates from availability targets, and the disaster recovery procedures that restore business operations when failures occur.
The Salesforce SLA covers the platform, but you own reliability for everything above the infrastructure layer. You're responsible for defining and meeting your own service level objectives (SLOs) — the reliability targets that your business requires. Application-layer reliability remains your responsibility regardless of the platform SLA tier. The same platform that guarantees availability also limits it. Governor limits cap each tenant’s resource consumption so that no single tenant can degrade the platform for others. Your solution must therefore scale gracefully within those limits, rather than simply requesting more capacity.
Unreliable solutions can create cascading business impacts. Revenue flow slows when commerce platforms are unavailable. Productivity drops when internal tools fail mid-workflow. Trust erodes when data is corrupted or records are lost. These problems compound over time as workarounds accumulate and technical debt grows.
Reliability isn’t about preventing all failures. Failures happen in distributed systems. Reliability is about designing systems that anticipate failure, help contain blast radius, and restore service automatically. Reliability requirements vary by business impact. For example, a customer-facing Experience Cloud portal requiring 99.9% availability involves fundamentally different architectural choices than an internal batch reporting process that tolerates occasional delays.
Reliability and Operational Excellence are deeply interconnected. Both address monitoring, incident response, and system availability. This overlap is intentional, not accidental. The distinction lies in design time vs. run time.
Reliability is what you architect into a system before it runs, including:
- Previously mentioned data models that scale within governor limits
- Transactions that anticipate and recover from failures
- Redundancy and circuit breakers that contain blast radius
- Recovery targets — recovery time objective (RTO) and recovery point objective (RPO) — that determine how your system behaves when things go wrong
Reliability is baked in; it’s a structural property of your solution.
Operational Excellence is how you operate, improve, and sustain the system once it’s running, including:
- Deployment practices that reduce change risk
- Runbooks and escalation paths that guide your team during incidents
- Observability pipelines that surface signals
- Feedback loops that improve the system over time
Operational Excellence is practiced; it’s the human and process discipline surrounding your solution.
The shared ground between the two pillars is the monitoring and observability layer. Monitoring is designed as a reliability concern. You should build observable systems. It is practiced as an Operational Excellence concern. Your team acts on what those systems tell you. Reliability covers architectural patterns that you build in, like alerting thresholds and health dashboards. Operational Excellence covers how your team responds to signals, such as runbooks and on-call response.
Consider this distinction: Reliability addresses "will this system survive?" while Operational Excellence addresses "can your team operate it?" A perfectly reliable system operated by a team with no runbooks, inconsistent deployments, or no feedback loops can still fail in practice. An operationally excellent team managing a brittle, poorly designed system will be overwhelmed by incidents they cannot prevent. Both pillars are necessary, and neither substitutes for the other.
Reliability does not operate in isolation. As described in the previous sections, Operational Excellence is its closest partner. The two pillars share the observability layer, with Reliability defining what you build into a system and Operational Excellence defining how your team operates. Trust also requires infrastructure that resists attacks and maintains data integrity: a system is not reliable if it can be compromised. Resource Optimization prevents governor limit exhaustion, ensuring that the platform remains reliable at scale. Cost Optimization balances reliability investments against the business value they deliver. Availability targets justify the architectural complexity required to achieve them. No single pillar produces a well-architected solution in isolation. Reliability provides the structural foundation that the other pillars depend on and reinforce.
Use these principles to guide your architectural decisions for reliability on the platform.
- Distribute workload through bulkification. Process multiple records in single transactions rather than relying on sequential per-record operations. Bulkification shares governor limits across a batch processed in a single transaction, respecting those limits while maximizing throughput. Collections-based Apex processing, batch jobs with configurable scope, and platform events consumed in bulk all embody this principle. Well-bulkified solutions process 200 records with the same number of SOQL and DML statements as one record in sequential processing. Distributed workload patterns provide resilience because no single record failure impacts processing of the entire batch.
- Assume everything fails. Governor limits, platform maintenance windows, and integration dependencies create failure modes related to Salesforce multitenant architecture. Design for these platform-specific failures from the start. SOQL queries exceed row limits under data skew. Apex CPU time can expire during complex calculations. Callout timeouts can occur as external services slow. DML row locks fail when concurrent transactions collide. Storage limits can fail when file uploads spike unexpectedly. Architects who plan for these failures build solutions that are reliable without manual intervention. These reliable systems detect approaching governor limits, contain blast radius through error handling, and recover automatically through retry frameworks and Platform Events.
- Build self-recovering systems. Design solutions that detect failures and recover automatically without human intervention. Platform Events enable custom retry patterns. Delivery to a subscriber can be retried with EventBus.RetryableException, though replaying the original transaction requires custom architecture. Flow error handling directs exceptions to recovery flows. Apex batch jobs isolate chunk failures (a failed chunk does not prevent other chunks from processing), enabling partial job completion and targeted retry. Implement explicit retry logic using error tracking on AsyncApexJob for transient failure recovery. Apply exponential backoff to these retry patterns when handling transient failures. Self-recovering systems maintain availability targets even during off-hours incidents, when human responders may be unavailable, reducing operational burden while improving mean time to recovery.
- Design for business requirements first. Define service level objectives based on actual business impact before selecting technical solutions. Not every component requires five-nines availability. Match reliability investment to business criticality and design graceful degradation for supporting features. Realistic targets enable appropriate architecture choices and avoid over-engineering or under-delivering.
- Validate recovery through drills. Schedule disaster recovery drills that test backup restoration, failover procedures, and incident response playbooks. Restore a Full Copy Sandbox from production backup to validate recovery processes. Introduce deliberate failures in sandbox environments to confirm that your monitoring detects issues, and that automated recovery executes correctly. Drills reveal gaps in procedures, tooling, and runbooks before real incidents expose them. Document drill results and track remediation of discovered gaps. Regular testing ensures recovery capabilities remain current as solutions evolve and team membership changes.
Understanding what Salesforce operates helps you focus reliability design efforts on what you control. The platform handles infrastructure concerns that would require dedicated teams in traditional IT environments, such as:
- Multi-region infrastructure and failover: Salesforce Hyperforce provides regional data centers with automated failover of internal services. It also provides multiple availability zones within regions, and data replication at the infrastructure layer. The platform handles redundancy and availability zone distribution within regions transparently.
- Platform SLA commitments: Guaranteed availability levels with contractual remedies are negotiated on a per-customer basis. trust.salesforce.com publishes real-time platform status and uptime history, but any specific availability guarantee and its remedies live in your negotiated agreement. Review your contract and current Salesforce Trust and Compliance documentation for the commitments that apply to your org.
- Infrastructure redundancy: The platform maintains redundant servers, network paths, database infrastructure, and storage systems. Infrastructure-level failover happens automatically during hardware failures without customer action. Platform backups protect against infrastructure-level data loss.
- Platform maintenance and updates: Major platform releases deliver features and security patches with backward compatibility managed by Salesforce. Platform maintenance windows are scheduled and communicated on trust.salesforce.com. Infrastructure patching happens transparently, without customer involvement.
- Core platform health monitoring: Salesforce monitors infrastructure performance including database response times, network latency, API gateway health, and storage system performance. Platform health status appears at trust.salesforce.com and includes real-time incident updates. Instance-specific status is available via the Status API.
These platform operations create the foundation upon which you build. You don't manage data centers, provision servers, or design infrastructure disaster recovery. You're responsible for what you design and configure on top of this foundation.
The Shared Responsibility Model specifies that you own reliability for everything you create with Salesforce. Platform reliability enables your work but doesn't replace it. Your reliability responsibilities span six interconnected areas.
Service level objectives (SLOs) quantify reliability requirements in measurable terms. SLOs bridge business requirements and technical architecture. Before selecting technologies or designing data models, establish SLOs that define success for each critical user flow.
SLOs typically measure:
- Availability – percentage of time the system is operational and accessible.
- Latency – time required to complete operations, measured as percentiles (p50, p95, p99).
- Throughput – volume of operations completed successfully per unit time.
- Error rate – percentage of requests that fail or return errors.
- Recovery time – duration required to restore service after incidents.
Define SLOs per business capability rather than per technical component. User-facing capabilities require more stringent SLOs than administrative or batch processes. Each SLO should be objectively measurable using available instrumentation.
Service level agreements (SLAs) are contractual commitments with consequences for failure. Any guaranteed availability level and its contractual remedies are negotiated per customer. Review your agreement and current Salesforce Trust and Compliance documentation for the commitments applicable to your org.
Solution SLOs should be less stringent than platform SLAs to preserve the error budget. If your platform SLA and your solution SLO both target 99.9%, any significant platform downtime directly consumes your error budget—leaving no buffer for application-layer failures, integration issues, or planned maintenance within the same measurement period. An SLO is formally violated only when cumulative downtime exhausts the full error budget for the measurement period. If your SLA and SLO are set to the same target, a single platform incident can exhaust that budget entirely. For example, when the platform provides 99.9%, target a solution SLO of 99.5% to maintain a meaningful buffer for the issues you must address—application bugs, integration failures, and deployment windows.
Service level indicators (SLIs) are measurements used to assess SLO achievement. SLIs must be objectively measurable, consistently collected, and directly tied to user experience.
For Salesforce solutions, SLIs include:
- Platform uptime via trust.salesforce.com
- Page load time via Experience Cloud analytics
- API response time via Event Monitoring (requires the Event Monitoring add-on or Salesforce Shield)
- Transaction success rate via custom application logging
- Batch job completion via AsyncApexJob monitoring
Higher availability targets create exponentially increasing complexity and cost. Understand the architectural implications before committing to targets.
| Target | Annual Downtime | Monthly Downtime | Architectural Requirements |
|---|---|---|---|
| 99% | 3.65 days | 7.3 hours | Standard platform capabilities |
| 99.5% | 1.83 days | 3.6 hours | Basic redundancy, active monitoring |
| 99.9% | 8.76 hours | 43.8 minutes | Multi-region awareness, automated failover |
| 99.95% | 4.38 hours | 21.9 minutes | Active-active patterns, chaos testing |
| 99.99% | 52.6 minutes | 4.4 minutes | Multi-org architecture, comprehensive automation |
Avoid arbitrary targets like "five nines for everything." Instead, assess the business impact of downtime per capability and set targets accordingly. Internal batch reporting that tolerates 7 hours of monthly downtime requires fundamentally different architecture than revenue-critical order processing requiring sub-hour recovery.
Define reliability from a user perspective rather than from technical metrics alone. A system reporting 99.9% uptime but experiencing frequent timeouts fails user-experience-based reliability. Users care about completing their workflows successfully more than individual API uptime.
Design SLOs that reflect user journeys rather than individual API calls. A multi-step checkout flow requires every step to complete successfully within acceptable time. Measure end-to-end user flow completion rates as a primary reliability indicator. Component availability is necessary, but insufficient for user-experience reliability.
Salesforce Hyperforce provides regional data centers, enabling geographic distribution. The platform handles infrastructure redundancy within regions including multiple availability zones, automatic failover of internal services, and data replication at the infrastructure layer. Platform SLAs reflect this infrastructure redundancy.
For most solutions, single-region deployment with platform-managed redundancy provides sufficient availability. Trust Salesforce infrastructure for foundational availability and focus solution architecture on application-layer reliability, including fault-tolerant integration patterns, graceful degradation, and automated recovery.
Intra-region failover across availability zones is automatic and included in standard platform SLA commitments—Salesforce manages this transparently at the infrastructure layer. Cross-region (out-of-region) disaster recovery is a separate paid offering and is not included by default in any standard edition. If your business continuity requirements demand cross-region failover, document this dependency explicitly in your disaster recovery plan so stakeholders understand the distinction between included platform resilience and purchased cross-region DR capabilities.
Multi-org architecture provides the strongest isolation and geographic redundancy, but multiplies operational complexity, including data synchronization, user provisioning, deployment coordination, and license costs. Reserve multi-org patterns for scenarios where business requirements clearly justify the complexity. For example, consider a multi-org pattern for these scenarios:
- The business requires guaranteed RPO/RTO beyond platform capabilities
- Regulatory requirements mandate geographic data isolation, business continuity planning requires complete independence from a single region
- Org consolidation is infeasible due to business unit autonomy requirements.
Active-Passive Pattern: The primary org serves all traffic under normal conditions. Secondary orgs in different regions remain synchronized but idle. Failover occurs during a primary region outage. This solution provides the simplest multi-org pattern but leaves secondary capacity unused. DNS routing or user authentication layers direct users to the active org.
Active-Active Pattern: Both orgs serve production traffic continuously. Users are allocated by geography, business unit, or workload type. Active-active maximizes capacity utilization but requires sophisticated data synchronization and user routing. Conflict resolution is critical when the same record is modified in both orgs.
Design data synchronization appropriate to RPO requirements. Platform Events provide near real-time event streaming for critical data changes. Change Data Capture delivers automatic change tracking for selected objects with minimal development. Scheduled API replication via Bulk API 2.0 on fixed intervals suits less time-sensitive reference data.
Apply redundancy at data, application, and integration layers to prevent single points of failure. Layered redundancy ensures that failure at any single layer does not compromise overall system availability.
- Data redundancy: The platform provides data redundancy through infrastructure backups. Supplement this redundancy with application-level replication when the business requires faster recovery than platform restore procedures provide. Use Change Data Capture or platform events to replicate critical data to secondary storage or to external systems continuously. This enables recovery from logical corruption or from configuration errors that infrastructure backups can’t address.
- Application redundancy: Design stateless application logic so that any application server can process any request. Avoid server-side state that prevents horizontal scaling. Use custom metadata types and custom settings for configuration that must be instantly available across all application servers. Stateless design enables an application server to process requests without depending on specific server state.
- Integration redundancy: Design integrations that tolerate temporary external system unavailability. Implement circuit breaker patterns detecting failing integrations. Queue requests via Platform Events when external systems are down rather than blocking user operations. This isolates external system failures from user-facing functionality.
Monitor Salesforce platform health using trust.salesforce.com and instance-specific status APIs. Subscribe to status notifications for your instance to receive alerts about incidents, maintenance windows, and performance impacts. Platform health signals enable proactive response rather than reactive troubleshooting.
Design solutions that respond to platform health status. When platform performance degrades, reduce non-critical batch processing load. Defer background jobs during maintenance windows using scheduled job monitoring. Disable non-essential integrations to protect critical user-facing operations during incidents. This dynamic load shedding maintains reliability for critical capabilities under stress.
Use Scale Center to identify long-running transactions and operations that consume disproportionate platform resources. Scale Center provides transaction-level visibility, enabling architects to detect reliability risks before they become user-facing incidents. Weekly Scale Center review reveals patterns requiring architectural remediation.
Implement failure detection at multiple levels to catch issues before they cascade into full outages. Layered detection provides defense in depth against undetected failures.
| Detection Layer | Signal Source | What It Catches |
|---|---|---|
| Platform failures | trust.salesforce.com, Status API | Infrastructure incidents, maintenance |
| Integration failures | Timeout monitoring, error rate tracking | External system issues, network problems |
| Application failures | Exception logging, transaction success rates | Code defects, configuration errors |
| Performance degradation | Latency percentile monitoring | Slowdowns before complete failures |
| Capacity warnings | Proactive Monitoring alerts | Governor limits approaching, API exhaustion |
Design alert thresholds that balance early detection against false positives. Alert when error rates exceed thresholds or sustained degradation occurs, not on isolated failures. Single errors are normal in distributed systems. Patterns of errors indicate reliability problems that require attention.
Salesforce governor limits cap each tenant’s resource consumption in the multitenant platform so no single tenant can degrade performance for others. These are not arbitrary restrictions; they are architectural boundaries that shape solution design. Understand governor limits before designing a reliable architecture. Solutions that regularly approach governor limits under normal load will likely fail under stress.
Critical governor limits affecting architectural decisions:
| Resource | Synchronous Limit | Asynchronous Limit | Architectural Impact |
|---|---|---|---|
| SOQL queries | 100 per transaction | 200 per transaction | Query consolidation, relationship queries |
| DML statements | 150 per transaction | 150 per transaction | Bulk DML, collection operations |
| Heap size | 6 MB synchronous | 12 MB asynchronous | Data chunking, streaming patterns |
| CPU time | 10,000 ms synchronous | 60,000 ms asynchronous | Algorithm efficiency, async offloading |
| Callout timeout | 120 seconds total | 120 seconds total | Timeout budgeting across callouts |
| API calls (24hr) | Varies by edition | N/A | Integration batching, caching |
Design transactions that complete well within limits, even under peak load. Build margin by targeting 70% of governor limits as the operational ceiling under normal conditions, reserving 30% for unexpected spikes. This buffer accommodates temporary load increases, typically without hitting hard limits.
Bulkification is the foundational scalability pattern for Salesforce. Process multiple records in a single transaction rather than in individual record operations. Bulkification reduces governor limit consumption while increasing throughput. Every Salesforce architect must master bulkification patterns, as they underpin all scalable solutions.
Design all Apex triggers, batch classes, and integrations to process record collections efficiently. Collect record identifiers first, then process all records with single-query and DML statements. Use maps and sets for efficient lookups rather than nested loops with individual queries. Collection-based processing provides order-of-magnitude efficiency improvements over record-by-record approaches.
Record-triggered automation must handle 200 records per trigger invocation, because the platform processes trigger execution in batches of up to 200 records. Lightning Data Service operations batch automatically, but custom components must implement bulk patterns explicitly when performing DML operations.
Asynchronous processing distributes work across time rather than attempting immediate completion within a single transaction's governor limits. Use asynchronous patterns when operations process large data volumes exceeding synchronous governor limits, depend on external systems with variable response times, can tolerate delayed completion, or require extended execution time beyond synchronous CPU limits.
Salesforce asynchronous capabilities and their architectural fit:
- Batch Apex: Process large record volumes in chunks of up to 2,000 records per execute method. Batch provides dedicated governor limits per chunk and failure isolation — a failed chunk does not prevent other chunks from completing. This enables partial success and targeted retry. Implement custom retry logic for transient failures by tracking failed chunk ranges on the AsyncApexJob object, and re-enqueuing targeted batch jobs. Use batch for data migrations, scheduled bulk updates, and large-scale data processing. There’s a maximum of five batch jobs running or pending execution concurrently per org. Additional jobs queue in the Apex Flex Queue (up to 100 jobs in Holding status) and execute automatically as slots open.
- Queueable Apex: Execute asynchronous jobs with chaining capability to enable multi-step workflows and complex object parameters. Queueable Apex shares the org-wide DailyAsyncApexExecutions limit of 250,000 executions per 24 hours with all other async Apex — Batch, Future, and Scheduled Apex — rather than holding a dedicated Queueable-specific allocation. Use Queueable Apex for multi-step orchestration and integration workflows requiring sequential processing with better monitoring than @future methods.
- Platform Events: Platform events are used in a publish-subscribe event architecture that decouples publishers from subscribers. Events replay from a retention window of 72 hours (3 days). Extended retention beyond 72 hours is available as a paid add-on — verify current maximum limits and GA status in the latest Salesforce Platform Events documentation before committing to SLA obligations that depend on extended replay. Use Platform Events for event-driven automation, cross-system integration, and real-time data streaming. Platform Events provide natural asynchronous boundaries between transaction phases.
- Scheduled Apex: Execute jobs on a fixed schedule using CRON expressions via System.schedule(). A job can be scheduled to run at most once an hour — the CRON seconds and minutes fields must use fixed values, not ranges. Stay within the maximum of 100 scheduled Apex jobs per org by consolidating similar operations into single Schedulable classes.
When data volumes exceed practical processing limits even with bulkification and async patterns, partition data across logical boundaries to enable parallel processing. Data partitioning converts large sequential operations into smaller parallel operations that complete faster and stay within governor limits.
- Date-based partitioning: Process data in time windows including this month's transactions or last quarter's cases. Archive historical data to Big Objects or external storage to keep the working set manageable. Most transactional queries focus on recent data making time-based partitioning naturally efficient.
- Record type partitioning: Process different record types independently including Partner cases vs. Customer cases or Enterprise accounts vs. SMB accounts. Separate batch jobs per type enable parallelization. Record type often correlates with distinct business processes justifying independent processing.
- Owner-based partitioning: Distribute processing by record owner, such as processing each sales region's opportunities independently. Owner-based partitioning is particularly effective when combined with a sharing model, as security is enforced through existing mechanisms. Owner-based partitioning enables geographic distribution of processing load.
Project future capacity requirements based on business growth rather than reacting to limit exhaustion. Proactive capacity planning prevents reliability incidents caused by running out of platform resources.
- User licenses – headcount growth driving API call allocation and storage entitlements per user.
- Data storage – transaction volumes and retention policies driving storage consumption (nominally plan for 10–20% annual growth at a minimum).
- API calls – integration count and frequency driving 24-hour API allocation (each new integration pattern adds recurring consumption).
- Processing capacity – Batch job count and complexity driving async processing queues and concurrent execution limits.
Use Proactive Monitoring to continuously evaluate org capacity use. Proactive Monitoring surfaces capacity risks including API use approaching request limit spikes, storage nearing limits, and batch job queue depth growing beyond sustainable levels. Weekly capacity review enables procurement lead time for additional licenses or limits before business impact occurs.
Validate scalability assumptions through load testing before production deployment. Load testing reveals governor limit issues, integration bottlenecks, and capacity constraints invisible in low-volume development testing. Test with production-scale data volumes and concurrency to validate reliability under realistic conditions.
- Data volume testing: Populate production-scale data volumes in Full Copy Sandbox to validate query performance with real data skew, relationship depth, and record counts. Test with 10 million+ records when production will reach that scale. Query-optimizer behavior changes dramatically as data volume increases, which can result in misleading small-scale test results.
- Concurrent user testing: Simulate peak concurrent user load to validate transaction throughput and contention. Use scale tests available for qualifying orgs to simulate production workloads in sandbox environments before deployment. Concurrent execution reveals locking issues invisible in single-user testing.
- API load testing: Generate peak API volumes to validate integration scalability, rate limit handling, and circuit breaker behavior under sustained load. API load tests reveal whether retry logic and error handling operate correctly under stress conditions.
- Scale Test: Scale Test is a Salesforce product used to simulate production workloads against Full Copy Sandbox environments scaled to match production capacity. Scale Test runs against Full Copy Sandboxes on Hyperforce. Your production instance does not need to be on Hyperforce to use it. You create test plans in your production org, while the tests execute against the sandbox. Use Scale Test to validate governor limit headroom, async processing throughput, and integration response behavior under peak load conditions before major deployments.
Graceful degradation maintains core functionality when non-critical components fail. Design systems prioritizing critical user flows over supporting features during failures. Not all capabilities have equal business importance and architectures should reflect these priorities.
Define feature criticality hierarchy:
| Tier | Description | Degradation Behavior | Example |
|---|---|---|---|
| Critical | Revenue or compliance | Never degraded, full redundancy | Payment processing, audit logging |
| Important | Core user workflows | Degraded only during major incidents | Case creation, opportunity updates |
| Supporting | Enhanced experience | Disabled during any integration failure | Recommendations, enrichment |
| Optional | Nice-to-have features | Disabled proactively during high load | Analytics widgets, social feeds |
This hierarchy enables architects to design degradation policies that maintain business continuity even during partial system failures. Users prefer reduced functionality over complete unavailability.
The circuit breaker pattern prevents cascading failures when integrations become unavailable. Instead of accumulating timeouts that consume transaction time and governor limits, detect failure patterns and stop calling failing systems. Circuit breakers provide fast failure rather than slow failure.
Circuit breaker states:
- Closed – normal operation, requests flow to external system as designed.
- Open – failure threshold exceeded, requests fail immediately without attempting external calls, saving resources.
- Half-open – recovery test period, limited requests probe external systems to detect recovery before fully closing the circuit.
Implement circuit breakers using Platform Cache to store circuit state accessible across all transactions. Use Platform Events to broadcast state changes across the org. Circuit breaker logic checks state before attempting external calls, avoiding wasted callout limits on known-failed systems.
Transient failures are normal in distributed systems. Network interruptions, temporary service unavailability, and rate limit responses often resolve within seconds. Implement retry logic that repeats failed operations after progressive delays rather than failing immediately.
Exponential backoff prevents retry storms that overwhelm recovering systems. A first retry can occur after 1 second, a second after 2 seconds, a third after 4 seconds, and fourth after 8 seconds. Cap maximum delay at 30–60 seconds regardless of exponential growth. This backoff pattern gives failing systems time to recover while limiting total retry duration.
Match retry strategy to failure type.
- Network timeouts: Retry with a short backoff (the operation may not have reached the server).
- Rate limit errors (429): Retry after the Retry-After header value or after the rate limit reset time.
- Server errors (5xx): Retry with exponential backoff, as the server may be temporarily overloaded.
- Client errors (4xx except 429): Don’t retry; fix the request as error indicates invalid input.
- Governor limit errors: Don’t retry in the same transaction; re-queue as an async operation with dedicated limits, for example by publishing a failure event that an asynchronous subscriber reprocesses with backoff under fresh transaction limits.
Fallback strategies define alternative approaches when primary methods fail, enabling continued operation under degraded conditions.
- Alternative data source: Retrieve data from the platform cache session or from an org partition when the real-time API is unavailable. Pre-populate the cache during successful operations. The cache provides stale but available data, which is better than complete failure for many use cases.
- Default behavior: Apply standard business rules when a personalization or enrichment service is unavailable. Process with default values, and flag for enrichment when the service recovers. Default behavior maintains throughput at the cost of reduced precision.
- Manual process: Enable manual operation completion when automation fails. Provide an admin interface for completing stuck transactions. Manual fallback prevents data loss and maintains business continuity when automation is impaired.
- Queue for retry: Store operations in platform events or custom queue objects for processing when an external system recovers. Platform event replay with 72-hour standard retention enables subscriber recovery after temporary failures, without data loss.
Configure appropriate timeouts for all integration callouts. Salesforce enforces a maximum of 120 seconds total callout time per transaction. Budget this time across all callouts within a single transaction to avoid exhausting transaction time on hung connections.
Timeout design considerations:
- User-facing synchronous callouts: Use 5–10 seconds maximum to maintain responsive UI as users tend not to wait longer.
- Background async callouts: Use 30–60 seconds to accommodate variable external performance without user waiting.
- Batch processing callouts: Use the full 120 seconds allowed when no user is waiting for a response.
- Multiple callouts per transaction: Budget total time across all callouts, such as three callouts at 10 seconds each consuming 30 seconds of your 120-second budget.
Shorter timeouts fail faster, enabling fallback strategies to engage sooner. Longer timeouts increase success rate for slow-but-functional external systems. Balance considerations are based on whether the user is waiting for a response, and the availability of fallback strategies.
Design comprehensive error handling to transform failures from crashes into managed degradation:
- Fail fast: Validate inputs and preconditions at entry points. Check governor limit consumption before expensive operations. Detect failures immediately rather than propagating invalid state through multiple processing layers. Early detection reduces blast radius and simplifies debugging.
- Fail gracefully: Maintain user capabilities even when operations partially fail. If 3 of 200 records in batch fail validation, process the 197 successful records and report the 3 failures rather than failing the entire batch. Partial success is better than total failure for batch operations.
- Fail informatively: Log errors with transaction ID, user context, input parameters, and stack trace. Insufficient error context is the primary obstacle to rapid incident resolution. Every error log should enable the responder to understand what failed, why, and how to reproduce.
- Fail safely: Ensure failures don’t compromise data integrity or security. Rollback partial transactions rather than leave data in an inconsistent state. Never expose internal error details to end users, as stack traces reveal implementation details useful to attackers.
Recovery Time Objective defines maximum acceptable downtime after disaster. RTO drives architectural decisions about failover automation, backup frequency, and recovery testing investment. Different business capabilities justify different RTO investments. Because RTO is the downtime your users experience directly, a missed target translates into prolonged outages and eroded customer trust.
RTO varies by business capability:
| Capability Type | Typical RTO | Architectural Implication |
|---|---|---|
| Revenue-critical operations | Minutes | Automated failover, hot standby |
| Customer-facing services | 1–4 hours | Warm standby, scripted recovery |
| Internal business tools | 4–24 hours | Cold standby, manual recovery |
| Historical reporting | Days | Restore from backup on demand |
Define RTO per capability before designing disaster recovery architecture. RTO shapes technology selection, automation investment, and testing cadence. More aggressive RTO targets require greater investment in automation and redundancy.
Recovery Point Objective defines the maximum acceptable data loss window measured in time. RPO determines backup frequency, replication strategy, and synchronization patterns. More stringent RPO requires more frequent data replication, increasing complexity and cost. Because RPO is the data loss your business absorbs, a missed target can mean lost transactions and unrecoverable gaps in your records.
| Data Type | Typical RPO | Replication Strategy |
|---|---|---|
| Financial transactions | Near-zero (seconds) | Event-driven async replication on every commit |
| Customer records | Near-zero (minutes) | Change Data Capture, async replication |
| Analytics data | Hours | Scheduled batch sync |
| Temporary workflow state | Days | No replication needed |
Balance RPO requirements against cost and complexity. Near-zero RPO requires continuous data replication with significant infrastructure investment. Daily backup provides 24-hour RPO with minimal complexity. Most organizations can tolerate some data loss for non-financial data.
Platform infrastructure redundancy protects against infrastructure failure, but it replicates the results of user error, flawed deployments, and integration defects, which cause most data loss. Backups exist to recover from these application-layer failures, not to compensate for platform reliability. Implement backup strategies covering data, metadata, and files as each requires different backup approaches:
- Data backups: Implement a comprehensive backup strategy addressing both data and metadata. Export critical object data using the native Data Export Service — every 7 days for Enterprise, Performance, and Unlimited editions; every 29 days for Professional and lower editions. Export files are available for 48 hours after the notification email is sent, not including weekends, before automatic deletion. Set up an automated download process so files aren’t permanently lost. Use the Metadata API and Salesforce CLI (sf project retrieve) to version-control org configuration, custom code, and declarative automation in a source control system such as Git. Treat metadata backup as part of your standard CI/CD pipeline. Supplement data backup with dedicated backup and recovery services such as Own for point-in-time restore, granular record-level recovery, and retention beyond native export cadence. Native Data Export does not support point-in-time restore, and third-party tooling is required if your RTO/RPO demands granular recovery windows. Test restore procedures regularly; a backup that has never been restored is an untested assumption.
- Metadata backups: Version control all metadata using Salesforce DX source format in Git repositories. Metadata version control enables rapid configuration restore after corruption or unintended changes. Every deployment should be reproducible from source control. Metadata in Git provides point-in-time recovery for configuration.
- File backups: Export ContentVersion records, attachments, and documents to external storage. Salesforce works best for active data, not long-term file archiving - export files to external storage for regulatory retention. Implement automated file export for regulatory retention requirements that exceed platform capabilities.
- Validation: Periodically restore backups to scratch orgs or to developer sandboxes to validate both procedures and backup integrity. Untested backups often fail when needed due to incomplete backup scope or corrupted archives. Schedule quarterly restore validation to catch issues before disasters occur. Monitor backup freshness in addition to restore validation. Alert when the most recent successful backup is older than its expected cadence - for example, when a weekly export has not completed in more than 8 days. With this validation, a silently failed backup job surfaces immediately rather than at the next drill.
Design replication appropriate to RPO and multi-org requirements:
- Change Data Capture (CDC): Subscribe to change events for tracked objects. CDC delivers create, update, delete, and undelete events with changed field values. Provides near real-time replication with minimal development effort for supported objects including standard and custom. Subject to daily delivery allocation based on edition.
- Platform Events: Custom event architecture for replicating business events and state changes. More flexible than CDC supporting custom payloads and complex event structures but requires explicit publish logic in triggers or processes. The 72-hour replay window standard enables recovery from temporary subscriber failures.
- Scheduled API replication: Schedule API replication is batch extraction via Bulk API 2.0 on a fixed schedule. It’s the simplest implementation with an RPO equal to extraction frequency. Scheduled API replication is suitable for non-critical data where near real-time execution is unnecessary, such as with reference data or historical analytics.
- MuleSoft-orchestrated replication: For complex multi-system replication topologies, MuleSoft Anypoint Platform provides orchestration, transformation, and monitoring. Its use is appropriate for replication across Salesforce and multiple external systems, which requires sophisticated routing and transformation logic.
Test disaster recovery with scheduled drills that validate people, processes, and technology together:
- Tabletop exercises: The team walks through disaster scenarios, discussing roles and decision points without actual failover. This activity is low cost, and it reveals procedural gaps and communication breakdowns. Conduct Tabletop exercises regularly to maintain team readiness as personnel change.
- Partial failover: Test specific recovery procedures such as metadata restore from source control, data restore from backup service, or sandbox refresh. This validates technical procedures with limited business impact. Conduct regularly, rotating which procedures are tested to cover all recovery capabilities annually.
- Full failover drill: Full failover is complete failover to the disaster recovery environment with production traffic cutover. It provides the highest confidence but requires business coordination and user communication. Conduct this drill annually for critical systems. The full failover drill validates the entire recovery capability, including cutover procedures and user communication.
Document lessons learned after every drill. Update runbooks based on findings. Recovery capabilities can degrade as teams change and solutions evolve. Treat DR documentation as living artifacts that require regular maintenance rather than one-time deliverables.
Business continuity extends beyond technical recovery to encompass people, processes, and vendor dependencies:
- Team availability: Document escalation procedures and backup personnel for critical roles. Ensure no single point of failure exists in operational knowledge. Primary responders may be unavailable during disasters, which makes backup staffing critical.
- Communication procedures: Define how incidents are communicated to users, customers, and executives. Establish communication channels that function when primary tools (for example, external status pages or SMS notification systems), including Salesforce itself, are unavailable.
- Vendor dependencies: Map external vendor dependencies critical to solution operation. Document escalation paths and contractual SLAs for each critical vendor including Salesforce, integration partners, and ISV package providers. Understand, for example, which vendors provide 24/7 support and which have business-hours-only support affecting recovery timing.
- Regulatory obligations: Identify notification requirements triggered by extended outages. Financial services, healthcare, and government contracts often mandate incident notification within specific timeframes. Non-compliance can create regulatory and legal risk, both of which compound disaster impact.
Define a health model that aggregates multiple signals into overall system health status. Health models surface operational status at a glance, without requiring analysis of detailed metrics. For example:
| Health Dimension | Signals | Green | Yellow | Red |
|---|---|---|---|---|
| Service health | Transaction success rate | >99.5% | 98–99.5% | <98% |
| Integration health | External system availability | All responding | Degraded response | Circuit breaker open |
| Data health | Sync job success, data quality | All current | Behind schedule | Failed or stale |
| Capacity health | Governor limit consumption | <70% | 70–85% | >85% |
Design health dashboards that show operational status immediately. Health status guides operational response, including normal operations under green status, heightened monitoring under yellow status, and active incident response under red status.
Platform status signals (covered earlier under Platform Health Monitoring) reveal when Salesforce infrastructure is degraded, but not how your own solution is performing.
Augment those signals with solution-specific observability:
- Event Monitoring: Event Monitoring includes detailed logs that capture API calls, page views, report exports, login activity, and Apex execution. EventLogFile objects deliver logs with 24-hour (daily) or 1-hour frequency – hourly delivery requires the Event Monitoring add-on or Salesforce Shield. Retention is configurable up to 365 days via Setup, but extended retention requires Salesforce Shield or the Event Monitoring add-on. Without an add-on, log files are retained for 1 day. Route events to an external Security Information and Event Management (SIEM) system or to a log aggregation platform for correlation, alerting, and retention beyond native limits. Use Event Monitoring to detect abnormal API consumption patterns, to identify runaway Apex processes, and to audit data access in regulated environments.
- Scale Center: Scale Center provides transaction-level visibility into long-running operations, row lock contention, and resource-intensive transactions. Scale Center enables architects to identify reliability risks from specific transaction patterns before they cause user-facing incidents. Weekly reviews can reveal optimization opportunities.
- Proactive Monitoring: Proactive Monitoring enables continuous evaluation of org health, surfacing performance and scalability risks. Proactive Monitoring provides alerts on API request limit spikes, concurrent Apex execution failures, SOQL row limit issues, and storage consumption trends. It’s available to customers with a Signature Success (formerly Signature Support) entitlement — it is not a self-service feature included with standard editions.
Monitor application performance from the user perspective rather than the infrastructure perspective:
- Real User Monitoring (RUM): Measure actual user experience through Experience Cloud analytics or custom instrumentation. RUM captures real latency, reflecting actual network conditions, device performance, and geographic distribution. Synthetic monitoring can’t replicate this variability.
- Synthetic monitoring: Execute automated transactions periodically from multiple locations to validate availability and performance. Synthetic monitoring detects issues before users report them. Implement synthetic monitoring using scheduled Apex, executing critical operations and reporting results with Platform Events.
- Transaction tracing: Instrument complex multi-step operations to capture timing per step. Next, identify which step in the five-step workflow introduces latency. Don’t treat the whole flow as black box. Step-level timing reveals optimization opportunities invisible in aggregate metrics.
Monitor all external integrations using error rates, latency percentiles, and throughput. Integration failures are a leading cause of reliability incidents:
- Error rate – The percentage of calls returning errors (target: <1% for healthy integration).
- Latency – The response time measured at p50, p95, p99 (set SLO per integration based on timeout budget).
- Timeout rate – The percentage by which configured timeout (target: <0.1%) is exceeded.
- Circuit breaker state – Open state indicates sustained failure requiring immediate attention.
- Queue depth – For async integrations, growing queue indicates processing falling behind production rate.
Log all integration calls with request ID, endpoint, response code, and duration. This data enables rapid root cause analysis when integration failures affect reliability. Integration logs should enable aggregation and trend analysis.
Design alerts to problems before users are impacted, enabling proactive responses:
- Actionable alerts: Every alert has a defined response action and an assigned responder. Alerts without clear response create fatigue and obscure critical signals. Alert design should cover who responds, what they check, and how they remediate.
- Appropriate urgency: Page on-call personnel for user-impacting failures. Send email for degraded performance. Include issues in a daily digest to capture concerning trends. Mismatched urgency creates either alert fatigue from over-escalation, or missed incidents from under-escalation.
- Context-rich notifications: Include the threshold crossed, the current value, the recent trend, and a link to a relevant dashboard or runbook. Enable responders to begin diagnosis immediately without gathering context. Every alert should include enough information to triage without additional queries.
- Storm suppression: When multiple systems fail simultaneously, suppress redundant alerts. One alert indicating integration platform failure is more actionable than 50 individual integration failure alerts obscuring the root cause.
Anomaly detection identifies unusual patterns, and can indicate emerging problems invisible to static threshold alerts:
- Volume anomalies: Transaction volumes significantly above or below expected daily patterns may indicate runaway processes or user access issues.
- Error rate anomalies: Error rates elevated compared to same-time-last-week baselines catch gradual degradation before a threshold breach occurs.
- Latency anomalies: Response times drifting upward across multiple days indicate capacity saturation or performance regression.
- Behavioral anomalies: Unusual login patterns, unexpected API usage spikes, and batch jobs running outside scheduled windows can indicate suspicious use.
For customers with the Signature Success entitlement, Proactive Monitoring provides platform-level anomaly detection without additional configuration. Supplement it with application-specific anomaly detection for custom SLIs exported to external analytics platforms. Use anomaly signals to guide investigation rather than triggering immediate escalation, as anomaly detection has higher false positive rates than threshold alerts.
Use this checklist during architecture reviews, before production deployment, and periodically for ongoing reliability assessment.
Reliability Targets and SLOs
- Define SLOs for all critical user flows before design begins.
- Establish measurable SLIs tied to user experience, not just infrastructure metrics.
- Set realistic availability targets based on business impact analysis, not arbitrary goals.
- Ensure solution SLOs are less stringent than platform SLAs to provide error budget.
- Document availability targets and rationale in architecture decision records.
High Availability Architecture
- Design redundancy at data, application, and integration layers.
- Monitor platform health via trust.salesforce.com and instance status API.
- Implement application health checks independent of platform status.
- Consider multi-org architecture only when business requirements clearly justify complexity.
- Design automated failover with tested runbooks for multi-org patterns.
Scalability and Capacity Planning
- Design transactions completing within 70% of governor limits under normal load.
- Implement bulkification patterns in all Apex triggers, batch classes, and integrations.
- Use asynchronous processing for operations exceeding synchronous limits.
- Batch all API integrations rather than making individual record calls.
- Conduct load testing with production-scale data volumes before deployment.
- Monitor capacity utilization via the OrgLimits API or custom Apex and alert as consumption approaches the 70% operational ceiling.
- Project capacity requirements based on 12-month growth expectations.
- Partition high-volume data by date, record type, or owner to enable parallel processing when volumes exceed sequential limits.
Fault Tolerance and Resilience
- Design graceful degradation with defined feature criticality tiers.
- Implement circuit breakers for all external system integrations.
- Apply retry logic with exponential backoff for transient failures.
- Configure timeouts appropriate to operation type (5–10s user-facing, 30–60s async).
- Design fallback strategies using Platform Cache and queue-based patterns.
- Implement structured error handling with sufficient diagnostic context.
Disaster Recovery and Business Continuity
- Define RTO and RPO per business capability before design.
- Implement automated backup for data, metadata (source control), and files.
- Validate backup restore procedures quarterly in non-production environments.
- Monitor backup freshness and alert when a scheduled backup is overdue.
- Design data replication strategy matching RPO requirements.
- Conduct disaster recovery testing annually (tabletop quarterly).
- Document business continuity procedures including vendor escalation paths.
Monitoring and Observability
- Define health model aggregating service, integration, data, and capacity signals.
- Subscribe to platform status notifications for your Salesforce instance.
- Implement real user monitoring for critical Experience Cloud flows.
- Monitor integration health with error rate, latency, and circuit breaker state.
- Design actionable alerts with defined response procedures and ownership.
- Route Event Monitoring data to external platforms for long-term retention and analysis.
- Use Proactive Monitoring and Scale Center for continuous reliability risk assessment.
- Apply anomaly detection for volume, error rate, and latency patterns to catch degradation that static thresholds miss.