failovers
A failover shifts Workflow Execution processing from an active Temporal Namespace to replicated Temporal Namespace during outages or other incidents. Standby Namespace replicas duplicate data and prevent data loss during failover.
Triggering failovers
Temporal automatically initiates failovers when an incident or outage affects a high availability Namespace. You can also trigger a failover based on your own custom alerts and for testing purposes. This section explains how to manually trigger a failover and what to expect afterward.
Always check the metric replication lag before initiating a failover. A forced failover when there is a large replication lag has a higher likelihood of rolling back Workflow progress.
For details on how Temporal detects conditions and triggers failovers automatically, see the failover process.
Initiating manual failovers
You can trigger a failover manually using the Temporal Cloud Web UI or the tcld
CLI, depending on your preference and setup.
The following table outlines the steps for each method:
Method | Instructions |
---|---|
Temporal Cloud Web UI | 1. Visit the Namespace page on the Temporal Cloud Web UI. 2. Navigate to your Namespace details page and select the Trigger a failover option from the menu. 3. After confirmation, Temporal initiates the failover. |
Temporal tcld CLI | To manually trigger a failover, run the following command in your terminal: tcld namespace failover \ --namespace <namespace_id>.<account_id> \ --region <target_region> Temporal fails over the Namespace to the target region. High availability Namespaces using a single region will failover to the standby isolation domain. |
Post-failover event information
After any failover, whether triggered by you or by Temporal, event information appears in both the Temporal Cloud Web UI (on the Namespace detail page) and in your audit logs.
The audit log entry for Failover uses the "operation": "FailoverNamespace"
event.
After failover, the replica becomes active, taking over and the Namespace is active in the new isolation domain or region.
You don't need to monitor Temporal Cloud's failover response in real-time. Whenever there is a failover event, users with the Account Owner and Global Admin roles automatically receive an alert email.
Failbacks
After Temporal-initiated failovers, Temporal Cloud shifts Workflow Execution processing back to the original region or isolation zone that was active before the incident once the incident is resolved. This is called a "failback".
Disabling Temporal-initiated failovers
When you add a replica to a Namespace, it becomes a high availability Namespace. In the event of an incident or an outage Temporal Cloud automatically fails over a high availability Namespaces to its replica. This is the recommended and default option.
If you prefer to disable Temporal-initiated failovers and handle your own failovers, you can do so by navigating to the Namespace detail page in Temporal Cloud. Choose the "Disable Temporal-initiated failovers" option.