On November 18, AWS released cross-region inference routing for SageMaker AI and published a case with Salesforce about using it in an internal gateway. The provider can select among already deployed endpoints in different AWS regions and accounts. Salesforce keeps requests with a regional residency requirement on a local-only path and allows other traffic to use approved external regions when the local endpoint is in trouble. The adoption case is strong for a company already running the same model in several places with uneven spare capacity. The advantage depends on being able to say what may move, who has reserved the capacity that receives it and how the team will know a request took the intended path. The Salesforce and AWS case

The test supplies an important limit. In one saturation exercise, latency went up for several minutes while calls still stayed in the original region. Traffic began moving to a second region after errors appeared. The case is therefore a useful illustration of capacity sharing under defined rules, with a separate business question about the time users can wait before the rules take effect. I would evaluate the service as a way to take some routing operations off a team’s hands and would leave the employer responsible for eligibility, endpoints, recovery and customer communication. Buying a managed decision about where a call goes still leaves decisions about what the call is allowed to contain and when it has to finish.

The headline footprint describes the problem, not the rollout#

The Salesforce Agentforce Foundation team describes serving models in more than 30 global regions across numerous AWS accounts. That is the footprint of its model service. The published routing test is narrower and the authors leave wider adoption as another step. This difference matters when executives read customer studies. A company already supporting many regions can give a useful description of the problem without showing that a new approach has been tested everywhere. In Salesforce’s case, usage peaks and new customers leave some endpoints short of capacity while others have spare room. Reserving enough in each region to meet every overlap can be expensive. Sharing some capacity could help if the requests and receiving region are permitted to participate.

The conventional option is to own the regional switching in the application or a gateway. Salesforce says its earlier gateway used health checks, retries and region switching, but had incomplete visibility into the capacity of origin and destination endpoints. It also wanted to limit the effect of shifting load during an incident. Moving part of that logic into a managed service can reduce the amount of behaviour each application maintains. It changes how the business sees and tests the routing decisions. A manager should ask which decisions the new provider takes, which policies the company keeps and which emergencies the team still needs to rehearse. These are responsibilities that survive a shorter calling path.

The launch offers a useful comparison to buying raw capacity. AWS says the feature started in 32 commercial regions for real-time inference and 25 for asynchronous inference. Those are provider coverage figures, separate from the Salesforce footprint and the two-region real-time exercise. They indicate where to check availability but do not place endpoints in the customer’s account. The rollout choice begins with the regions and models the customer actually operates. An organisation with a second endpoint running a different model or without room to receive traffic has not acquired usable capacity simply by adding that region to a sentence in a contract. AWS’s release announcement

Decide eligibility before making a capacity decision#

Salesforce’s internal gateway checks a request’s residency requirement before it enters the broader routing path. A request that must stay in a region is assigned to a local-only destination. A request that can leave may use other explicitly approved endpoints. This is a useful form of separation because the two classes have different failure behaviour. One has fewer destinations and may face more limited capacity when its region is strained. The other can take advantage of regional spillover. A single service-level statistic that mixes them can hide the people whose requests had no permitted backup. The group buying the routing service needs to understand the availability it can offer across both classes.

Stylized diagram of requests split at a residency gate, one looping to a local server and one crossing to approved regional endpoints.

An employer’s policies cannot be inferred from a region selection screen. A contract can tell an organisation which data or workloads must remain within an agreed boundary. A model serving team can then translate that into a list of eligible destinations and a local behaviour when none is available. Other teams may need a different definition for the same geography. One service might handle customer conversations while another processes aggregate examples that can take a different route. The application owner needs a reliable way to identify which policy a request belongs to and a record of the decision. Capacity logic is easier to accept when it operates within a boundary the customer can explain.

It is also useful to test what happens when eligibility information is absent or contradictory. An application with one policy should have a defined outcome when it cannot establish which route is allowed. A local-only or paused request may be appropriate for one workload, while a broader allowed set might be documented for another. The decision belongs to the organisation and the commitments it made, not to the traffic level at the time. These are exactly the situations that can arise during a busy release or a change in a product’s input. The case is persuasive because Salesforce keeps a business rule visible before using a managed mechanism for the rest. A buyer can ask which rule it needs to preserve in its own gateway.

The receiving endpoint needs a budget of its own#

A router can direct requests to a destination the customer already runs. It cannot deploy a model, copy the model’s artifacts or supply new instances by itself. The endpoint owner therefore has a capacity decision even when someone else runs the router. If an already busy region accepts every request from a neighbour, a local problem can spread. AWS provides an optional cross-region limit on real-time endpoints to preserve room for local traffic. That is a practical protection to evaluate. The limit has value if the people who share the resource can agree how much traffic is compatible with their own customers’ needs, and measure what the routing path does when it reaches the limit.

The budget has two parts. There is the capacity a receiving endpoint already needs during its own peaks and the spare capacity available for another region. The two regions may have overlapping patterns or independent ones. A second region that is usually quiet can be an effective reserve for a known spike. It can be much less effective during an incident that increases traffic in both. In the Salesforce account, evaluating cascading behaviour at the cross-region cap is part of the work to do before a larger rollout. That is the right distinction. A team should measure normal operation and an unfavourable overlap before it can base a new availability promise on shared capacity.

Cross-account access gives this a responsibility question. Salesforce describes an endpoint policy that permits a named router, leaving the owner of the endpoint in control of authorisation. The receiving owner has to know the service owners, which paths are approved and when an authorisation can be changed. In an incident, it helps to have a person who can say how much overflow a receiving endpoint is willing to take and how that decision affects other customers. Buying a common routing layer can simplify the path for callers. It does not eliminate the agreement between the teams that supply and use the resource. AWS’s data protection and router documentation

Read the Salesforce timing as a sequence, not an availability pledge#

In the reported exercise, Salesforce saturated its Oregon endpoint and measured with a separate probe. For roughly five minutes while the local endpoint was saturated, median probe latency rose from 0.2 seconds to approximately 45 to 58 seconds and the traffic stayed in Oregon. The case says latency alone did not cause rerouting. Once the model container’s timeout began to produce errors, some requests reached Ohio about two minutes after the first errors. Within another minute about 78 percent of the probe traffic used Ohio. Keeping those observations in order explains why a technology buyer should care about the trigger as much as the spare capacity. AWS on how the router selects a destination

After the shift, the reported median returned to about 0.2 seconds, with the cross-region path adding under 100 milliseconds at the median in that setup. Following recovery, traffic returned gradually over about ten minutes. The probe recorded no errors in the failover and return phases according to the case. That is a result about the probe in this exercise, separate from the errors on the saturated endpoint and independent of what another workload would see. Capacity and latency in different regions can vary. A company should decide which aspect of the demonstration to reproduce rather than put the test’s times in every new service agreement.

From a business perspective, the period before rerouting can be more significant than the fast response after it. An application that expects a short interactive response may have already lost some of its audience if its calls spend a minute waiting. A batch operation can make a different trade. It might tolerate a longer recovery and prefer to avoid requests being routed to a secondary region on the basis of slow responses alone. A product manager needs to decide what the user should see or be able to do in the period while the service is determining the state. The routing report gives an example of how that period starts and ends. It leaves the application owner with an important part of the user experience.

Track a request’s destination and why it got there#

The Salesforce case says the gateway records the endpoint that served a response and the reason the router chose it alongside request identifiers. AWS reports aggregate destination metrics through CloudWatch. That combination helps the operator distinguish normal local calls from calls that moved after errors. A team can use the same model for an operating review. The observability worth budgeting for describes what the system did under the business policy, with enough information to see if the policy held. An application owner needs a way to tell where a request was handled and which owner to contact when a result is unexpected. AWS’s routing metrics guide

Recording location and reason does not require making the input text broadly visible. A request identifier, the approved service destination and a reason can support investigation with restricted access to details when needed. The available records still require retention choices and people responsible for them. It is a good time to ask how the organisation will verify that the choices are correct before it needs to reconstruct a mistake. It can use approved test inputs and compare the origin and destination history with the documented policy. A useful dashboard should explain enough to measure demand and errors. A useful record should allow a particular decision about a particular request to be examined later.

Version parity is another decision that will appear in those records. One region may run a different model version, container or artifact after an update. The feature can reroute to the named endpoint without making that deployment equivalent. If the outputs differ, a reported recovery in availability could still be a regression in quality for the user. A team can require comparable behaviour on approved examples before allowing an endpoint to receive overflow. It can record the version used and plan what happens when a region is ahead or behind. The platform gives the service a place to route the call. The employer decides which destinations can answer the same question on its behalf.

No extra charge for the router changes one line of the budget#

The launch says there is no additional charge for the routing feature. AWS continues charging standard endpoint prices and standard CloudWatch rates for router metrics. An organisation should put those lines beside capacity held in reserve, time to qualify models across regions, support and monitoring, and whatever other charges apply to its architecture. The business case improves if useful work can draw on capacity that would otherwise sit idle. It becomes weaker if the receiving region needs nearly as much extra reserve as a locally resilient design and the team has added substantial work for request classification and review. There is enough in the announcement to propose a budget comparison. It does not contain the company’s next bill.

Compare the cost of a successful response at an acceptable time in the two proposed arrangements. Record where traffic was sent during normal work, which calls were retried and what capacity was still needed locally. A finance owner can then trace the bill to the specific groups of requests that benefit. The local-only workload may need independent investment and should not be counted in a saving it cannot use. A shared reserve can pay off when distinct teams or regions use it at different times. If the peaks overlap, each may still have to reserve enough capacity for the same bad afternoon. The managed feature changes how it is used, while the customer still chooses how much to buy.

The operational budget should also include the person who can intervene. Switching to the provider’s managed logic may remove repeated work in several applications. It adds a need to know the provider’s reasons, monitoring and control points under pressure. A team that does not work through the tests first may simply trade writing application code for troubleshooting a more distant decision. If the people who own the application can see requests move, handle failure and keep customer promises, the work can genuinely shrink. A responsible buyer asks who covers the initial rollout, ongoing service review and continued ownership of the targets, then compares that with the current workload.

Define acceptable service during the slow period and after it#

The first useful commitment states which result can still be offered while an eligible request takes longer than expected. For a person interacting in a webpage, that might mean giving an honest status and an alternative action if the requested task cannot finish soon. For a scheduled process, the owner might be able to wait safely until the operation finishes or is clearly recorded as failed. These are different service designs around the same decision to use routing. A manager can test how long people wait, what happens to attempts that lost their connection, and whether the eventual response still belongs to the original request. A short recovery chart will not show those parts of the experience unless the application captures them.

The retry boundary also deserves attention. The case reports that the router can retry certain failures once when the model has not run, and that the Salesforce gateway avoids adding another set of retries in that path. An operator should understand which attempt was the one that served the result and whether callers add more attempts on their own. Repeating work can add pressure to a receiving region precisely when other regions are trying to use it. For a customer with strict timing or workload limits, the quality test is whether it gets a result without surprising duplication or queues that compound across services. That result depends on both the behaviour of the provider and the settings used by the application.

There are also reasons to include what recovery looks like in the contract between regions. When a primary endpoint appears healthy again, the managed service moves requests home gradually. Local users at a receiving endpoint may continue sharing capacity while the transfer reverses. They need to know whether normal traffic and costs return as expected, and who can pause a wider rollout if the next shift changes the pattern. A team can agree before an exercise who is watching the origin, who watches the receiving region and who tells the product team when the experiment is over. That is part of deploying the feature, even when the network behaviour does not require a person to move requests by hand.

The agreement should cover times when both endpoints are short of room. Each team can name the amount it is willing to share and the routine to follow when a limit is reached. A lower limit may protect local customers and leave fewer resources for the calling region. A higher allocation may improve one failure case and create a slower experience for others. Evaluate both and write down who approves a change. This prevents an apparent recovery at the originating application from hiding a cost at the receiving one. It also makes the decision reversible if usage patterns change after a new product is introduced.

A limited trial should include conditions in which the design has to refuse a move#

A well-framed pilot can start with one model and two endpoints that are already in approved regions. Use test data suitable for both, verify comparable responses and establish how much traffic either endpoint may receive. Simulate a loss of local service in a test environment and check where eligible calls go. Also run a local-only workload to check that errors do not lead to unauthorised destinations. An exercise should include the case in which no destination has enough capacity and the case in which an endpoint returns to service. Those are plausible states of a regional system. The point is to learn what choices the company can handle as well as see a successful redirect.

Before expanding, keep the elapsed time of degraded responses, failure rates and recovery visible along with the effects on the destination’s normal callers. Attach the conditions to the report so a director can explain which promise has been tested. The owner should know how to pause broader routing and what the customer sees if there is no acceptable target. The new feature is worth a trial for a group with uneven load and several viable deployments. I would change that view if the trial showed repeated slowdowns before failover, pressure on people in the receiving region or inability to prove where a protected call stayed. Those outcomes matter more than a successful connection in one demonstration.

The Salesforce test and AWS release make a case for treating spare regional capacity as something a company can deliberately share. They also show that the primary decisions are visible before and after the router receives a request. A company approves the destinations and categories, people provision the endpoints and someone verifies the trace after a shift. Combining those responsibilities with a managed service can make operations simpler. It succeeds at a broader deployment when the organisation has evidence that requests actually keep the promises it made about where they may go and how soon they need a result.