Retry

Default Retry Strategy

See here for information on the settings used for the default retry strategy.

oci.retry.DEFAULT_RETRY_STRATEGY = <oci.retry.retry.ExponentialBackoffWithFullJitterEqualForThrottlesRetryStrategy object>

A retry strategy that does exponential backoff and full jitter for most retries, but uses exponential backoff with equal jitter for throttles. This provides a reasonable distribution of retry times for most retryable error cases, but for throttles guarantees some sleep time

Retry

class oci.retry.RetryStrategyBuilder(**kwargs)

A class which can build a retry strategy based on provided criteria. Criteria can be provided at construction time or afterwards via using the add_* (to add/enable criteria) and no_* (to disable/remove criteria) methods.

Trying to build a strategy when there are no enabled checks will result in a oci.retry.NoneRetryStrategy being produced.

This builder is intended as a convenience, but callers are also able to bypass this and construct retry strategies directly.

__init__(**kwargs)

Creates a new builder and initializes it based on any provided parameters.

Parameters:
  • max_attempts_check (optional) (Boolean) -- Whether to enable a check that we don't exceed a certain number of attempts. If not provided this defaults to False (i.e. this check will not be done)
  • service_error_check (optional) (Boolean) -- Whether to enable a check that will retry on connection errors, timeouts and service errors which match given combinations of HTTP statuses and textual error codes. If not provided this defaults to False (i.e. this check will not be done)
  • total_elapsed_time_check (optional) (Boolean) -- Whether to enable a check that we don't exceed a certain amount of time when retrying. This is intended for scenarios such as "keep retrying for 5 minutes". If not provided this defaults to False (i.e. this check will not be done)
  • max_atttemps (optional) (int) -- If we are checking that we don't exceed a certain number of attempts, what that number of attempts should be. This only applies if we are performing a check on the maximum number of attempts and will be ignored otherwise. If we are performing a check on the maximum number of attempts and this value is not provided, we will default to a maximum of 5 attempts
  • total_elapsed_time_seconds (optional) (int) -- If we are checking that we don't exceed a certain amount of time when retrying, what that amount of time should be (in seconds). This only applies if we are perfoming a check on the total elapsed time and will be ignored otherwise. If we are performing a check on the total elapsed time and this value is not provided, we will default to 300 seconds (5 minutes)
  • service_error_retry_config (optional) (dict) --

    If we are checking on service errors, we can configure what HTTP statuses (e.g. 429) to retry on and, optionally, whether the textual code (e.g. TooManyRequests) matches a given value.

    This is a dictionary where the key is an integer representing the HTTP status, and the value is a list(str) where we will test if the textual code in the service error is a member of the list. If an empty list is provided, then only the numeric status is checked for retry purposes.

    If we are performing a check on service errors and this value is not provided, then by default we will retry on HTTP 429's (throttles) without any textual code check.

  • service_error_retry_on_any_5xx (optional) (Boolean) -- If we are checking on service errors, whether to retry on any HTTP 5xx received from the service. If we are performing a check on service errors and this value is not provided, it defaults to True (retry on any 5xx)
  • retry_base_sleep_time_seconds (optional) (int) -- For exponential backoff with jitter, the base time to use in our retry calculation in seconds. If not provided, this value defaults to 1 second
  • retry_exponential_growth_factor (optional) (int) -- For exponential backoff with jitter, the exponent which we will raise to the power of the number of attempts. If not provided, this value defaults to 2
  • retry_max_wait_between_calls_seconds (optional) (int) -- For exponential backoff with jitter, the maximum amount of time to wait between retries. If not provided, this value defaults to 30 seconds
  • backoff_type (optional) (str) -- The type of backoff we want to do (e.g. full jitter). The convenience constants in this retry module: BACKOFF_FULL_JITTER_VALUE, BACKOFF_EQUAL_JITTER_VALUE, and BACKOFF_FULL_JITTER_EQUAL_ON_THROTTLE_VALUE can be used as values here. If no value is specified then the value BACKOFF_FULL_JITTER_EQUAL_ON_THROTTLE_VALUE will be used. This will use exponential backoff and full jitter on all retries except for throttles, in which case it will use exponential backoff and equal jitter.
add_max_attempts(max_attempts=None)
add_service_error_check(**kwargs)
add_total_elapsed_time(total_elapsed_time_seconds=None)
get_retry_strategy()
no_max_attemps()
no_service_error_check()
no_total_elapsed_time()
class oci.retry.NoneRetryStrategy

A strategy that does not retry

__init__

Initialize self. See help(type(self)) for accurate signature.

make_retrying_call(func_ref, *func_args, **func_kwargs)

Calls the function given by func_ref. Any positional (*func_args) and keyword (**func_kwargs) arguments are passed as-is to func_ref.

Parameters:func_ref (function) -- The function that we should call with retries
Returns:the result of calling func_ref
class oci.retry.ExponentialBackoffRetryStrategyBase(base_sleep_time_seconds, exponent_growth_factor, max_wait_between_calls_seconds, checker_container, **kwargs)

A base retry strategy from which other retry strategies inherit. Implementors can create a subclass of this to define their own retry logic. This is primarily useful when an implementor wishes to customize the sleep strategy used - to customize the checking logic on whether a retry should be done, implementors can define their own subclasses of oci.retry.retry_checkers.BaseRetryChecker and provide this in the checker container.

__init__(base_sleep_time_seconds, exponent_growth_factor, max_wait_between_calls_seconds, checker_container, **kwargs)

Creates a new instance of an exponential backoff with full jitter retry strategy.

Parameters:
  • base_sleep_time_seconds (int) -- The base amount to sleep by, in seconds
  • exponent_growth_factor (int) -- The exponent part of our backoff. We will raise take this value and raising it to the power of attemps and then multiply this with base_sleep_time_seconds
  • max_wait_between_calls_seconds (int) -- The maximum time we will wait between calls, in seconds
  • checker_container (retry_checkers.RetryCheckerContainer) -- The checks to run to determine whether a failed call should be retried
do_sleep(attempt, exception)
make_retrying_call(func_ref, *func_args, **func_kwargs)

Calls the function given by func_ref. Any positional (*func_args) and keyword (**func_kwargs) arguments are passed as-is to func_ref.

Parameters:func_ref (function) -- The function that we should call with retries
Returns:the result of calling func_ref
class oci.retry.ExponentialBackoffWithFullJitterRetryStrategy(base_sleep_time_seconds, exponent_growth_factor, max_wait_between_calls_seconds, checker_container, **kwargs)

A retry strategy which does exponential backoff and full jitter. Times used are in milliseconds and the strategy can be described as:

random(0, min(base_sleep_time_seconds * exponent_growth_factor ** (attempt), max_wait_seconds))
__init__(base_sleep_time_seconds, exponent_growth_factor, max_wait_between_calls_seconds, checker_container, **kwargs)

Creates a new instance of an exponential backoff with full jitter retry strategy.

Parameters:
  • base_sleep_time_seconds (int) -- The base amount to sleep by, in seconds
  • exponent_growth_factor (int) -- The exponent part of our backoff. We will raise take this value and raising it to the power of attemps and then multiply this with base_sleep_time_seconds
  • max_wait_between_calls_seconds (int) -- The maximum time we will wait between calls, in seconds
  • checker_container (retry_checkers.RetryCheckerContainer) -- The checks to run to determine whether a failed call should be retried
do_sleep(attempt, exception)
make_retrying_call(func_ref, *func_args, **func_kwargs)

Calls the function given by func_ref. Any positional (*func_args) and keyword (**func_kwargs) arguments are passed as-is to func_ref.

Parameters:func_ref (function) -- The function that we should call with retries
Returns:the result of calling func_ref
class oci.retry.ExponentialBackoffWithEqualJitterRetryStrategy(base_sleep_time_seconds, exponent_growth_factor, max_wait_between_calls_seconds, checker_container, **kwargs)

A retry strategy which does exponential backoff and equal jitter. Times used are in milliseconds and the strategy can be described as:

exponential_backoff_sleep = min(base_sleep_time_seconds * exponent_growth_factor ** (attempt), max_wait_seconds)
sleep_with_jitter = (exponential_backoff_sleep / 2) + random(0, exponential_backoff_sleep / 2)
__init__(base_sleep_time_seconds, exponent_growth_factor, max_wait_between_calls_seconds, checker_container, **kwargs)

Creates a new instance of an exponential backoff with equal jitter retry strategy.

Parameters:
  • base_sleep_time_seconds (int) -- The base amount to sleep by, in seconds
  • exponent_growth_factor (int) -- The exponent part of our backoff. We will raise take this value and raising it to the power of attemps and then multiply this with base_sleep_time_seconds
  • max_wait_between_calls_seconds (int) -- The maximum time we will wait between calls, in seconds
  • checker_container (retry_checkers.RetryCheckerContainer) -- The checks to run to determine whether a failed call should be retried
do_sleep(attempt, exception)
make_retrying_call(func_ref, *func_args, **func_kwargs)

Calls the function given by func_ref. Any positional (*func_args) and keyword (**func_kwargs) arguments are passed as-is to func_ref.

Parameters:func_ref (function) -- The function that we should call with retries
Returns:the result of calling func_ref
class oci.retry.ExponentialBackoffWithFullJitterEqualForThrottlesRetryStrategy(base_sleep_time_seconds, exponent_growth_factor, max_wait_between_calls_seconds, checker_container, **kwargs)

A retry strategy that does exponential backoff and full jitter for most retries, but uses exponential backoff with equal jitter for throttles. This provides a reasonable distribution of retry times for most retryable error cases, but for throttles guarantees some sleep time

__init__(base_sleep_time_seconds, exponent_growth_factor, max_wait_between_calls_seconds, checker_container, **kwargs)

Creates a new instance of the retry strategy.

Parameters:
  • base_sleep_time_seconds (int) -- The base amount to sleep by, in seconds
  • exponent_growth_factor (int) -- The exponent part of our backoff. We will raise take this value and raising it to the power of attemps and then multiply this with base_sleep_time_seconds
  • max_wait_between_calls_seconds (int) -- The maximum time we will wait between calls, in seconds
  • checker_container (retry_checkers.RetryCheckerContainer) -- The checks to run to determine whether a failed call should be retried
do_sleep(attempt, exception)
make_retrying_call(func_ref, *func_args, **func_kwargs)

Calls the function given by func_ref. Any positional (*func_args) and keyword (**func_kwargs) arguments are passed as-is to func_ref.

Parameters:func_ref (function) -- The function that we should call with retries
Returns:the result of calling func_ref
oci.retry.DEFAULT_RETRY_STRATEGY = <oci.retry.retry.ExponentialBackoffWithFullJitterEqualForThrottlesRetryStrategy object>

A retry strategy which has all options enabled and which will use the default settings for those options. These defaults are:

  • 5 total attempts
  • Total allowed elapsed time for all requests of 300 seconds (5 minutes)
  • Exponential backoff with jitter using a base time of 1 second, an exponent of 2 and a maximum wait time between calls of 30 seconds
  • Exponential backoff with equal jitter is used for throttles as this guarantees some sleep time between attempts
  • Exponential backoff with full jitter is used for other scenarios where we need to retry (e.g. timeouts, HTTP 5xx)
  • Retries on the following exception types: timeouts and connection errors, HTTP 429s (throttles), any HTTP 5xx

Retry Checkers

class oci.retry.retry_checkers.BaseRetryChecker(**kwargs)

The base class from which all retry checkers should derive. This has no implementation but just defines the contract for a checker. Implementors can extend this class to define their own checking logic.

__init__(**kwargs)
should_retry(exception=None, response=None, **kwargs)

Determines if a retry should be performed based on either an exception or a response.

Parameters:
  • exception (Exception) -- An exception received from the service
  • response (Response) -- The Response received from a service call
Returns:

True if we should retry, and False otherwise

Return type:

Boolean

class oci.retry.retry_checkers.LimitBasedRetryChecker(max_attempts=5, **kwargs)

A retry checker which can retry as long as some threshold (# of attempts/tries) has not been breached. It is the responsiblity of the caller to track how many attempts/tries it has done - objects of this class will not track this.

If not specified, the default number of tries allowed is 5. Tries are also assumed to be one-based (i.e. the first attempt/try is 1, the second is 2 etc)

__init__(max_attempts=5, **kwargs)
should_retry(exception=None, response=None, **kwargs)
class oci.retry.retry_checkers.RetryCheckerContainer(checkers, **kwargs)

A container which holds at least one retry checker. This lets us chain together different retry checkers into an overall evaluation of whether we should retry a request.

Checkers are evaluated in the order they appear in the provided list of checkers, and if one checker reports failure we consider this to be an overall failure and no more retries should happen.

__init__(checkers, **kwargs)
add_checker(checker)
should_retry(exception=None, response=None, **kwargs)

Determines if a retry should be performed based on either an exception or a response. We will retry if all the checkers held in this container indicate that they should retry; if any checker indicates that the call should not be retried then we will not retry.

Parameters:
  • exception (Exception) -- An exception received from the service
  • response (Response) -- The Response received from a service call
Returns:

True if we should retry, and False otherwise

Return type:

Boolean

class oci.retry.retry_checkers.TimeoutConnectionAndServiceErrorRetryChecker(service_error_retry_config={429: [], -1: []}, retry_any_5xx=True, **kwargs)

A checker which will retry on certain exceptions. Retries are enabled for the following exception types:

  • Timeouts from the requests library (we will always retry on these)
  • ConnectionErrors from the requests library (we will always retry on these)
  • Service errors where the status is 500 or above (i.e. a server-side error)
  • Service errors where a status (e.g. 429) and, optionally, the code meet a given criteria

The last item is configurable via dictionary where the key is some numeric status representing a HTTP status and the value is a list of strings with each string representing a textual error code (such as those error codes documented at https://docs.cloud.oracle.com/Content/API/References/apierrors.htm). If an empty list is provided, then only the numeric status is checked for retry purposes. For a populated array, we are looking for where the numeric status matches and the code from the exception appears in the array. As an example:

{
    400: ['QuotaExceeded'],
    500: []
}

If no configuration is provided, then the default for service errors is to retry on HTTP 429's and 5xx's without any code checks. If a specific 5xx code (e.g. 500, 502) is provided in the dictionary then it takes precedence over the option to retry on any 500. For example it is possible to retry on only 502s (either by status or by status and matching some code ) by disabling the general "retry on any 5xx" configuration and placing an entry for 502 in the dictionary

RETRYABLE_STATUSES_AND_CODES = {429: [], -1: []}
__init__(service_error_retry_config={429: [], -1: []}, retry_any_5xx=True, **kwargs)
should_retry(exception=None, response=None, **kwargs)
class oci.retry.retry_checkers.TotalTimeExceededRetryChecker(time_limit_seconds=300, **kwargs)

A retry checker which can retry as long as some upper time limit (in seconds) has not been breached. This is intended for scenarios such as "keep retrying for 5 minutes". It is the responsiblity of the caller to track the total time elapsed - objects of this class will not track this.

If not specified, the default time limit is 300 seconds (5 minutes).

__init__(time_limit_seconds=300, **kwargs)
should_retry(exception=None, response=None, **kwargs)