Functional retry handling with cats-retry in Scala

In a production environment, the application sometimes fails because of network connection timeout; quota exceeded calling External APIs and so on.
In this case, by retrying the action after a while, the application might succeed. Implementing a retry mechanism tends to be complicated as it requires many things; how many times the program retries the action, how long the program waits between attempts, how to output the log during retrying and so on.

In this article, I will explain how to implement a functional retry mechanism in Scala.

Simple retry handling

First of all, below is the simple program with a retry mechanism in Scala. This program retries up to three times and waits ten milliseconds between each attempt.

Sample code with retry mechanism

If this program fails three times, the output would be as followed.

Error has occurred on program function, retry[counter=0] after 10 milliseconds [ms] sleeping...
Error has occurred on program function, retry[counter=1] after 10 milliseconds [ms] sleeping...
Error has occurred on program function, retry[counter=2] after 10 milliseconds [ms] sleeping...
Giving up on program function, after 3 retries, finally total delay was 30 milliseconds [ms]
Unexpected error occurred.

Make the program abstraction

In the above code, the ‘runRetry’ method is defined in the ‘program’ method, and the ‘runRetry’ method depends on the ‘execute’ method. If you would like to implement another program with a retry mechanism, you would have to write a similar code, which is inconvenient. Therefore, abstraction is a better option.
Next is the ‘retryFunc’ method, which is transformed into abstraction.
This function enables us to set maximum retry count, delay algorithm, the method of your choice and side effects which have a MonadError type class instance.

Common Retry Function

When one implements the program with a retry mechanism, input method name, child class inherited ‘RetryPolicy’ trait and ‘execute’ method. If one would want to use another delay algorithm such as Fibonacci-Backoff, by defining a new child class of ‘RetryPolicy’ trait, you can do so. In this instance, there is no need to change ‘retryFunc’ method. While this can make the logging abstraction the same way ‘RetryPolicy’ trait.

def program(input: Input): IO[Output] = 
  retryFunc[IO, Output]("program method")(ExponentialBackOff(3, 1.second))(execute(input))
program(input).handleErrorWith(
  t => IO(println(t.getMessage))
).unsafeRunSync()

I think this retry mechanism is useful, on the other hand, cats-retry provides more powerful retry mechanism.

Advanced retry handling with cats-retry

cats-retry is a Scala library for retrying actions that can fail. There is the Getting Started page, which allows you to begin immediately.

Let’s rewrite the program with cats-retry.

The ‘retryingOnAllErrors’ method which is one of the combinators provided by cats-retry is suited for the program. This method takes the ‘RetryPolicy’, ‘onError’ and ‘action’ as arguments.

def retryingOnAllErrors[A] = new RetryingOnAllErrorsPartiallyApplied[A]

private[retry] class RetryingOnAllErrorsPartiallyApplied[A] {
  def apply[M[_], E](
      policy: RetryPolicy[M],
      onError: (E, RetryDetails) => M[Unit]
  )(
      action: => M[A]
  )(
      implicit
      ME: MonadError[M, E],
      S: Sleep[M]
  ): M[A]

‘RetryPolicy’ enables us to define maximum retry count and delay algorithm. In the code below, retry up to three times with Exponential-Backoff delay between attempts. cats-retry provides the other delay algorithms; no delay, constant delay, Fibonacci-Backoff and etc. And you can define a custom retry policy. Please have a look at the Retry Policy page.

import cats.syntax.semigroup._
val policy = 
  RetryPolicies.limitRetries[IO](3) |+|  
  RetryPolicies.exponentialBackoff[IO](10.milliseconds)

‘onError’ enables us to define how to handle the error. Usually, This method is used for logging. In this code below, logging is defined for when the program attempts to retry and gives up after retry maximum times defined.

def logError(action: String)(err: Throwable, details: RetryDetails): IO[Unit] = details match {
  case WillDelayAndRetry(nextDelay: FiniteDuration,
      retriesSoFar: Int,
      cumulativeDelay: FiniteDuration) =>
    IO {
      logger.info(
        s"Error has occurred on $action, retry[counter=$retriesSoFar] after ${nextDelay.toMillis} [ms] sleeping..., total delay was ${cumulativeDelay.toMillis} [ms] so far")
    }

  case GivingUp(totalRetries: Int, totalDelay: FiniteDuration) =>
    IO {
      logger.info(s"Giving up on $action after $totalRetries retries, finally total delay was ${totalDelay.toMillis} [ms]")
    }
}

Finally, by inputting ‘policy’ variable, ‘onError’ method and ‘execute’ method, this program can have retry mechanism.

def program(input: Input): IO[Output] = {
  retryingOnAllErrors[Output](
    policy = policy,
    onError = logError("program method")
  )(execute(input))
}
program(input).handleErrorWith (
  t => IO(println(t.getMessage))
).unsafeRunSync()

If this program fails 3 times, the output would be as follow.

Error has occurred on program method, retry[counter=0] after 1000 [ms] sleeping..., total delay was 0 [ms] so far
Error has occurred on program method, retry[counter=1] after 2000 [ms] sleeping..., total delay was 1000 [ms] so far
Error has occurred on program method, retry[counter=2] after 4000 [ms] sleeping..., total delay was 3000 [ms] so far
Giving up on program after 3 retries, finally total delay was 7000 [ms]
Unexpected error occurred.

That is it! cats-retry is very easy to use and flexible.
Furthermore, as it is written in the official web page, cats-retry is defined to work with cats and cats-effect or Monix. So if someone uses these libraries, they should try cats-retry! It might be helpful for them.

Thank you for reading.