Graceful Shutdown of Spring Boot Applications in Kubernetes: SIGTERM, preStopHook, and Actuator

Why is Graceful Shutdown Important?

In modern application development using microservices architecture and container environments, especially Kubernetes, "application termination" means more than just shutting down a process. Abrupt termination can lead to serious problems like data loss, service inconsistencies, and degraded user experience.

Imagine a user sending a payment request, and the server Pod processing that request suddenly terminates.

To prevent these issues, applications must undergo a Graceful Shutdown process, essentially announcing: "I'm about to shut down, so I'll finish what I'm doing and clean up neatly!"

Understanding Kubernetes Pod Termination Lifecycle

Kubernetes follows a defined sequence when terminating a Pod. Understanding this process is key to implementing Graceful Shutdown.

1. Pod Termination Request and Transition to Terminating State: Kubernetes decides a Pod needs to be terminated (e.g., due to kubectl delete pod, deployment updates, or scaling down). The Pod changes from Running to Terminating status.
2. Removal of Pod IP from Service Endpoints: Almost simultaneously with the Pod entering the Terminating state, the Kubernetes control plane begins removing the Pod's IP address from the endpoint list of all relevant Services. This is crucial! New traffic will now no longer be routed to this Pod.
3. preStopHook Execution (Optional): If a preStopHook is defined in the Pod specification, this hook executes before the SIGTERM signal is sent to the container.
   • preStopHook can be an exec (executing a command inside the container) or httpGet (sending an HTTP request).
   • Key Point: Kubernetes waits for the preStopHook to complete before sending SIGTERM.
   • The terminationGracePeriodSeconds countdown begins at this point.
4. SIGTERM Signal Sent: Immediately after the preStopHook successfully completes, or if no preStopHook is defined, as soon as the Pod enters the Terminating state and is removed from endpoints, Kubernetes sends a SIGTERM signal to the container's main process (PID 1).
   • SIGTERM is a soft termination signal, asking the application to "wrap up what you're doing and shut down gracefully."
5. terminationGracePeriodSeconds Wait: From the moment SIGTERM is sent, Kubernetes waits for the duration specified by terminationGracePeriodSeconds for the application to shut down on its own. During this period, the application should complete ongoing requests, close database connections, release resources, etc.
6. SIGKILL Signal Sent (Last Resort): If the application does not terminate within the terminationGracePeriodSeconds, Kubernetes stops waiting and forcibly sends a SIGKILL signal. SIGKILL immediately terminates the process in any situation, risking data loss or resource leaks because the application cannot complete its shutdown tasks.

Graceful Shutdown in Spring Boot: server.shutdown=graceful

Spring Boot has built-in functionality to gracefully shut down when it receives a SIGTERM signal. Since Spring Boot 2.3, you can activate this with a simple configuration.

Add the following to application.yml or application.properties:

server:
  shutdown: graceful # Enable graceful shutdown
spring:
  lifecycle:
    timeout-per-shutdown-phase: 30s # Optional: Max wait time per shutdown phase

With this setting, when a Spring Boot application receives SIGTERM:

• It immediately stops accepting new requests.
• It waits for currently processing requests to complete.
• Once all requests are processed, it performs cleanup tasks such as shutting down Tomcat (or other web servers) and closing the Spring context.

When is preStopHook Useful? (Primarily for sleep)

Even with the server.shutdown=graceful setting, Spring Boot applications will gracefully shut down upon receiving SIGTERM. However, using a preStopHook enhances the certainty of Traffic Draining.

It's common to use a short sleep command in preStopHook.

apiVersion: apps/v1
kind: Deployment
# ... other deployment configurations ...
spec:
  template:
    spec:
      containers:
      - name: my-app-container
        image: your-docker-image:latest
        lifecycle:
          preStop:
            exec:
              command: ["/bin/sh", "-c", "sleep 10"] # Wait for 10 seconds
        terminationGracePeriodSeconds: 60 # Sufficient time for app to shut down (e.g., 60 seconds)

Why use sleep?

Mitigate Load Balancer/Service Mesh Synchronization Delays: Even when Kubernetes removes a Pod's IP from endpoints, it can take a few seconds for external load balancers or service meshes (like Istio, Nginx Ingress) to detect this change and actually stop routing traffic to the Pod.

The sleep in preStopHook explicitly ensures this "detection delay" time is compensated, providing enough time for all new traffic to be certainly blocked from reaching the Pod before SIGTERM arrives.

How long should the sleep be? It's important to keep it short, generally recommended between 5 to 15 seconds. Too long will unnecessarily increase deployment time and reduce the time available for SIGTERM processing, increasing the risk of SIGKILL.

Role and Security of Actuator Shutdown (/actuator/shutdown)

Spring Boot Actuator's /actuator/shutdown endpoint explicitly commands the application to "shut down now." You can call this endpoint with curl or similar in a preStopHook to initiate shutdown.

# Example of calling Actuator shutdown in preStopHook (Security Warning!)
lifecycle:
  preStop:
    exec:
      command: ["/bin/sh", "-c", "curl -X POST -u ${ACTUATOR_USERNAME}:${ACTUATOR_PASSWORD} http://localhost:8080/actuator/shutdown"]

Important: Security is a Must!

The /actuator/shutdown endpoint is highly sensitive and must never be exposed externally without proper security configurations! Use Spring Security to restrict access and securely pass authentication information (e.g., using Kubernetes Secrets) when calling it from preStopHook.

When to use this method? This approach might be considered in specific scenarios where fine-grained control is difficult with only SIGTERM processing, or if the application needs to initiate shutdown upon receiving an explicit external signal. However, considering the complexity of security configuration and potential risks, the server.shutdown=graceful and SIGTERM handling approach is generally more recommended.

Final Recommendations and Best Practices

For graceful shutdown of Spring Boot applications in Kubernetes, the best practices are as follows:

By adopting this approach, your Spring Boot applications in a Kubernetes environment will be more stable, predictable, and capable of shutting down gracefully without data loss. This ultimately leads to a more robust and reliable microservices architecture.