Memoryleak

Hi @sergei.terentev
We tried below options:

livenessProbe:
            httpGet:
              path: /
              port: 80
            initialDelaySeconds: 10
            periodSeconds: 30
            failureThreshold: 3

# Env

env:
  - name: DOTNET_GCConserveMemory
    value: "9"

This caused more restarts with OOM. What might be the issue ?

Thanks for the update — sorry to hear the restarts got worse, that’s not what we expected.

To figure out what’s going on, a few questions:

Do you have a memory limit set on the pod? Something like resources.limits.memory: "1Gi" in your deployment spec? If so, what’s the value? If not, do you know roughly how much memory the node has available?

What does kubectl describe pod <pod-name> show for the last restart? Specifically the Last State section under the container — it should say whether the exit code was OOMKilled or something else.

What’s the typical workload? Rough doc sizes and how many concurrent requests you’re sending would help a lot.

My suspicion is that the memory limit might be set too low for the conversion workload, and the more aggressive GC is actually exposing that by being less tolerant of spikes. But I’d rather confirm before suggesting a fix.

Hi @sergei.terentev
Here is the detail:

  1. Pod Resource config and last state
  State:          Running
      Started:      Thu, 25 Jun 2026 10:21:26 +0530
    Last State:     Terminated
      Reason:       OOMKilled
      Exit Code:    137
      Started:      Thu, 25 Jun 2026 07:44:30 +0530
      Finished:     Thu, 25 Jun 2026 10:21:24 +0530
    Ready:          True
    Restart Count:  7
    Limits:
      cpu:     8
      memory:  5Gi
    Requests:
      cpu:     1
      memory:  1Gi
  1. We have a document size limit 400Mb.
  2. And there is not much concurrency, in last 24hrs from now we have got ~ 2K requests.

@piyush.rajput ,
Thanks, this is helpful.

The issue is likely DOTNET_GCConserveMemory=9 working against you. A 400MB source file can expand to 1–2GB in working memory during conversion, and with the GC running aggressively on top of that, you’re hitting the 5Gi ceiling.

Try dropping it to 5 instead of 9:

env:
  - name: DOTNET_GCConserveMemory
    value: "5"

Also, your liveness probe has no timeoutSeconds set (defaults to 1s) — under load that can cause false restarts. Add timeoutSeconds: 5 to be safe.

Also, in addition, from our’s cloud version experience, the multiple restarts are normal, we have hundreds of them. But also we have multiple instances (pods), so load is distributed between them - when some are being restarted, other continue handle requests.

Thank you @sergei.terentev for the quick reply.
Will try your suggestion for

DOTNET_GCConserveMemory = 5

We are running 20 Pods in production.

Hi @sergei.terentev ,
Deployed above suggestions, but we are still seeing restarts, here is the pod state:

 State:          Running
      Started:      Sun, 28 Jun 2026 23:43:21 +0530
    Last State:     Terminated
      Reason:       OOMKilled
      Exit Code:    137
      Started:      Sun, 28 Jun 2026 23:42:17 +0530
      Finished:     Sun, 28 Jun 2026 23:43:08 +0530
    Ready:          True
    Restart Count:  2
    Limits:
      cpu:     8
      memory:  5Gi
    Requests:
      cpu:     1
      memory:  1Gi
    Liveness:  http-get http://:80/ delay=10s timeout=5s period=30s #success=1 #failure=3
    Environment:
      LICENSE_PUBLIC_KEY:       <set to the key 'LICENSE_PUBLIC_KEY' in secret 'groupdocs-secrets'>   Optional: false
      LICENSE_PRIVATE_KEY:      <set to the key 'LICENSE_PRIVATE_KEY' in secret 'groupdocs-secrets'>  Optional: false
      DOTNET_GCConserveMemory:  5

Hi, @piyush.rajput

Thank you for the details. You are correct that an OOMKilled event will interrupt any conversion request that is in progress at that moment — the client will receive a connection error and would need to retry.

To handle this more gracefully, we are releasing an update in the next couple of days that introduces proactive memory monitoring. Once deployed, the service will detect when memory usage approaches the container limit and report itself as unhealthy via the existing liveness probe endpoint (GET /). Kubernetes will then trigger a graceful pod restart before the hard OOM kill occurs, allowing in-flight requests to complete first.

The behavior is controlled by a new MEMORY_THRESHOLD environment variable (a decimal fraction, default 0.85). For example, with your current 5Gi limit the health check will start reporting unhealthy above ~4.25 GB, giving Kubernetes time to drain and replace the pod cleanly.

We will notify you as soon as the updated image is available.

1 Like

Hi @sergei.terentev
Thank you for the update.
Any tentative timeline for the release of that update ?

Hi, @piyush.rajput,
The update is just published at DockerHub, please use latest or 26.4.2 tag. Refer " Kubernetes memory pressure and graceful restart" section in readme at DockerHub, for enabling memory threshold feature.

Thanks for the update @sergei.terentev
We will try this and update you.

HI @sergei.terentev
Deployed the new image, but pods are still restarting.
Pod last state:

 Last State:     Terminated
      Reason:       OOMKilled
      Exit Code:    137
      Started:      Thu, 02 Jul 2026 17:58:39 +0530
      Finished:     Thu, 02 Jul 2026 18:01:21 +0530
    Ready:          True
    Restart Count:  1
    Limits:
      cpu:     8
      memory:  12Gi
    Requests:
      cpu:     1
      memory:  1Gi
    Liveness:  http-get http://:80/ delay=10s timeout=5s period=30s #success=1 #failure=3
    Environment:
      LICENSE_PUBLIC_KEY:       <set to the key 'LICENSE_PUBLIC_KEY' in secret 'groupdocs-secrets'>   Optional: false
      LICENSE_PRIVATE_KEY:      <set to the key 'LICENSE_PRIVATE_KEY' in secret 'groupdocs-secrets'>  Optional: false
      DOTNET_GCConserveMemory:  5

@piyush.rajput , have you set MEMORY_THRESHOLD parameter to 0.8?

Hi @sergei.terentev
No, did not set this parameter.
Let me set in env and try.

Hi @sergei.terentev
Pod still restarting

 State:          Running
      Started:      Mon, 06 Jul 2026 10:03:47 +0530
    Last State:     Terminated
      Reason:       OOMKilled
      Exit Code:    137
      Started:      Sun, 05 Jul 2026 16:08:33 +0530
      Finished:     Mon, 06 Jul 2026 10:03:47 +0530
    Ready:          True
    Restart Count:  1
    Limits:
      cpu:     8
      memory:  12Gi
    Requests:
      cpu:     1
      memory:  1Gi
    Liveness:  http-get http://:80/ delay=10s timeout=5s period=30s #success=1 #failure=3
    Environment:
      LICENSE_PUBLIC_KEY:       <set to the key 'LICENSE_PUBLIC_KEY' in secret 'groupdocs-secrets'>   Optional: false
      LICENSE_PRIVATE_KEY:      <set to the key 'LICENSE_PRIVATE_KEY' in secret 'groupdocs-secrets'>  Optional: false
      DOTNET_GCConserveMemory:  5
      MEMORY_THRESHOLD:         0.8

Hi, @piyush.rajput , got it. We will contunue search for solution. Meantime, you may also take a look on the CONVERSIONS_LIMIT parameter. It also makes the container unhealthy after reaching the conversions limit, and triggers graceful restart.

1 Like

Hi, @piyush.rajput !

We identified a bug in the initial implementation of the MEMORY_THRESHOLD feature: the memory check was measuring only the .NET managed heap (GC.GetGCMemoryInfo().TotalCommittedBytes), while the conversion libraries also allocate native (unmanaged) memory that is invisible to the GC. This is why the health check never reported pressure and the pod was killed without warning even with MEMORY_THRESHOLD=0.8 set.

We have fixed this — the check now uses the total process working set (managed + native memory), which accurately reflects what Kubernetes sees against the container limit. An updated image will be available within the next couple of days.

In the meantime, as a workaround, you can reduce DOTNET_GCConserveMemory to 3 or remove it entirely — the current value of 5 makes the GC hold on to more managed memory than necessary, which leaves less headroom before the native allocations push the total over the limit.

The update is just published at DockerHub, please use latest or 26.4.3 tag.

Hi @sergei.terentev
Thank you for the update.
I have deployed the latest image with removing env variable DOTNET_GCConserveMemory.
We will monitor and update you on this.

Hi @sergei.terentev ,
Seems some improvements in memory usage in the latest image but does not seem to be resolved. Check the attached screenshot for reference. It did not restart but memory has spiked and staying at the top.
Screenshot 2026-07-16 at 9.54.14 AM.png (6.6 KB)

Hi, @piyush.rajput , Thank you for details. We continue analysis and search solution.

Hi @sergei.terentev ,
Today pods restarted many times with OOM exception.