From 585e56a2337c81b21dadf4d3209320024e3aed36 Mon Sep 17 00:00:00 2001 From: Ayan Alam Date: Sat, 3 Oct 2026 17:09:51 +0530 Subject: [PATCH 01/23] docs(store): document the auto-sized RocksDB memory budget The Store fix for apache/hugegraph#3254 ships rocksdb.total_memory_size as 0 and sizes the budget from the memory left after the heap, direct memory and a 1.5 GiB reserve, starting from the cgroup limit in a container. The HStore page still listed the old 32000000000 value and said 0 meant the JVM max heap. Update the rocksdb.total_memory_size row with the new rule, note the minimum of 1 GB in the raft.disruptorBufferSize derivation, and add HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE to the Store env var tables in the HStore quick start and the Docker cluster guide, in English and Chinese. --- content/cn/docs/guides/hugegraph-docker-cluster.md | 3 +++ content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md | 5 +++-- content/en/docs/guides/hugegraph-docker-cluster.md | 3 +++ content/en/docs/quickstart/hugegraph/hugegraph-hstore.md | 5 +++-- 4 files changed, 12 insertions(+), 4 deletions(-) diff --git a/content/cn/docs/guides/hugegraph-docker-cluster.md b/content/cn/docs/guides/hugegraph-docker-cluster.md index 49722ff265..d61ff1aa59 100644 --- a/content/cn/docs/guides/hugegraph-docker-cluster.md +++ b/content/cn/docs/guides/hugegraph-docker-cluster.md @@ -195,9 +195,12 @@ curl -o /dev/null -w '%{http_code}\n' -u "admin:${HUGEGRAPH_ADMIN_PASSWORD}" \ | `HG_STORE_GRPC_PORT` | 否 | `8500` | `grpc.port` | | `HG_STORE_REST_PORT` | 否 | `8520` | `server.port` | | `HG_STORE_DATA_PATH` | 否 | `/hugegraph-store/storage` | `app.data-path` | +| `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` | 否 | 无 | `rocksdb.total_memory_size`,单位字节 | > **已弃用的别名**:`PD_ADDRESS` → `HG_STORE_PD_ADDRESS`、`GRPC_HOST` → `HG_STORE_GRPC_HOST`、`RAFT_ADDRESS` → `HG_STORE_RAFT_ADDRESS` +RocksDB 缓存位于 Java 堆外。不设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` 时按容器内存限制自动计算,也可以设为纯字节数;详见 [HStore 配置参考](/cn/docs/quickstart/hugegraph/hugegraph-hstore/)中的 `rocksdb.total_memory_size`。 + ### Server 变量 与 PD、Store 不同,Server 入口脚本没有必填变量:只有实际设置了的变量才会被写入配置文件。但分布式部署至少需要 `HG_SERVER_BACKEND` 和 `HG_SERVER_PD_PEERS`。 diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index 17a3dcb643..59c81f96b2 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -131,6 +131,7 @@ docker run -d \ | `HG_STORE_GRPC_PORT` | 否 | `8500` | `grpc.port` | gRPC 服务端口。 | | `HG_STORE_REST_PORT` | 否 | `8520` | `server.port` | REST API 端口。 | | `HG_STORE_DATA_PATH` | 否 | `/hugegraph-store/storage` | `app.data-path` | 数据存储路径。 | +| `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` | 否 | 无 | `rocksdb.total_memory_size` | RocksDB 内存预算,单位字节,只能是数字;其他值会使入口脚本以退出码 2 退出。未设置时使用 `application-pd.yml` 中的值或自动计算。 | 当前主线 entrypoint 将这些变量写入 `SPRING_APPLICATION_JSON`,覆盖镜像内 `conf/application.yml`;其余配置仍从该文件和 `application-pd.yml` 读取,然后以前台方式启动 Java。旧变量 `PD_ADDRESS`、`GRPC_HOST`、`RAFT_ADDRESS` 仍可用,但会输出弃用警告。 @@ -176,7 +177,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ | 配置项 | 当前主线目录值 | 代码默认值 | 说明 | |--------|--------|------------|------| | `raft.address` | `127.0.0.1:8510` | 必填 | 本节点的 Raft 服务地址,格式为 `host:port`,必须能被其他 Store 节点访问。这里不需要配置 peer 列表:分区 Raft 组的成员由 PD 下发。 | -| `raft.disruptorBufferSize` | `1024` | `0` | Raft 任务队列大小。设为 `0` 时按 `rocksdb.total_memory_size` 推导:将该内存量的 GB 数取最接近的 2 的幂,再乘以 32。 | +| `raft.disruptorBufferSize` | `1024` | `0` | Raft 任务队列大小。设为 `0` 时按 `rocksdb.total_memory_size` 推导:将该内存量的 GB 数(至少为 1)取最接近的 2 的幂,再乘以 32。 | | `raft.max-log-file-size` | `600000000000` | `50000000000` | Raft 日志的最大字节数。 | | `raft.snapshotInterval` | `1800` | `300` | 生成 Raft 快照的时间间隔,单位秒。 | | `raft.snapshotLogIndexMargin` | 未设置 | `0` | 距上次快照的最小 applied index 差值,达到后才真正生成快照。设为 `0` 关闭该判断。 | @@ -203,7 +204,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ | 配置项 | 当前主线目录值 | 代码默认值 | 说明 | |--------|--------|------------|------| -| `rocksdb.total_memory_size` | `32000000000` | `51539607552` | 本节点所有 RocksDB 实例共享的内存预算。缺失或为 `0` 时使用 JVM 最大堆内存。 | +| `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算,最多 1 GiB)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | | `rocksdb.write_buffer_size` | `32000000` | `33554432` | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`。 | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | 落盘前合并的 memtable 数量。 | | `rocksdb.write_buffer_ratio` | 未设置 | `0.66` | `total_memory_size` 中分配给写缓存的比例,其余作为 block cache。 | diff --git a/content/en/docs/guides/hugegraph-docker-cluster.md b/content/en/docs/guides/hugegraph-docker-cluster.md index 0360a7c634..a07e433227 100644 --- a/content/en/docs/guides/hugegraph-docker-cluster.md +++ b/content/en/docs/guides/hugegraph-docker-cluster.md @@ -195,9 +195,12 @@ The following tables describe current master entrypoints and Compose files, not | `HG_STORE_GRPC_PORT` | No | `8500` | `grpc.port` | | `HG_STORE_REST_PORT` | No | `8520` | `server.port` | | `HG_STORE_DATA_PATH` | No | `/hugegraph-store/storage` | `app.data-path` | +| `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` | No | None | `rocksdb.total_memory_size`, in bytes | > **Deprecated aliases**: `PD_ADDRESS` → `HG_STORE_PD_ADDRESS`, `GRPC_HOST` → `HG_STORE_GRPC_HOST`, `RAFT_ADDRESS` → `HG_STORE_RAFT_ADDRESS`. +RocksDB caches live outside the Java heap. Leave `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` unset to size them from the container memory limit, or set it to a plain number of bytes; see `rocksdb.total_memory_size` in the [HStore configuration reference](/docs/quickstart/hugegraph/hugegraph-hstore/). + ### Server Variables Unlike PD and Store, Server has no mandatory entrypoint variables: only set values are written to configuration. Distributed deployment nevertheless requires at least `HG_SERVER_BACKEND` and `HG_SERVER_PD_PEERS`. diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index 5e987e695b..cb8e74d0b4 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -131,6 +131,7 @@ docker run -d \ | `HG_STORE_GRPC_PORT` | No | `8500` | `grpc.port` | gRPC service port. | | `HG_STORE_REST_PORT` | No | `8520` | `server.port` | REST API port. | | `HG_STORE_DATA_PATH` | No | `/hugegraph-store/storage` | `app.data-path` | Data path. | +| `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` | No | None | `rocksdb.total_memory_size` | RocksDB memory budget in bytes, digits only; other values stop the entrypoint with exit code 2. When unset, the value in `application-pd.yml` or auto-sizing applies. | The master entrypoint writes these values into `SPRING_APPLICATION_JSON`, overriding image `conf/application.yml`. Other options still come from that file and `application-pd.yml`; Java runs in the foreground. Deprecated `PD_ADDRESS`, `GRPC_HOST`, and `RAFT_ADDRESS` remain supported with warnings. @@ -177,7 +178,7 @@ This file contains only `rocksdb` and Actuator exposure settings. | Key | Master distribution value | Code default | Meaning | |-----|---------|--------------|---------| | `raft.address` | `127.0.0.1:8510` | required | Raft service address of this node, `host:port`. Must be reachable from every other Store node. There is no peer list to configure: PD tells each node which peers belong to a partition's Raft group. | -| `raft.disruptorBufferSize` | `1024` | `0` | Raft task queue size. `0` derives it from `rocksdb.total_memory_size`, by rounding that size in GB to the nearest power of two and multiplying by 32. | +| `raft.disruptorBufferSize` | `1024` | `0` | Raft task queue size. `0` derives it from `rocksdb.total_memory_size`, by rounding that size in GB (at least 1) to the nearest power of two and multiplying by 32. | | `raft.max-log-file-size` | `600000000000` | `50000000000` | Maximum byte size of Raft logs. | | `raft.snapshotInterval` | `1800` | `300` | Seconds between Raft snapshots. | | `raft.snapshotLogIndexMargin` | not set | `0` | Minimum applied-index distance since the last snapshot before a snapshot is actually written. `0` disables the distance check. | @@ -204,7 +205,7 @@ This file contains only `rocksdb` and Actuator exposure settings. | Key | Master distribution value | Code default | Meaning | |-----|---------|--------------|---------| -| `rocksdb.total_memory_size` | `32000000000` | `51539607552` | Memory budget shared by all RocksDB instances on this node. When absent or `0`, the node uses the JVM max heap instead. | +| `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size, at most 1 GiB, when unset) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit, including a limit set only on a parent cgroup the Store can see. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | | `rocksdb.write_buffer_size` | `32000000` | `33554432` | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`. | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | Number of memtables merged together before a flush. | | `rocksdb.write_buffer_ratio` | not set | `0.66` | Share of `total_memory_size` given to the write cache. The rest becomes the block cache. | From 120c614e75e5072408796f62c8a73ba412e5461c Mon Sep 17 00:00:00 2001 From: Ayan Alam Date: Sat, 17 Oct 2026 01:35:35 +0530 Subject: [PATCH 02/23] docs(store): note the 1 MiB floor on the derived write_buffer_size apache/hugegraph#3355 fills an absent or 0 rocksdb.write_buffer_size with total_memory_size / 1000, but never below the 1 MiB that RocksDBOptions accepts. Any budget under about 1 GB, including the 256 MiB auto-sizing floor, therefore gets 1048576. Say so in the EN and CN rows. --- content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- content/en/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index 23f7e6ce69..8114aa8979 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -205,7 +205,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ | 配置项 | 当前主线目录值 | 代码默认值 | 说明 | |--------|--------|------------|------| | `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算,最多 1 GiB)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | -| `rocksdb.write_buffer_size` | `32000000` | `33554432` | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`。 | +| `rocksdb.write_buffer_size` | `32000000` | `33554432` | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`,最小为 `1048576`(1 MiB)。 | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | 落盘前合并的 memtable 数量。 | | `rocksdb.write_buffer_ratio` | 未设置 | `0.66` | `total_memory_size` 中分配给写缓存的比例,其余作为 block cache。 | diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index 943fea9cab..c6e59d9971 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -207,7 +207,7 @@ This file contains only `rocksdb` and Actuator exposure settings. | Key | Master distribution value | Code default | Meaning | |-----|---------|--------------|---------| | `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size, at most 1 GiB, when unset) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit, including a limit set only on a parent cgroup the Store can see. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | -| `rocksdb.write_buffer_size` | `32000000` | `33554432` | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`. | +| `rocksdb.write_buffer_size` | `32000000` | `33554432` | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`, at least `1048576` (1 MiB). | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | Number of memtables merged together before a flush. | | `rocksdb.write_buffer_ratio` | not set | `0.66` | Share of `total_memory_size` given to the write cache. The rest becomes the block cache. | From 29bfda5eb5546fcc9e7f1753e75e506242be4e91 Mon Sep 17 00:00:00 2001 From: ayanx Date: Sat, 17 Oct 2026 03:51:12 +0530 Subject: [PATCH 03/23] docs(store): say where the RocksDB budget comes from without a limit The Docker guide said an unset HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE sizes the budget from the container memory limit, but the Compose files it documents set no limit, so each Store sizes from the whole Docker host. Say so, and tell readers to set a limit or pin the variable when several Stores share a host. The HStore row now says the cgroup limit applies when one is set, otherwise the host's memory. The HStore page also said its tables were checked at a master commit from before apache/hugegraph#3355; note that the three RocksDB and disruptor rows describe that PR until its merge commit replaces it. --- content/cn/docs/guides/hugegraph-docker-cluster.md | 2 +- content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md | 4 ++-- content/en/docs/guides/hugegraph-docker-cluster.md | 2 +- content/en/docs/quickstart/hugegraph/hugegraph-hstore.md | 4 ++-- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/content/cn/docs/guides/hugegraph-docker-cluster.md b/content/cn/docs/guides/hugegraph-docker-cluster.md index 5ee6d8e4f4..e1199cb8e5 100644 --- a/content/cn/docs/guides/hugegraph-docker-cluster.md +++ b/content/cn/docs/guides/hugegraph-docker-cluster.md @@ -199,7 +199,7 @@ curl -o /dev/null -w '%{http_code}\n' -u "admin:${HUGEGRAPH_ADMIN_PASSWORD}" \ > **已弃用的别名**:`PD_ADDRESS` → `HG_STORE_PD_ADDRESS`、`GRPC_HOST` → `HG_STORE_GRPC_HOST`、`RAFT_ADDRESS` → `HG_STORE_RAFT_ADDRESS` -RocksDB 缓存位于 Java 堆外。不设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` 时按容器内存限制自动计算,也可以设为纯字节数;详见 [HStore 配置参考](/cn/docs/quickstart/hugegraph/hugegraph-hstore/)中的 `rocksdb.total_memory_size`。 +RocksDB 缓存位于 Java 堆外。不设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` 时按容器内存限制自动计算;容器没有内存限制时,按 Docker 宿主机(Docker Desktop 虚拟机)的内存计算。上面的 Compose 文件都没有设置内存限制,因此每个 Store 都会按整台宿主机的内存计算预算;多个 Store 共用一台宿主机时,请为 Store 服务设置内存限制,或把该变量设为纯字节数。详见 [HStore 配置参考](/cn/docs/quickstart/hugegraph/hugegraph-hstore/)中的 `rocksdb.total_memory_size`。 ### Server 变量 diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index 8114aa8979..3892515b2a 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -146,7 +146,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ #### 4.1 application.yml -以下配置参考表核对的是 server 主线提交 `2f827d6e8c9c62ae858f2fc122b3a192d015e2f4` 的发行目录值和 Java 默认注入值;1.7.0 发布包请使用其版本标签对应的配置文件,不要跨版本覆盖。 +以下配置参考表核对的是 server 主线提交 `2f827d6e8c9c62ae858f2fc122b3a192d015e2f4` 的发行目录值和 Java 默认注入值,其中 `rocksdb.total_memory_size`、`rocksdb.write_buffer_size` 和 `raft.disruptorBufferSize` 三行描述的是 apache/hugegraph#3355 的行为;1.7.0 发布包请使用其版本标签对应的配置文件,不要跨版本覆盖。 #### 4.2 application-pd.yml @@ -204,7 +204,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ | 配置项 | 当前主线目录值 | 代码默认值 | 说明 | |--------|--------|------------|------| -| `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算,最多 1 GiB)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | +| `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算,最多 1 GiB)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | | `rocksdb.write_buffer_size` | `32000000` | `33554432` | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`,最小为 `1048576`(1 MiB)。 | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | 落盘前合并的 memtable 数量。 | | `rocksdb.write_buffer_ratio` | 未设置 | `0.66` | `total_memory_size` 中分配给写缓存的比例,其余作为 block cache。 | diff --git a/content/en/docs/guides/hugegraph-docker-cluster.md b/content/en/docs/guides/hugegraph-docker-cluster.md index 5e6cad27fb..d4886be89e 100644 --- a/content/en/docs/guides/hugegraph-docker-cluster.md +++ b/content/en/docs/guides/hugegraph-docker-cluster.md @@ -199,7 +199,7 @@ The following tables describe current master entrypoints and Compose files, not > **Deprecated aliases**: `PD_ADDRESS` → `HG_STORE_PD_ADDRESS`, `GRPC_HOST` → `HG_STORE_GRPC_HOST`, `RAFT_ADDRESS` → `HG_STORE_RAFT_ADDRESS`. -RocksDB caches live outside the Java heap. Leave `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` unset to size them from the container memory limit, or set it to a plain number of bytes; see `rocksdb.total_memory_size` in the [HStore configuration reference](/docs/quickstart/hugegraph/hugegraph-hstore/). +RocksDB caches live outside the Java heap. Leave `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` unset to size them from the container memory limit, or from the memory of the Docker host (the Docker Desktop VM) when the container has no limit. The Compose files above set no limit, so each Store sizes its budget from the whole host; set a memory limit on the Store services, or set this variable to a plain number of bytes, when several Stores share one host. See `rocksdb.total_memory_size` in the [HStore configuration reference](/docs/quickstart/hugegraph/hugegraph-hstore/). ### Server Variables diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index c6e59d9971..e15de0e36d 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -147,7 +147,7 @@ Store reads two files under `conf/`. Release 1.7.0 sources: [application.yml](ht #### 4.1 application.yml -The following tables were checked against distribution values and Java defaults at Server master commit `2f827d6e8c9c62ae858f2fc122b3a192d015e2f4`. Use tagged configuration for release 1.7.0 without replacing it with another version's files. +The following tables were checked against distribution values and Java defaults at Server master commit `2f827d6e8c9c62ae858f2fc122b3a192d015e2f4`, except the `rocksdb.total_memory_size`, `rocksdb.write_buffer_size` and `raft.disruptorBufferSize` rows, which describe apache/hugegraph#3355. Use tagged configuration for release 1.7.0 without replacing it with another version's files. #### 4.2 application-pd.yml @@ -206,7 +206,7 @@ This file contains only `rocksdb` and Actuator exposure settings. | Key | Master distribution value | Code default | Meaning | |-----|---------|--------------|---------| -| `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size, at most 1 GiB, when unset) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit, including a limit set only on a parent cgroup the Store can see. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | +| `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size, at most 1 GiB, when unset) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | | `rocksdb.write_buffer_size` | `32000000` | `33554432` | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`, at least `1048576` (1 MiB). | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | Number of memtables merged together before a flush. | | `rocksdb.write_buffer_ratio` | not set | `0.66` | Share of `total_memory_size` given to the write cache. The rest becomes the block cache. | From 1585f969989ba25199ce6138898f8857b363fc63 Mon Sep 17 00:00:00 2001 From: ayanx Date: Sat, 17 Oct 2026 12:14:03 +0530 Subject: [PATCH 04/23] docs(store): count unset direct memory as the full heap apache/hugegraph#3355 now budgets the JVM's real direct-memory allowance: an unset -XX:MaxDirectMemorySize counts as the heap, and bin/start-hugegraph-store.sh sets the flag to the heap it picks unless -j passes one. Drop the "at most 1 GiB" from the EN and CN rows. --- content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- content/en/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index 3892515b2a..84e97cd507 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -204,7 +204,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ | 配置项 | 当前主线目录值 | 代码默认值 | 说明 | |--------|--------|------------|------| -| `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算,最多 1 GiB)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | +| `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小,可用 `-j` 传入更小的值)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | | `rocksdb.write_buffer_size` | `32000000` | `33554432` | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`,最小为 `1048576`(1 MiB)。 | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | 落盘前合并的 memtable 数量。 | | `rocksdb.write_buffer_ratio` | 未设置 | `0.66` | `total_memory_size` 中分配给写缓存的比例,其余作为 block cache。 | diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index e15de0e36d..6588f93863 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -206,7 +206,7 @@ This file contains only `rocksdb` and Actuator exposure settings. | Key | Master distribution value | Code default | Meaning | |-----|---------|--------------|---------| -| `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size, at most 1 GiB, when unset) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | +| `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` passes a smaller one) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | | `rocksdb.write_buffer_size` | `32000000` | `33554432` | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`, at least `1048576` (1 MiB). | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | Number of memtables merged together before a flush. | | `rocksdb.write_buffer_ratio` | not set | `0.66` | Share of `total_memory_size` given to the write cache. The rest becomes the block cache. | From b9e2b3483b212076504fbcbde92d3d89668f06c9 Mon Sep 17 00:00:00 2001 From: ayanx Date: Sat, 17 Oct 2026 12:21:04 +0530 Subject: [PATCH 05/23] docs(store): describe when the launcher adds the direct-memory cap apache/hugegraph#3355 now leaves -XX:MaxDirectMemorySize out when -j sets -Xmx or the cap itself, so a heap-only -j gets the JVM default. Update the EN and CN rows. --- content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- content/en/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index 84e97cd507..538bdeeadc 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -204,7 +204,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ | 配置项 | 当前主线目录值 | 代码默认值 | 说明 | |--------|--------|------------|------| -| `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小,可用 `-j` 传入更小的值)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | +| `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx` 或该上限时不再添加)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | | `rocksdb.write_buffer_size` | `32000000` | `33554432` | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`,最小为 `1048576`(1 MiB)。 | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | 落盘前合并的 memtable 数量。 | | `rocksdb.write_buffer_ratio` | 未设置 | `0.66` | `total_memory_size` 中分配给写缓存的比例,其余作为 block cache。 | diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index 6588f93863..8731ad45b8 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -206,7 +206,7 @@ This file contains only `rocksdb` and Actuator exposure settings. | Key | Master distribution value | Code default | Meaning | |-----|---------|--------------|---------| -| `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` passes a smaller one) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | +| `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx` or the cap itself) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | | `rocksdb.write_buffer_size` | `32000000` | `33554432` | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`, at least `1048576` (1 MiB). | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | Number of memtables merged together before a flush. | | `rocksdb.write_buffer_ratio` | not set | `0.66` | Share of `total_memory_size` given to the write cache. The rest becomes the block cache. | From 0bb82783834b4b89e67c6a3e88c580713c831614 Mon Sep 17 00:00:00 2001 From: imbajin Date: Sat, 17 Oct 2026 21:39:47 +0800 Subject: [PATCH 06/23] docs(store): include -XX:MaxHeapSize when launcher skips direct-memory cap --- content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- content/en/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index 538bdeeadc..c0b190e4d4 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -204,7 +204,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ | 配置项 | 当前主线目录值 | 代码默认值 | 说明 | |--------|--------|------------|------| -| `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx` 或该上限时不再添加)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | +| `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | | `rocksdb.write_buffer_size` | `32000000` | `33554432` | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`,最小为 `1048576`(1 MiB)。 | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | 落盘前合并的 memtable 数量。 | | `rocksdb.write_buffer_ratio` | 未设置 | `0.66` | `total_memory_size` 中分配给写缓存的比例,其余作为 block cache。 | diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index 8731ad45b8..cc5a60f9b2 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -206,7 +206,7 @@ This file contains only `rocksdb` and Actuator exposure settings. | Key | Master distribution value | Code default | Meaning | |-----|---------|--------------|---------| -| `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx` or the cap itself) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | +| `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | | `rocksdb.write_buffer_size` | `32000000` | `33554432` | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`, at least `1048576` (1 MiB). | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | Number of memtables merged together before a flush. | | `rocksdb.write_buffer_ratio` | not set | `0.66` | Share of `total_memory_size` given to the write cache. The rest becomes the block cache. | From d11a103085012e9d3caef9cb9382f775664c2c5b Mon Sep 17 00:00:00 2001 From: ayanx Date: Sun, 18 Oct 2026 01:31:58 +0530 Subject: [PATCH 07/23] docs(store): add a direct-memory cap to the production launcher example With apache/hugegraph#3355 a -j that sets -Xmx drops the launcher's -XX:MaxDirectMemorySize, so the JVM allows direct memory equal to the heap and the RocksDB budget counts it. The example -j "-Xmx32g -Xms32g" therefore left any host under about 66 GB on the 256 MiB floor. Add -XX:MaxDirectMemorySize=2g to the example, list the cap among the flags the script adds, and say why, in EN and CN. --- content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- content/en/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index c0b190e4d4..09bb05bafa 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -313,7 +313,7 @@ master 构建接受 `g1`、`G1`、`zgc` 和 `ZGC`。 - `-d true`(默认):以后台守护进程方式运行,脚本立即返回,并把 Java 进程号写入 `bin/pid`。 - `-d false`:以前台模式运行,脚本通过 `exec` 替换为 Java 进程,容器或进程管理器的进程即为 Java 本身。在 Docker 或进程管理器(systemd、supervisord)下运行时请使用此参数,以便在崩溃时自动检测并重启服务。 -JVM 内存方面,如果没有自行设置 `JAVA_OPTIONS`:`-Xms512m`,`-Xmx` 取空闲内存的一半并限制在 512 MB 到 2048 MB 之间。脚本还会加上 `-XX:MetaspaceSize=256M`、OOM 时把堆转储写入 `logs/`,以及滚动的 GC 日志 `logs/gc.log`。生产节点通常需要大得多的堆,请显式指定,例如 `-j "-Xmx32g -Xms32g"`。 +JVM 内存方面,如果没有自行设置 `JAVA_OPTIONS`:`-Xms512m`,`-Xmx` 取空闲内存的一半并限制在 512 MB 到 2048 MB 之间。脚本还会加上与该堆大小相同的 `-XX:MaxDirectMemorySize`、`-XX:MetaspaceSize=256M`、OOM 时把堆转储写入 `logs/`,以及滚动的 GC 日志 `logs/gc.log`。生产节点通常需要大得多的堆,请显式指定,并同时设置直接内存上限,例如 `-j "-Xmx32g -Xms32g -XX:MaxDirectMemorySize=2g"`。不设上限时,JVM 允许的直接内存与 32 GB 的堆相同,RocksDB 预算会把它计算在内,在内存不足约 66 GB 的主机上会降到 256 MiB 的下限。 当 `ulimit -n` 或 `ulimit -u` 低于 1024 时脚本会拒绝启动;在 x86_64 和 arm64 上,如果能成功下载并校验对应的动态库,脚本会预加载 jemalloc。 diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index cc5a60f9b2..bb79f4a4a7 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -319,7 +319,7 @@ Daemon mode: - `-d true` (default): run as a background daemon. The script returns immediately and writes the Java pid to `bin/pid`. - `-d false`: run in the foreground. The script `exec`s Java, so the container or supervisor process is Java itself. Use this under Docker or a process supervisor (systemd, supervisord) so crashes are detected and the service is restarted automatically. -JVM memory, unless you set `JAVA_OPTIONS` yourself: `-Xms512m`, and `-Xmx` set to half the free memory, clamped to the 512 MB to 2048 MB range. The script also adds `-XX:MetaspaceSize=256M`, a heap dump on out-of-memory into `logs/`, and a rolling GC log at `logs/gc.log`. Production nodes normally need a much larger heap, so pass one explicitly, for example `-j "-Xmx32g -Xms32g"`. +JVM memory, unless you set `JAVA_OPTIONS` yourself: `-Xms512m`, and `-Xmx` set to half the free memory, clamped to the 512 MB to 2048 MB range. The script also adds `-XX:MaxDirectMemorySize` equal to that heap, `-XX:MetaspaceSize=256M`, a heap dump on out-of-memory into `logs/`, and a rolling GC log at `logs/gc.log`. Production nodes normally need a much larger heap, so pass one explicitly together with a direct-memory cap, for example `-j "-Xmx32g -Xms32g -XX:MaxDirectMemorySize=2g"`. Without the cap the JVM allows as much direct memory as the 32 GB heap, the RocksDB budget counts it, and on a host with less than about 66 GB it falls to the 256 MiB floor. The script refuses to start if `ulimit -n` or `ulimit -u` is below 1024, and it preloads jemalloc on x86_64 and arm64 when the shared object can be downloaded and verified. From a451d1da877fcea338095ad9837009a4dec5ec79 Mon Sep 17 00:00:00 2001 From: ayanx Date: Mon, 19 Oct 2026 19:40:47 +0530 Subject: [PATCH 08/23] docs(store): note the unknown-memory fallback and size the JVM for a limit The HStore row now says what the budget falls back to when neither the JVM nor the cgroup files report a usable memory size: the JVM max heap, clamped to 256 MiB to 32 GB. The Docker guide said a Store memory limit sizes the RocksDB budget, but that is all it bounds: the launcher still picks -Xmx from the host's free memory and sets direct memory to the same size. Tell operators to size both through JAVA_OPTS for the limit, with an example, and note that the defaults already exceed a 2 GiB limit. EN and CN. --- content/cn/docs/guides/hugegraph-docker-cluster.md | 2 +- content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- content/en/docs/guides/hugegraph-docker-cluster.md | 2 +- content/en/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/content/cn/docs/guides/hugegraph-docker-cluster.md b/content/cn/docs/guides/hugegraph-docker-cluster.md index d6eee56f8c..6174acf619 100644 --- a/content/cn/docs/guides/hugegraph-docker-cluster.md +++ b/content/cn/docs/guides/hugegraph-docker-cluster.md @@ -201,7 +201,7 @@ curl -o /dev/null -w '%{http_code}\n' -u "admin:${HUGEGRAPH_ADMIN_PASSWORD}" \ > **已弃用的别名**:`PD_ADDRESS` → `HG_STORE_PD_ADDRESS`、`GRPC_HOST` → `HG_STORE_GRPC_HOST`、`RAFT_ADDRESS` → `HG_STORE_RAFT_ADDRESS` -RocksDB 缓存位于 Java 堆外。不设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` 时按容器内存限制自动计算;容器没有内存限制时,按 Docker 宿主机(Docker Desktop 虚拟机)的内存计算。上面的 Compose 文件都没有设置内存限制,因此每个 Store 都会按整台宿主机的内存计算预算;多个 Store 共用一台宿主机时,请为 Store 服务设置内存限制,或把该变量设为纯字节数。详见 [HStore 配置参考](/cn/docs/quickstart/hugegraph/hugegraph-hstore/)中的 `rocksdb.total_memory_size`。 +RocksDB 缓存位于 Java 堆外。不设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` 时按容器内存限制自动计算;容器没有内存限制时,按 Docker 宿主机(Docker Desktop 虚拟机)的内存计算。上面的 Compose 文件都没有设置内存限制,因此每个 Store 都会按整台宿主机的内存计算预算;多个 Store 共用一台宿主机时,请为 Store 服务设置内存限制,或把该变量设为纯字节数。容器内存限制只约束 RocksDB 预算:Store 启动脚本仍会按宿主机的空闲内存选择 `-Xmx`(最多 2 GiB),并把直接内存设为相同大小。请同样按限制通过 `JAVA_OPTS`(镜像会以 `-j` 传给启动脚本)设置这两项,例如 4 GiB 限制时设为 `JAVA_OPTS="-Xmx1g -XX:MaxDirectMemorySize=512m"`;使用默认值时,堆、直接内存、1.5 GiB 预留和 256 MiB 最低预算之和已超过 2 GiB 的限制。详见 [HStore 配置参考](/cn/docs/quickstart/hugegraph/hugegraph-hstore/)中的 `rocksdb.total_memory_size`。 ### Server 变量 diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index befe6db5e9..59f93c9244 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -208,7 +208,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ | 配置项 | 当前主线目录值 | 代码默认值 | 说明 | |--------|--------|------------|------| -| `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | +| `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。JVM 和 cgroup 文件都拿不到可用的内存大小时,改用 JVM 最大堆,同样限制在 256 MiB 到 32 GB 之间。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | | `rocksdb.write_buffer_size` | `32000000` | `33554432` | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`,最小为 `1048576`(1 MiB)。 | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | 落盘前合并的 memtable 数量。 | | `rocksdb.write_buffer_ratio` | 未设置 | `0.66` | `total_memory_size` 中分配给写缓存的比例,其余作为 block cache。 | diff --git a/content/en/docs/guides/hugegraph-docker-cluster.md b/content/en/docs/guides/hugegraph-docker-cluster.md index aa8475abdf..da824c1a05 100644 --- a/content/en/docs/guides/hugegraph-docker-cluster.md +++ b/content/en/docs/guides/hugegraph-docker-cluster.md @@ -201,7 +201,7 @@ The following tables describe current master entrypoints and Compose files, not > **Deprecated aliases**: `PD_ADDRESS` → `HG_STORE_PD_ADDRESS`, `GRPC_HOST` → `HG_STORE_GRPC_HOST`, `RAFT_ADDRESS` → `HG_STORE_RAFT_ADDRESS`. -RocksDB caches live outside the Java heap. Leave `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` unset to size them from the container memory limit, or from the memory of the Docker host (the Docker Desktop VM) when the container has no limit. The Compose files above set no limit, so each Store sizes its budget from the whole host; set a memory limit on the Store services, or set this variable to a plain number of bytes, when several Stores share one host. See `rocksdb.total_memory_size` in the [HStore configuration reference](/docs/quickstart/hugegraph/hugegraph-hstore/). +RocksDB caches live outside the Java heap. Leave `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` unset to size them from the container memory limit, or from the memory of the Docker host (the Docker Desktop VM) when the container has no limit. The Compose files above set no limit, so each Store sizes its budget from the whole host; set a memory limit on the Store services, or set this variable to a plain number of bytes, when several Stores share one host. A container limit bounds only the RocksDB budget: the Store launcher still picks `-Xmx` from the host's free memory (up to 2 GiB) and sets direct memory to the same size. Size those for the limit too through `JAVA_OPTS`, which the image passes to the launcher as `-j`, for example `JAVA_OPTS="-Xmx1g -XX:MaxDirectMemorySize=512m"` for a 4 GiB limit; with the defaults, heap plus direct memory, the 1.5 GiB reserve and the 256 MiB minimum budget already exceed a 2 GiB limit. See `rocksdb.total_memory_size` in the [HStore configuration reference](/docs/quickstart/hugegraph/hugegraph-hstore/). ### Server Variables diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index 8c8f54839f..efd9a23147 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -210,7 +210,7 @@ This file contains only `rocksdb` and Actuator exposure settings. | Key | Master distribution value | Code default | Meaning | |-----|---------|--------------|---------| -| `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | +| `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. If neither the JVM nor the cgroup files report a usable memory size, the budget falls back to the JVM max heap, clamped to the same 256 MiB to 32 GB. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | | `rocksdb.write_buffer_size` | `32000000` | `33554432` | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`, at least `1048576` (1 MiB). | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | Number of memtables merged together before a flush. | | `rocksdb.write_buffer_ratio` | not set | `0.66` | Share of `total_memory_size` given to the write cache. The rest becomes the block cache. | From 781384bf8e474f7b2efe6eeabe8e4c98fa1a4bb5 Mon Sep 17 00:00:00 2001 From: imbajin Date: Tue, 20 Oct 2026 03:42:28 +0800 Subject: [PATCH 09/23] docs(store): polish memory budget phrasing and wrap long lines --- content/cn/docs/guides/hugegraph-docker-cluster.md | 6 +++++- content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md | 5 ++++- content/en/docs/guides/hugegraph-docker-cluster.md | 8 +++++++- content/en/docs/quickstart/hugegraph/hugegraph-hstore.md | 6 +++++- 4 files changed, 21 insertions(+), 4 deletions(-) diff --git a/content/cn/docs/guides/hugegraph-docker-cluster.md b/content/cn/docs/guides/hugegraph-docker-cluster.md index 6174acf619..f6dda9d098 100644 --- a/content/cn/docs/guides/hugegraph-docker-cluster.md +++ b/content/cn/docs/guides/hugegraph-docker-cluster.md @@ -201,7 +201,11 @@ curl -o /dev/null -w '%{http_code}\n' -u "admin:${HUGEGRAPH_ADMIN_PASSWORD}" \ > **已弃用的别名**:`PD_ADDRESS` → `HG_STORE_PD_ADDRESS`、`GRPC_HOST` → `HG_STORE_GRPC_HOST`、`RAFT_ADDRESS` → `HG_STORE_RAFT_ADDRESS` -RocksDB 缓存位于 Java 堆外。不设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` 时按容器内存限制自动计算;容器没有内存限制时,按 Docker 宿主机(Docker Desktop 虚拟机)的内存计算。上面的 Compose 文件都没有设置内存限制,因此每个 Store 都会按整台宿主机的内存计算预算;多个 Store 共用一台宿主机时,请为 Store 服务设置内存限制,或把该变量设为纯字节数。容器内存限制只约束 RocksDB 预算:Store 启动脚本仍会按宿主机的空闲内存选择 `-Xmx`(最多 2 GiB),并把直接内存设为相同大小。请同样按限制通过 `JAVA_OPTS`(镜像会以 `-j` 传给启动脚本)设置这两项,例如 4 GiB 限制时设为 `JAVA_OPTS="-Xmx1g -XX:MaxDirectMemorySize=512m"`;使用默认值时,堆、直接内存、1.5 GiB 预留和 256 MiB 最低预算之和已超过 2 GiB 的限制。详见 [HStore 配置参考](/cn/docs/quickstart/hugegraph/hugegraph-hstore/)中的 `rocksdb.total_memory_size`。 +RocksDB 缓存位于 Java 堆外。不设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` 时,将按容器内存限制自动计算;若容器未设置限制,则按宿主机(或 Docker Desktop 虚拟机)内存计算。 +上面的 Compose 文件默认未设置内存限制,每个 Store 均会按整台宿主机估算预算;多 Store 共用单机时,建议为服务配置内存限制,或将该变量显式设为字节数。 +需要注意,容器限制仅约束 RocksDB 预算:Store 启动脚本仍会按宿主机空闲内存自动分配 `-Xmx`(最多 2 GiB),并将直接内存设为相同大小。 +因此请同步通过 `JAVA_OPTS`(镜像会以 `-j` 传给启动脚本)显式约束 JVM 内存,例如容器限制为 4 GiB 时可设置 `JAVA_OPTS="-Xmx1g -XX:MaxDirectMemorySize=512m"`; +若直接使用默认值,堆内存、直接内存、1.5 GiB 预留加上 256 MiB 最低预算总和已超过 2 GiB。详见 [HStore 配置参考](/cn/docs/quickstart/hugegraph/hugegraph-hstore/)中的 `rocksdb.total_memory_size`。 ### Server 变量 diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index 59f93c9244..a7e5e9b5f7 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -326,7 +326,10 @@ master 构建接受 `g1`、`G1`、`zgc` 和 `ZGC`。 - `-d true`(默认):以后台守护进程方式运行,脚本立即返回,并把 Java 进程号写入 `bin/pid`。 - `-d false`:以前台模式运行,脚本通过 `exec` 替换为 Java 进程,容器或进程管理器的进程即为 Java 本身。在 Docker 或进程管理器(systemd、supervisord)下运行时请使用此参数,以便在崩溃时自动检测并重启服务。 -JVM 内存方面,如果没有自行设置 `JAVA_OPTIONS`:`-Xms512m`,`-Xmx` 取空闲内存的一半并限制在 512 MB 到 2048 MB 之间。脚本还会加上与该堆大小相同的 `-XX:MaxDirectMemorySize`、`-XX:MetaspaceSize=256M`、OOM 时把堆转储写入 `logs/`,以及滚动的 GC 日志 `logs/gc.log`。生产节点通常需要大得多的堆,请显式指定,并同时设置直接内存上限,例如 `-j "-Xmx32g -Xms32g -XX:MaxDirectMemorySize=2g"`。不设上限时,JVM 允许的直接内存与 32 GB 的堆相同,RocksDB 预算会把它计算在内,在内存不足约 66 GB 的主机上会降到 256 MiB 的下限。 +JVM 内存方面,如果没有自行设置 `JAVA_OPTIONS`:`-Xms512m`,`-Xmx` 取空闲内存的一半并限制在 512 MB 到 2048 MB 之间。 +脚本还会加上与该堆大小相同的 `-XX:MaxDirectMemorySize`、`-XX:MetaspaceSize=256M`、OOM 时把堆转储写入 `logs/`,以及滚动的 GC 日志 `logs/gc.log`。 +生产节点通常需要大得多的堆,请显式指定,并同时设置直接内存上限,例如 `-j "-Xmx32g -Xms32g -XX:MaxDirectMemorySize=2g"`。 +不设上限时,JVM 允许的直接内存与 32 GB 的堆相同,RocksDB 预算会把它计算在内,在内存不足约 66 GB 的主机上会降到 256 MiB 的下限。 当 `ulimit -n` 或 `ulimit -u` 低于 1024 时脚本会拒绝启动;在 x86_64 和 arm64 上,如果能成功下载并校验对应的动态库,脚本会预加载 jemalloc。 diff --git a/content/en/docs/guides/hugegraph-docker-cluster.md b/content/en/docs/guides/hugegraph-docker-cluster.md index da824c1a05..2eff9aeace 100644 --- a/content/en/docs/guides/hugegraph-docker-cluster.md +++ b/content/en/docs/guides/hugegraph-docker-cluster.md @@ -201,7 +201,13 @@ The following tables describe current master entrypoints and Compose files, not > **Deprecated aliases**: `PD_ADDRESS` → `HG_STORE_PD_ADDRESS`, `GRPC_HOST` → `HG_STORE_GRPC_HOST`, `RAFT_ADDRESS` → `HG_STORE_RAFT_ADDRESS`. -RocksDB caches live outside the Java heap. Leave `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` unset to size them from the container memory limit, or from the memory of the Docker host (the Docker Desktop VM) when the container has no limit. The Compose files above set no limit, so each Store sizes its budget from the whole host; set a memory limit on the Store services, or set this variable to a plain number of bytes, when several Stores share one host. A container limit bounds only the RocksDB budget: the Store launcher still picks `-Xmx` from the host's free memory (up to 2 GiB) and sets direct memory to the same size. Size those for the limit too through `JAVA_OPTS`, which the image passes to the launcher as `-j`, for example `JAVA_OPTS="-Xmx1g -XX:MaxDirectMemorySize=512m"` for a 4 GiB limit; with the defaults, heap plus direct memory, the 1.5 GiB reserve and the 256 MiB minimum budget already exceed a 2 GiB limit. See `rocksdb.total_memory_size` in the [HStore configuration reference](/docs/quickstart/hugegraph/hugegraph-hstore/). +RocksDB caches live outside the Java heap. Leave `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` unset to size them from the container memory limit, or from the +memory of the Docker host (such as the Docker Desktop VM) when the container has no limit. The Compose files above set no limit, so each Store sizes its budget +from the whole host; set a memory limit on Store services or set this variable to a plain number of bytes when several Stores share one host. +A container limit bounds only the RocksDB budget: the Store launcher still picks `-Xmx` from the host's free memory (up to 2 GiB) and sets direct memory to the +same size. Size those for the limit too through `JAVA_OPTS` (passed to the launcher as `-j`), for example `JAVA_OPTS="-Xmx1g -XX:MaxDirectMemorySize=512m"` for +a 4 GiB container limit; with defaults, heap plus direct memory, the 1.5 GiB reserve, and the 256 MiB minimum budget already exceed a 2 GiB limit. +See `rocksdb.total_memory_size` in the [HStore configuration reference](/docs/quickstart/hugegraph/hugegraph-hstore/). ### Server Variables diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index efd9a23147..2d12c7fd74 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -331,7 +331,11 @@ Daemon mode: - `-d true` (default): run as a background daemon. The script returns immediately and writes the Java pid to `bin/pid`. - `-d false`: run in the foreground. The script `exec`s Java, so the container or supervisor process is Java itself. Use this under Docker or a process supervisor (systemd, supervisord) so crashes are detected and the service is restarted automatically. -JVM memory, unless you set `JAVA_OPTIONS` yourself: `-Xms512m`, and `-Xmx` set to half the free memory, clamped to the 512 MB to 2048 MB range. The script also adds `-XX:MaxDirectMemorySize` equal to that heap, `-XX:MetaspaceSize=256M`, a heap dump on out-of-memory into `logs/`, and a rolling GC log at `logs/gc.log`. Production nodes normally need a much larger heap, so pass one explicitly together with a direct-memory cap, for example `-j "-Xmx32g -Xms32g -XX:MaxDirectMemorySize=2g"`. Without the cap the JVM allows as much direct memory as the 32 GB heap, the RocksDB budget counts it, and on a host with less than about 66 GB it falls to the 256 MiB floor. +JVM memory, unless you set `JAVA_OPTIONS` yourself: `-Xms512m`, and `-Xmx` set to half the free memory, clamped to the 512 MB to 2048 MB range. +The script also adds `-XX:MaxDirectMemorySize` equal to that heap, `-XX:MetaspaceSize=256M`, a heap dump on out-of-memory into `logs/`, and a rolling GC log at +`logs/gc.log`. Production nodes normally need a much larger heap, so pass one explicitly together with a direct-memory cap, for example +`-j "-Xmx32g -Xms32g -XX:MaxDirectMemorySize=2g"`. Without the cap the JVM allows as much direct memory as the 32 GB heap, the RocksDB budget counts it, +and on a host with less than about 66 GB it falls to the 256 MiB floor. The script refuses to start if `ulimit -n` or `ulimit -u` is below 1024, and it preloads jemalloc on x86_64 and arm64 when the shared object can be downloaded and verified. From 638a816e3e6f3c328d24fecd6cf89a0bf26daecd Mon Sep 17 00:00:00 2001 From: ayanx Date: Tue, 20 Oct 2026 13:09:12 +0530 Subject: [PATCH 10/23] docs(store): clarify the memory budget's preconditions and limits - Docker guide: leaving HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE unset auto-sizes only when the loaded rocksdb.total_memory_size is missing or 0; an application-pd.yml kept from an earlier release still sets 32000000000. Link the upgrade note. - HStore env row: an empty value counts as unset; only a non-empty value must be digits. - HStore config row: the budget is a sizing target, not a hard cap. With rocksdb.write_buffer_allow_stall at its default false, native memory can exceed it under heavy writes, so an OOM is still possible. EN and CN. --- content/cn/docs/guides/hugegraph-docker-cluster.md | 2 ++ content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md | 4 ++-- content/en/docs/guides/hugegraph-docker-cluster.md | 5 ++++- content/en/docs/quickstart/hugegraph/hugegraph-hstore.md | 4 ++-- 4 files changed, 10 insertions(+), 5 deletions(-) diff --git a/content/cn/docs/guides/hugegraph-docker-cluster.md b/content/cn/docs/guides/hugegraph-docker-cluster.md index f6dda9d098..14ebcd4056 100644 --- a/content/cn/docs/guides/hugegraph-docker-cluster.md +++ b/content/cn/docs/guides/hugegraph-docker-cluster.md @@ -202,6 +202,8 @@ curl -o /dev/null -w '%{http_code}\n' -u "admin:${HUGEGRAPH_ADMIN_PASSWORD}" \ > **已弃用的别名**:`PD_ADDRESS` → `HG_STORE_PD_ADDRESS`、`GRPC_HOST` → `HG_STORE_GRPC_HOST`、`RAFT_ADDRESS` → `HG_STORE_RAFT_ADDRESS` RocksDB 缓存位于 Java 堆外。不设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` 时,将按容器内存限制自动计算;若容器未设置限制,则按宿主机(或 Docker Desktop 虚拟机)内存计算。 +前提是加载到的 `rocksdb.total_memory_size` 缺失或为 `0`:沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,会保留该预算, +请按 [HStore 配置参考](/cn/docs/quickstart/hugegraph/hugegraph-hstore/)中 `rocksdb.total_memory_size` 一行的升级说明处理。 上面的 Compose 文件默认未设置内存限制,每个 Store 均会按整台宿主机估算预算;多 Store 共用单机时,建议为服务配置内存限制,或将该变量显式设为字节数。 需要注意,容器限制仅约束 RocksDB 预算:Store 启动脚本仍会按宿主机空闲内存自动分配 `-Xmx`(最多 2 GiB),并将直接内存设为相同大小。 因此请同步通过 `JAVA_OPTS`(镜像会以 `-j` 传给启动脚本)显式约束 JVM 内存,例如容器限制为 4 GiB 时可设置 `JAVA_OPTS="-Xmx1g -XX:MaxDirectMemorySize=512m"`; diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index a7e5e9b5f7..11a40d5876 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -131,7 +131,7 @@ docker run -d \ | `HG_STORE_GRPC_PORT` | 否 | `8500` | `grpc.port` | gRPC 服务端口。 | | `HG_STORE_REST_PORT` | 否 | `8520` | `server.port` | REST API 端口。 | | `HG_STORE_DATA_PATH` | 否 | `/hugegraph-store/storage` | `app.data-path` | 数据存储路径。 | -| `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` | 否 | 无 | `rocksdb.total_memory_size` | RocksDB 内存预算,单位字节,只能是数字;其他值会使入口脚本以退出码 2 退出。未设置时使用 `application-pd.yml` 中的值或自动计算。 | +| `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` | 否 | 无 | `rocksdb.total_memory_size` | RocksDB 内存预算,单位字节。空值视为未设置;非空值只能是数字,否则入口脚本以退出码 2 退出。未设置时使用 `application-pd.yml` 中的值或自动计算。 | {.td-table--wide} 当前主线 entrypoint 将这些变量写入 `SPRING_APPLICATION_JSON`,覆盖镜像内 `conf/application.yml`;其余配置仍从该文件和 `application-pd.yml` 读取,然后以前台方式启动 Java。旧变量 `PD_ADDRESS`、`GRPC_HOST`、`RAFT_ADDRESS` 仍可用,但会输出弃用警告。 @@ -208,7 +208,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ | 配置项 | 当前主线目录值 | 代码默认值 | 说明 | |--------|--------|------------|------| -| `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。JVM 和 cgroup 文件都拿不到可用的内存大小时,改用 JVM 最大堆,同样限制在 256 MiB 到 32 GB 之间。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | +| `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中。该预算是容量目标,并非硬上限:`rocksdb.write_buffer_allow_stall` 默认为 `false`,写缓存写满时不会阻塞写入,因此在大量写入时本地内存可能超出预算,容器仍可能被 OOM 终止。将其设为 `true` 能更严格地守住上限,但会增加写入延迟。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。JVM 和 cgroup 文件都拿不到可用的内存大小时,改用 JVM 最大堆,同样限制在 256 MiB 到 32 GB 之间。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | | `rocksdb.write_buffer_size` | `32000000` | `33554432` | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`,最小为 `1048576`(1 MiB)。 | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | 落盘前合并的 memtable 数量。 | | `rocksdb.write_buffer_ratio` | 未设置 | `0.66` | `total_memory_size` 中分配给写缓存的比例,其余作为 block cache。 | diff --git a/content/en/docs/guides/hugegraph-docker-cluster.md b/content/en/docs/guides/hugegraph-docker-cluster.md index 2eff9aeace..25a156ca07 100644 --- a/content/en/docs/guides/hugegraph-docker-cluster.md +++ b/content/en/docs/guides/hugegraph-docker-cluster.md @@ -202,7 +202,10 @@ The following tables describe current master entrypoints and Compose files, not > **Deprecated aliases**: `PD_ADDRESS` → `HG_STORE_PD_ADDRESS`, `GRPC_HOST` → `HG_STORE_GRPC_HOST`, `RAFT_ADDRESS` → `HG_STORE_RAFT_ADDRESS`. RocksDB caches live outside the Java heap. Leave `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` unset to size them from the container memory limit, or from the -memory of the Docker host (such as the Docker Desktop VM) when the container has no limit. The Compose files above set no limit, so each Store sizes its budget +memory of the Docker host (such as the Docker Desktop VM) when the container has no limit. That needs the loaded +`rocksdb.total_memory_size` to be missing or `0`: an `application-pd.yml` kept from an earlier release still sets `32000000000` +and keeps that budget, so follow the upgrade note under `rocksdb.total_memory_size` in the +[HStore configuration reference](/docs/quickstart/hugegraph/hugegraph-hstore/). The Compose files above set no limit, so each Store sizes its budget from the whole host; set a memory limit on Store services or set this variable to a plain number of bytes when several Stores share one host. A container limit bounds only the RocksDB budget: the Store launcher still picks `-Xmx` from the host's free memory (up to 2 GiB) and sets direct memory to the same size. Size those for the limit too through `JAVA_OPTS` (passed to the launcher as `-j`), for example `JAVA_OPTS="-Xmx1g -XX:MaxDirectMemorySize=512m"` for diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index 2d12c7fd74..9f6eadf75e 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -132,7 +132,7 @@ docker run -d \ | `HG_STORE_GRPC_PORT` | No | `8500` | `grpc.port` | gRPC service port. | | `HG_STORE_REST_PORT` | No | `8520` | `server.port` | REST API port. | | `HG_STORE_DATA_PATH` | No | `/hugegraph-store/storage` | `app.data-path` | Data path. | -| `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` | No | None | `rocksdb.total_memory_size` | RocksDB memory budget in bytes, digits only; other values stop the entrypoint with exit code 2. When unset, the value in `application-pd.yml` or auto-sizing applies. | +| `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` | No | None | `rocksdb.total_memory_size` | RocksDB memory budget in bytes. An empty value counts as unset; a non-empty value must be digits only, or the entrypoint exits with code 2. When unset, the value in `application-pd.yml` or auto-sizing applies. | {.td-table--wide} The master entrypoint writes these values into `SPRING_APPLICATION_JSON`, overriding image `conf/application.yml`. Other options still come from that file and `application-pd.yml`; Java runs in the foreground. Deprecated `PD_ADDRESS`, `GRPC_HOST`, and `RAFT_ADDRESS` remain supported with warnings. @@ -210,7 +210,7 @@ This file contains only `rocksdb` and Actuator exposure settings. | Key | Master distribution value | Code default | Meaning | |-----|---------|--------------|---------| -| `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. If neither the JVM nor the cgroup files report a usable memory size, the budget falls back to the JVM max heap, clamped to the same 256 MiB to 32 GB. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | +| `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top. The budget is a sizing target, not a hard cap: with `rocksdb.write_buffer_allow_stall` at its default `false`, writes are not stalled when the write cache is full, so native memory can exceed the budget under heavy writes and a container can still be OOM-killed. Setting it to `true` holds the cap more firmly at the cost of write latency. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. If neither the JVM nor the cgroup files report a usable memory size, the budget falls back to the JVM max heap, clamped to the same 256 MiB to 32 GB. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | | `rocksdb.write_buffer_size` | `32000000` | `33554432` | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`, at least `1048576` (1 MiB). | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | Number of memtables merged together before a flush. | | `rocksdb.write_buffer_ratio` | not set | `0.66` | Share of `total_memory_size` given to the write cache. The rest becomes the block cache. | From 4b6ceaef6c5a2aedc2f7f7ae6f79b0e434e31011 Mon Sep 17 00:00:00 2001 From: ayanx Date: Tue, 20 Oct 2026 22:56:12 +0530 Subject: [PATCH 11/23] docs(store): mark the write_buffer_size code default as derived The row listed 33554432 as the code default but its own text says an absent or 0 value is derived from total_memory_size, which is what AppConfig does before RocksDBOptions' default is ever used. Say "Derived, see Meaning", as the total_memory_size row already does. --- content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- content/en/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index 11a40d5876..1793d9c6e2 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -209,7 +209,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ | 配置项 | 当前主线目录值 | 代码默认值 | 说明 | |--------|--------|------------|------| | `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中。该预算是容量目标,并非硬上限:`rocksdb.write_buffer_allow_stall` 默认为 `false`,写缓存写满时不会阻塞写入,因此在大量写入时本地内存可能超出预算,容器仍可能被 OOM 终止。将其设为 `true` 能更严格地守住上限,但会增加写入延迟。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。JVM 和 cgroup 文件都拿不到可用的内存大小时,改用 JVM 最大堆,同样限制在 256 MiB 到 32 GB 之间。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | -| `rocksdb.write_buffer_size` | `32000000` | `33554432` | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`,最小为 `1048576`(1 MiB)。 | +| `rocksdb.write_buffer_size` | `32000000` | 自动推导,见说明 | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`,最小为 `1048576`(1 MiB)。 | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | 落盘前合并的 memtable 数量。 | | `rocksdb.write_buffer_ratio` | 未设置 | `0.66` | `total_memory_size` 中分配给写缓存的比例,其余作为 block cache。 | {.td-table--wide} diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index 9f6eadf75e..b64d13d3b7 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -211,7 +211,7 @@ This file contains only `rocksdb` and Actuator exposure settings. | Key | Master distribution value | Code default | Meaning | |-----|---------|--------------|---------| | `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top. The budget is a sizing target, not a hard cap: with `rocksdb.write_buffer_allow_stall` at its default `false`, writes are not stalled when the write cache is full, so native memory can exceed the budget under heavy writes and a container can still be OOM-killed. Setting it to `true` holds the cap more firmly at the cost of write latency. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. If neither the JVM nor the cgroup files report a usable memory size, the budget falls back to the JVM max heap, clamped to the same 256 MiB to 32 GB. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | -| `rocksdb.write_buffer_size` | `32000000` | `33554432` | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`, at least `1048576` (1 MiB). | +| `rocksdb.write_buffer_size` | `32000000` | Derived, see Meaning | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`, at least `1048576` (1 MiB). | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | Number of memtables merged together before a flush. | | `rocksdb.write_buffer_ratio` | not set | `0.66` | Share of `total_memory_size` given to the write cache. The rest becomes the block cache. | {.td-table--wide} From 29ae9238844c4522d5e5b9c4cd9b92ad6d352230 Mon Sep 17 00:00:00 2001 From: ayanx Date: Wed, 21 Oct 2026 12:53:36 +0530 Subject: [PATCH 12/23] docs(store): cover the application-pd.yml copy that PD keeps In PD mode the first Store uploads its application-pd.yml to PD (KV key S:FS), and every Store whose local file isn't newer loads that copy and writes it over its own. An old 32000000000 there survives a Store upgrade, so the upgrade note now says to change it in PD's copy or set the per-Store override, matching apache/hugegraph#3355. EN and CN. --- content/cn/docs/guides/hugegraph-docker-cluster.md | 2 +- content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- content/en/docs/guides/hugegraph-docker-cluster.md | 2 +- content/en/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/content/cn/docs/guides/hugegraph-docker-cluster.md b/content/cn/docs/guides/hugegraph-docker-cluster.md index 14ebcd4056..010b7f018a 100644 --- a/content/cn/docs/guides/hugegraph-docker-cluster.md +++ b/content/cn/docs/guides/hugegraph-docker-cluster.md @@ -202,7 +202,7 @@ curl -o /dev/null -w '%{http_code}\n' -u "admin:${HUGEGRAPH_ADMIN_PASSWORD}" \ > **已弃用的别名**:`PD_ADDRESS` → `HG_STORE_PD_ADDRESS`、`GRPC_HOST` → `HG_STORE_GRPC_HOST`、`RAFT_ADDRESS` → `HG_STORE_RAFT_ADDRESS` RocksDB 缓存位于 Java 堆外。不设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` 时,将按容器内存限制自动计算;若容器未设置限制,则按宿主机(或 Docker Desktop 虚拟机)内存计算。 -前提是加载到的 `rocksdb.total_memory_size` 缺失或为 `0`:沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,会保留该预算, +前提是加载到的 `rocksdb.total_memory_size` 缺失或为 `0`:沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,会保留该预算,PD 保存并下发给各 Store 的副本也是如此, 请按 [HStore 配置参考](/cn/docs/quickstart/hugegraph/hugegraph-hstore/)中 `rocksdb.total_memory_size` 一行的升级说明处理。 上面的 Compose 文件默认未设置内存限制,每个 Store 均会按整台宿主机估算预算;多 Store 共用单机时,建议为服务配置内存限制,或将该变量显式设为字节数。 需要注意,容器限制仅约束 RocksDB 预算:Store 启动脚本仍会按宿主机空闲内存自动分配 `-Xmx`(最多 2 GiB),并将直接内存设为相同大小。 diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index 1793d9c6e2..16cb980ecb 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -208,7 +208,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ | 配置项 | 当前主线目录值 | 代码默认值 | 说明 | |--------|--------|------------|------| -| `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中。该预算是容量目标,并非硬上限:`rocksdb.write_buffer_allow_stall` 默认为 `false`,写缓存写满时不会阻塞写入,因此在大量写入时本地内存可能超出预算,容器仍可能被 OOM 终止。将其设为 `true` 能更严格地守住上限,但会增加写入延迟。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。JVM 和 cgroup 文件都拿不到可用的内存大小时,改用 JVM 最大堆,同样限制在 256 MiB 到 32 GB 之间。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | +| `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中。该预算是容量目标,并非硬上限:`rocksdb.write_buffer_allow_stall` 默认为 `false`,写缓存写满时不会阻塞写入,因此在大量写入时本地内存可能超出预算,容器仍可能被 OOM 终止。将其设为 `true` 能更严格地守住上限,但会增加写入延迟。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。JVM 和 cgroup 文件都拿不到可用的内存大小时,改用 JVM 最大堆,同样限制在 256 MiB 到 32 GB 之间。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。PD 模式下还要检查 PD 保存的副本(KV 键 `S:FS`):它由第一个启动的 Store 上传,此后本地文件不比它新的 Store 都会加载它并覆盖本地文件。请在 PD 的副本中修改该值,或在每个 Store 上设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` 或 `-Drocksdb.total_memory_size`,Store 会让这个值优先于 PD 的副本。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | | `rocksdb.write_buffer_size` | `32000000` | 自动推导,见说明 | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`,最小为 `1048576`(1 MiB)。 | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | 落盘前合并的 memtable 数量。 | | `rocksdb.write_buffer_ratio` | 未设置 | `0.66` | `total_memory_size` 中分配给写缓存的比例,其余作为 block cache。 | diff --git a/content/en/docs/guides/hugegraph-docker-cluster.md b/content/en/docs/guides/hugegraph-docker-cluster.md index 25a156ca07..11133eae49 100644 --- a/content/en/docs/guides/hugegraph-docker-cluster.md +++ b/content/en/docs/guides/hugegraph-docker-cluster.md @@ -204,7 +204,7 @@ The following tables describe current master entrypoints and Compose files, not RocksDB caches live outside the Java heap. Leave `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` unset to size them from the container memory limit, or from the memory of the Docker host (such as the Docker Desktop VM) when the container has no limit. That needs the loaded `rocksdb.total_memory_size` to be missing or `0`: an `application-pd.yml` kept from an earlier release still sets `32000000000` -and keeps that budget, so follow the upgrade note under `rocksdb.total_memory_size` in the +and keeps that budget, as does the copy PD keeps and hands to every Store, so follow the upgrade note under `rocksdb.total_memory_size` in the [HStore configuration reference](/docs/quickstart/hugegraph/hugegraph-hstore/). The Compose files above set no limit, so each Store sizes its budget from the whole host; set a memory limit on Store services or set this variable to a plain number of bytes when several Stores share one host. A container limit bounds only the RocksDB budget: the Store launcher still picks `-Xmx` from the host's free memory (up to 2 GiB) and sets direct memory to the diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index b64d13d3b7..140dcef7ce 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -210,7 +210,7 @@ This file contains only `rocksdb` and Actuator exposure settings. | Key | Master distribution value | Code default | Meaning | |-----|---------|--------------|---------| -| `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top. The budget is a sizing target, not a hard cap: with `rocksdb.write_buffer_allow_stall` at its default `false`, writes are not stalled when the write cache is full, so native memory can exceed the budget under heavy writes and a container can still be OOM-killed. Setting it to `true` holds the cap more firmly at the cost of write latency. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. If neither the JVM nor the cgroup files report a usable memory size, the budget falls back to the JVM max heap, clamped to the same 256 MiB to 32 GB. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | +| `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top. The budget is a sizing target, not a hard cap: with `rocksdb.write_buffer_allow_stall` at its default `false`, writes are not stalled when the write cache is full, so native memory can exceed the budget under heavy writes and a container can still be OOM-killed. Setting it to `true` holds the cap more firmly at the cost of write latency. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. If neither the JVM nor the cgroup files report a usable memory size, the budget falls back to the JVM max heap, clamped to the same 256 MiB to 32 GB. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. In PD mode also check the copy PD keeps (KV key `S:FS`), which the first Store uploads and every Store whose local file isn't newer then loads and writes over its local file: change the value there, or set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` or `-Drocksdb.total_memory_size` on each Store, which the Store keeps above PD's copy. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | | `rocksdb.write_buffer_size` | `32000000` | Derived, see Meaning | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`, at least `1048576` (1 MiB). | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | Number of memtables merged together before a flush. | | `rocksdb.write_buffer_ratio` | not set | `0.66` | Share of `total_memory_size` given to the write cache. The rest becomes the block cache. | From df36c542483b62b6560f958cc398a64600f41792 Mon Sep 17 00:00:00 2001 From: ayanx Date: Thu, 22 Oct 2026 11:54:32 +0530 Subject: [PATCH 13/23] docs(store): note that Raft log memtables are charged to the log cache apache/hugegraph#3355 now gives the Raft log DBs one shared 256 MiB write buffer charged to the fixed 1 GiB Raft log cache, so the memtables of all partitions don't add to it. --- content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- content/en/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index 16cb980ecb..f91b88da76 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -208,7 +208,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ | 配置项 | 当前主线目录值 | 代码默认值 | 说明 | |--------|--------|------------|------| -| `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中。该预算是容量目标,并非硬上限:`rocksdb.write_buffer_allow_stall` 默认为 `false`,写缓存写满时不会阻塞写入,因此在大量写入时本地内存可能超出预算,容器仍可能被 OOM 终止。将其设为 `true` 能更严格地守住上限,但会增加写入延迟。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。JVM 和 cgroup 文件都拿不到可用的内存大小时,改用 JVM 最大堆,同样限制在 256 MiB 到 32 GB 之间。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。PD 模式下还要检查 PD 保存的副本(KV 键 `S:FS`):它由第一个启动的 Store 上传,此后本地文件不比它新的 Store 都会加载它并覆盖本地文件。请在 PD 的副本中修改该值,或在每个 Store 上设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` 或 `-Drocksdb.total_memory_size`,Store 会让这个值优先于 PD 的副本。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | +| `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中;所有分区的 Raft 日志 memtable 共用一个 256 MiB 的写缓冲,并计入这 1 GiB 缓存,不会额外占用内存。该预算是容量目标,并非硬上限:`rocksdb.write_buffer_allow_stall` 默认为 `false`,写缓存写满时不会阻塞写入,因此在大量写入时本地内存可能超出预算,容器仍可能被 OOM 终止。将其设为 `true` 能更严格地守住上限,但会增加写入延迟。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。JVM 和 cgroup 文件都拿不到可用的内存大小时,改用 JVM 最大堆,同样限制在 256 MiB 到 32 GB 之间。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。PD 模式下还要检查 PD 保存的副本(KV 键 `S:FS`):它由第一个启动的 Store 上传,此后本地文件不比它新的 Store 都会加载它并覆盖本地文件。请在 PD 的副本中修改该值,或在每个 Store 上设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` 或 `-Drocksdb.total_memory_size`,Store 会让这个值优先于 PD 的副本。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | | `rocksdb.write_buffer_size` | `32000000` | 自动推导,见说明 | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`,最小为 `1048576`(1 MiB)。 | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | 落盘前合并的 memtable 数量。 | | `rocksdb.write_buffer_ratio` | 未设置 | `0.66` | `total_memory_size` 中分配给写缓存的比例,其余作为 block cache。 | diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index 140dcef7ce..92522ff2d2 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -210,7 +210,7 @@ This file contains only `rocksdb` and Actuator exposure settings. | Key | Master distribution value | Code default | Meaning | |-----|---------|--------------|---------| -| `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top. The budget is a sizing target, not a hard cap: with `rocksdb.write_buffer_allow_stall` at its default `false`, writes are not stalled when the write cache is full, so native memory can exceed the budget under heavy writes and a container can still be OOM-killed. Setting it to `true` holds the cap more firmly at the cost of write latency. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. If neither the JVM nor the cgroup files report a usable memory size, the budget falls back to the JVM max heap, clamped to the same 256 MiB to 32 GB. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. In PD mode also check the copy PD keeps (KV key `S:FS`), which the first Store uploads and every Store whose local file isn't newer then loads and writes over its local file: change the value there, or set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` or `-Drocksdb.total_memory_size` on each Store, which the Store keeps above PD's copy. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | +| `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top; the Raft log memtables of all partitions share a 256 MiB write buffer charged to that cache, so they don't add to it. The budget is a sizing target, not a hard cap: with `rocksdb.write_buffer_allow_stall` at its default `false`, writes are not stalled when the write cache is full, so native memory can exceed the budget under heavy writes and a container can still be OOM-killed. Setting it to `true` holds the cap more firmly at the cost of write latency. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. If neither the JVM nor the cgroup files report a usable memory size, the budget falls back to the JVM max heap, clamped to the same 256 MiB to 32 GB. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. In PD mode also check the copy PD keeps (KV key `S:FS`), which the first Store uploads and every Store whose local file isn't newer then loads and writes over its local file: change the value there, or set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` or `-Drocksdb.total_memory_size` on each Store, which the Store keeps above PD's copy. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | | `rocksdb.write_buffer_size` | `32000000` | Derived, see Meaning | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`, at least `1048576` (1 MiB). | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | Number of memtables merged together before a flush. | | `rocksdb.write_buffer_ratio` | not set | `0.66` | Share of `total_memory_size` given to the write cache. The rest becomes the block cache. | From e964a687c5c73f18b6e5fb2bc7d547b5809d4f3f Mon Sep 17 00:00:00 2001 From: ayanx Date: Fri, 23 Oct 2026 22:55:23 +0530 Subject: [PATCH 14/23] docs(store): say how a PD-held budget actually gets applied A Store loads PD's S:FS copy only when it has no local file or the file is no newer than S:Timestamp, which the first upload never sets. Editing S:FS alone therefore changes nothing. Recommend the per-Store override set to 0, and describe the shared-copy route: change S:FS, then publish S:Timestamp so running Stores reload and restart. --- content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- content/en/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index f91b88da76..00e1d11bf1 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -208,7 +208,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ | 配置项 | 当前主线目录值 | 代码默认值 | 说明 | |--------|--------|------------|------| -| `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中;所有分区的 Raft 日志 memtable 共用一个 256 MiB 的写缓冲,并计入这 1 GiB 缓存,不会额外占用内存。该预算是容量目标,并非硬上限:`rocksdb.write_buffer_allow_stall` 默认为 `false`,写缓存写满时不会阻塞写入,因此在大量写入时本地内存可能超出预算,容器仍可能被 OOM 终止。将其设为 `true` 能更严格地守住上限,但会增加写入延迟。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。JVM 和 cgroup 文件都拿不到可用的内存大小时,改用 JVM 最大堆,同样限制在 256 MiB 到 32 GB 之间。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。PD 模式下还要检查 PD 保存的副本(KV 键 `S:FS`):它由第一个启动的 Store 上传,此后本地文件不比它新的 Store 都会加载它并覆盖本地文件。请在 PD 的副本中修改该值,或在每个 Store 上设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` 或 `-Drocksdb.total_memory_size`,Store 会让这个值优先于 PD 的副本。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | +| `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中;所有分区的 Raft 日志 memtable 共用一个 256 MiB 的写缓冲,并计入这 1 GiB 缓存,不会额外占用内存。该预算是容量目标,并非硬上限:`rocksdb.write_buffer_allow_stall` 默认为 `false`,写缓存写满时不会阻塞写入,因此在大量写入时本地内存可能超出预算,容器仍可能被 OOM 终止。将其设为 `true` 能更严格地守住上限,但会增加写入延迟。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。JVM 和 cgroup 文件都拿不到可用的内存大小时,改用 JVM 最大堆,同样限制在 256 MiB 到 32 GB 之间。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。PD 模式下 PD 还可能保存一份副本(KV 键 `S:FS`),由第一个启动的 Store 上传。Store 只有在没有本地文件,或本地文件不比 `S:Timestamp` 新时,才会加载这份副本并覆盖本地文件;上传时不会写入时间戳,缺失的时间戳按 0 处理,因此只修改 `S:FS` 不会生效。升级时,请在每个 Store 上设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=0` 或 `-Drocksdb.total_memory_size=0` 并重启,Store 会让这个值优先于本地文件和 PD 的副本。也可以先修改 `S:FS` 中的值,再通过 PD 的 KV 服务(gRPC `KvService.put`)把当前时间(毫秒)写入 `S:Timestamp`:所有运行中的 Store 会重新加载 PD 的副本、覆盖本地文件,并在 60 秒内通过 `bin/restart-hugegraph-store.sh` 自行重启;已停止的 Store 会在下次启动时加载它。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | | `rocksdb.write_buffer_size` | `32000000` | 自动推导,见说明 | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`,最小为 `1048576`(1 MiB)。 | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | 落盘前合并的 memtable 数量。 | | `rocksdb.write_buffer_ratio` | 未设置 | `0.66` | `total_memory_size` 中分配给写缓存的比例,其余作为 block cache。 | diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index 92522ff2d2..40aeea150f 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -210,7 +210,7 @@ This file contains only `rocksdb` and Actuator exposure settings. | Key | Master distribution value | Code default | Meaning | |-----|---------|--------------|---------| -| `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top; the Raft log memtables of all partitions share a 256 MiB write buffer charged to that cache, so they don't add to it. The budget is a sizing target, not a hard cap: with `rocksdb.write_buffer_allow_stall` at its default `false`, writes are not stalled when the write cache is full, so native memory can exceed the budget under heavy writes and a container can still be OOM-killed. Setting it to `true` holds the cap more firmly at the cost of write latency. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. If neither the JVM nor the cgroup files report a usable memory size, the budget falls back to the JVM max heap, clamped to the same 256 MiB to 32 GB. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. In PD mode also check the copy PD keeps (KV key `S:FS`), which the first Store uploads and every Store whose local file isn't newer then loads and writes over its local file: change the value there, or set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` or `-Drocksdb.total_memory_size` on each Store, which the Store keeps above PD's copy. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | +| `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top; the Raft log memtables of all partitions share a 256 MiB write buffer charged to that cache, so they don't add to it. The budget is a sizing target, not a hard cap: with `rocksdb.write_buffer_allow_stall` at its default `false`, writes are not stalled when the write cache is full, so native memory can exceed the budget under heavy writes and a container can still be OOM-killed. Setting it to `true` holds the cap more firmly at the cost of write latency. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. If neither the JVM nor the cgroup files report a usable memory size, the budget falls back to the JVM max heap, clamped to the same 256 MiB to 32 GB. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. In PD mode PD may also keep a copy (KV key `S:FS`), uploaded by the first Store to start. A Store loads that copy over its local file only when it has no local file or the file is no newer than `S:Timestamp`; the upload sets no timestamp, and a missing one counts as 0, so changing `S:FS` alone changes nothing. To upgrade, set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=0` or `-Drocksdb.total_memory_size=0` on each Store and restart it; the Store keeps this value above both its local file and PD's copy. Alternatively, change the value in `S:FS`, then put the current time in milliseconds into `S:Timestamp` through PD's KV service (gRPC `KvService.put`): every running Store then reloads PD's copy, writes it over its local file and restarts itself with `bin/restart-hugegraph-store.sh` within 60 seconds, and a stopped Store loads it at its next start. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | | `rocksdb.write_buffer_size` | `32000000` | Derived, see Meaning | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`, at least `1048576` (1 MiB). | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | Number of memtables merged together before a flush. | | `rocksdb.write_buffer_ratio` | not set | `0.66` | Share of `total_memory_size` given to the write cache. The rest becomes the block cache. | From 64448e8df33ba6547418cbe6b3024b315c4e25e1 Mon Sep 17 00:00:00 2001 From: ayanx Date: Sat, 24 Oct 2026 00:42:49 +0530 Subject: [PATCH 15/23] ci: re-run the site build From 13737599ca36d67c776ce3ea4fdbc470342eb02c Mon Sep 17 00:00:00 2001 From: ayanx Date: Mon, 26 Oct 2026 22:41:35 +0530 Subject: [PATCH 16/23] docs(store): document auto for rocksdb.total_memory_size apache/hugegraph#3355 now ships total_memory_size: auto, keeps a missing, blank or 0 value as aliases, and lets HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE take auto. The reference rows and the Docker cluster guide say so, in English and Chinese, along with the reserve split, how a limit the Store can't see is handled, and what the startup log reports. --- content/cn/docs/guides/hugegraph-docker-cluster.md | 4 ++-- content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md | 4 ++-- content/en/docs/guides/hugegraph-docker-cluster.md | 4 ++-- content/en/docs/quickstart/hugegraph/hugegraph-hstore.md | 4 ++-- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/content/cn/docs/guides/hugegraph-docker-cluster.md b/content/cn/docs/guides/hugegraph-docker-cluster.md index 010b7f018a..168c489a13 100644 --- a/content/cn/docs/guides/hugegraph-docker-cluster.md +++ b/content/cn/docs/guides/hugegraph-docker-cluster.md @@ -196,13 +196,13 @@ curl -o /dev/null -w '%{http_code}\n' -u "admin:${HUGEGRAPH_ADMIN_PASSWORD}" \ | `HG_STORE_GRPC_PORT` | 否 | `8500` | `grpc.port` | | `HG_STORE_REST_PORT` | 否 | `8520` | `server.port` | | `HG_STORE_DATA_PATH` | 否 | `/hugegraph-store/storage` | `app.data-path` | -| `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` | 否 | 无 | `rocksdb.total_memory_size`,单位字节 | +| `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` | 否 | 无 | `rocksdb.total_memory_size`,`auto` 或字节数 | {.td-table--wide} > **已弃用的别名**:`PD_ADDRESS` → `HG_STORE_PD_ADDRESS`、`GRPC_HOST` → `HG_STORE_GRPC_HOST`、`RAFT_ADDRESS` → `HG_STORE_RAFT_ADDRESS` RocksDB 缓存位于 Java 堆外。不设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` 时,将按容器内存限制自动计算;若容器未设置限制,则按宿主机(或 Docker Desktop 虚拟机)内存计算。 -前提是加载到的 `rocksdb.total_memory_size` 缺失或为 `0`:沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,会保留该预算,PD 保存并下发给各 Store 的副本也是如此, +前提是加载到的 `rocksdb.total_memory_size` 为 `auto`(或缺失、为 `0`):沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,会保留该预算,PD 保存并下发给各 Store 的副本也是如此, 请按 [HStore 配置参考](/cn/docs/quickstart/hugegraph/hugegraph-hstore/)中 `rocksdb.total_memory_size` 一行的升级说明处理。 上面的 Compose 文件默认未设置内存限制,每个 Store 均会按整台宿主机估算预算;多 Store 共用单机时,建议为服务配置内存限制,或将该变量显式设为字节数。 需要注意,容器限制仅约束 RocksDB 预算:Store 启动脚本仍会按宿主机空闲内存自动分配 `-Xmx`(最多 2 GiB),并将直接内存设为相同大小。 diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index 00e1d11bf1..d641d642d4 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -131,7 +131,7 @@ docker run -d \ | `HG_STORE_GRPC_PORT` | 否 | `8500` | `grpc.port` | gRPC 服务端口。 | | `HG_STORE_REST_PORT` | 否 | `8520` | `server.port` | REST API 端口。 | | `HG_STORE_DATA_PATH` | 否 | `/hugegraph-store/storage` | `app.data-path` | 数据存储路径。 | -| `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` | 否 | 无 | `rocksdb.total_memory_size` | RocksDB 内存预算,单位字节。空值视为未设置;非空值只能是数字,否则入口脚本以退出码 2 退出。未设置时使用 `application-pd.yml` 中的值或自动计算。 | +| `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` | 否 | 无 | `rocksdb.total_memory_size` | RocksDB 内存预算:`auto` 或字节数。空值视为未设置;其他值会使入口脚本以退出码 2 退出。未设置时使用 `application-pd.yml` 中的值或自动计算。 | {.td-table--wide} 当前主线 entrypoint 将这些变量写入 `SPRING_APPLICATION_JSON`,覆盖镜像内 `conf/application.yml`;其余配置仍从该文件和 `application-pd.yml` 读取,然后以前台方式启动 Java。旧变量 `PD_ADDRESS`、`GRPC_HOST`、`RAFT_ADDRESS` 仍可用,但会输出弃用警告。 @@ -208,7 +208,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ | 配置项 | 当前主线目录值 | 代码默认值 | 说明 | |--------|--------|------------|------| -| `rocksdb.total_memory_size` | `0` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,单位字节,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中;所有分区的 Raft 日志 memtable 共用一个 256 MiB 的写缓冲,并计入这 1 GiB 缓存,不会额外占用内存。该预算是容量目标,并非硬上限:`rocksdb.write_buffer_allow_stall` 默认为 `false`,写缓存写满时不会阻塞写入,因此在大量写入时本地内存可能超出预算,容器仍可能被 OOM 终止。将其设为 `true` 能更严格地守住上限,但会增加写入延迟。缺失或为 `0` 时,取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。JVM 和 cgroup 文件都拿不到可用的内存大小时,改用 JVM 最大堆,同样限制在 256 MiB 到 32 GB 之间。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `0` 才会自动计算。PD 模式下 PD 还可能保存一份副本(KV 键 `S:FS`),由第一个启动的 Store 上传。Store 只有在没有本地文件,或本地文件不比 `S:Timestamp` 新时,才会加载这份副本并覆盖本地文件;上传时不会写入时间戳,缺失的时间戳按 0 处理,因此只修改 `S:FS` 不会生效。升级时,请在每个 Store 上设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=0` 或 `-Drocksdb.total_memory_size=0` 并重启,Store 会让这个值优先于本地文件和 PD 的副本。也可以先修改 `S:FS` 中的值,再通过 PD 的 KV 服务(gRPC `KvService.put`)把当前时间(毫秒)写入 `S:Timestamp`:所有运行中的 Store 会重新加载 PD 的副本、覆盖本地文件,并在 60 秒内通过 `bin/restart-hugegraph-store.sh` 自行重启;已停止的 Store 会在下次启动时加载它。正数按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | +| `rocksdb.total_memory_size` | `auto` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,取值为 `auto` 或字节数,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中;所有分区的 Raft 日志 memtable 共用一个 256 MiB 的写缓冲,并计入这 1 GiB 缓存,不会额外占用内存。该预算是容量目标,并非硬上限:`rocksdb.write_buffer_allow_stall` 默认为 `false`,写缓存写满时不会阻塞写入,因此在大量写入时本地内存可能超出预算,容器仍可能被 OOM 终止。将其设为 `true` 能更严格地守住上限,但会增加写入延迟。取值为 `auto` 时(缺失、空值或 `0` 与之等同),取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留(1 GiB Raft 日志缓存加 512 MiB 其他本地内存)后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。Store 看不到的限制(例如私有 cgroup 命名空间之上的限制)不会被猜测:Store 使用可见的最低限制并输出警告,此时请显式设置该值。启动日志会输出预算及其模式、物理内存限制及来源、堆、直接内存、预留,以及 cgroup 视图是否完整。JVM 和 cgroup 文件都拿不到可用的内存大小时,改用 JVM 最大堆,同样限制在 256 MiB 到 32 GB 之间。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `auto` 才会自动计算。PD 模式下 PD 还可能保存一份副本(KV 键 `S:FS`),由第一个启动的 Store 上传。Store 只有在没有本地文件,或本地文件不比 `S:Timestamp` 新时,才会加载这份副本并覆盖本地文件;上传时不会写入时间戳,缺失的时间戳按 0 处理,因此只修改 `S:FS` 不会生效。升级时,请在每个 Store 上设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` 或 `-Drocksdb.total_memory_size=auto` 并重启,Store 会让这个值优先于本地文件和 PD 的副本。也可以先修改 `S:FS` 中的值,再通过 PD 的 KV 服务(gRPC `KvService.put`)把当前时间(毫秒)写入 `S:Timestamp`:所有运行中的 Store 会重新加载 PD 的副本、覆盖本地文件,并在 60 秒内通过 `bin/restart-hugegraph-store.sh` 自行重启;已停止的 Store 会在下次启动时加载它。其他值按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | | `rocksdb.write_buffer_size` | `32000000` | 自动推导,见说明 | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`,最小为 `1048576`(1 MiB)。 | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | 落盘前合并的 memtable 数量。 | | `rocksdb.write_buffer_ratio` | 未设置 | `0.66` | `total_memory_size` 中分配给写缓存的比例,其余作为 block cache。 | diff --git a/content/en/docs/guides/hugegraph-docker-cluster.md b/content/en/docs/guides/hugegraph-docker-cluster.md index 11133eae49..fd1e5f0a47 100644 --- a/content/en/docs/guides/hugegraph-docker-cluster.md +++ b/content/en/docs/guides/hugegraph-docker-cluster.md @@ -196,14 +196,14 @@ The following tables describe current master entrypoints and Compose files, not | `HG_STORE_GRPC_PORT` | No | `8500` | `grpc.port` | | `HG_STORE_REST_PORT` | No | `8520` | `server.port` | | `HG_STORE_DATA_PATH` | No | `/hugegraph-store/storage` | `app.data-path` | -| `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` | No | None | `rocksdb.total_memory_size`, in bytes | +| `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` | No | None | `rocksdb.total_memory_size`, `auto` or bytes | {.td-table--wide} > **Deprecated aliases**: `PD_ADDRESS` → `HG_STORE_PD_ADDRESS`, `GRPC_HOST` → `HG_STORE_GRPC_HOST`, `RAFT_ADDRESS` → `HG_STORE_RAFT_ADDRESS`. RocksDB caches live outside the Java heap. Leave `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` unset to size them from the container memory limit, or from the memory of the Docker host (such as the Docker Desktop VM) when the container has no limit. That needs the loaded -`rocksdb.total_memory_size` to be missing or `0`: an `application-pd.yml` kept from an earlier release still sets `32000000000` +`rocksdb.total_memory_size` to be `auto` (or missing, or `0`): an `application-pd.yml` kept from an earlier release still sets `32000000000` and keeps that budget, as does the copy PD keeps and hands to every Store, so follow the upgrade note under `rocksdb.total_memory_size` in the [HStore configuration reference](/docs/quickstart/hugegraph/hugegraph-hstore/). The Compose files above set no limit, so each Store sizes its budget from the whole host; set a memory limit on Store services or set this variable to a plain number of bytes when several Stores share one host. diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index 40aeea150f..669c125541 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -132,7 +132,7 @@ docker run -d \ | `HG_STORE_GRPC_PORT` | No | `8500` | `grpc.port` | gRPC service port. | | `HG_STORE_REST_PORT` | No | `8520` | `server.port` | REST API port. | | `HG_STORE_DATA_PATH` | No | `/hugegraph-store/storage` | `app.data-path` | Data path. | -| `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` | No | None | `rocksdb.total_memory_size` | RocksDB memory budget in bytes. An empty value counts as unset; a non-empty value must be digits only, or the entrypoint exits with code 2. When unset, the value in `application-pd.yml` or auto-sizing applies. | +| `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` | No | None | `rocksdb.total_memory_size` | RocksDB memory budget: `auto` or a number of bytes. An empty value counts as unset; any other value makes the entrypoint exit with code 2. When unset, the value in `application-pd.yml` or auto-sizing applies. | {.td-table--wide} The master entrypoint writes these values into `SPRING_APPLICATION_JSON`, overriding image `conf/application.yml`. Other options still come from that file and `application-pd.yml`; Java runs in the foreground. Deprecated `PD_ADDRESS`, `GRPC_HOST`, and `RAFT_ADDRESS` remain supported with warnings. @@ -210,7 +210,7 @@ This file contains only `rocksdb` and Actuator exposure settings. | Key | Master distribution value | Code default | Meaning | |-----|---------|--------------|---------| -| `rocksdb.total_memory_size` | `0` | Auto-sized, see Meaning | Memory budget in bytes shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top; the Raft log memtables of all partitions share a 256 MiB write buffer charged to that cache, so they don't add to it. The budget is a sizing target, not a hard cap: with `rocksdb.write_buffer_allow_stall` at its default `false`, writes are not stalled when the write cache is full, so native memory can exceed the budget under heavy writes and a container can still be OOM-killed. Setting it to `true` holds the cap more firmly at the cost of write latency. When absent or `0`, the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve, clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. If neither the JVM nor the cgroup files report a usable memory size, the budget falls back to the JVM max heap, clamped to the same 256 MiB to 32 GB. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `0` to auto-size. In PD mode PD may also keep a copy (KV key `S:FS`), uploaded by the first Store to start. A Store loads that copy over its local file only when it has no local file or the file is no newer than `S:Timestamp`; the upload sets no timestamp, and a missing one counts as 0, so changing `S:FS` alone changes nothing. To upgrade, set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=0` or `-Drocksdb.total_memory_size=0` on each Store and restart it; the Store keeps this value above both its local file and PD's copy. Alternatively, change the value in `S:FS`, then put the current time in milliseconds into `S:Timestamp` through PD's KV service (gRPC `KvService.put`): every running Store then reloads PD's copy, writes it over its local file and restarts itself with `bin/restart-hugegraph-store.sh` within 60 seconds, and a stopped Store loads it at its next start. A positive value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | +| `rocksdb.total_memory_size` | `auto` | Auto-sized, see Meaning | Memory budget, `auto` or a number of bytes, shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top; the Raft log memtables of all partitions share a 256 MiB write buffer charged to that cache, so they don't add to it. The budget is a sizing target, not a hard cap: with `rocksdb.write_buffer_allow_stall` at its default `false`, writes are not stalled when the write cache is full, so native memory can exceed the budget under heavy writes and a container can still be OOM-killed. Setting it to `true` holds the cap more firmly at the cost of write latency. With `auto` (a missing, blank or `0` value means the same), the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve (the 1 GiB Raft log cache plus 512 MiB for other native memory), clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. A limit the Store can't see, such as one above a private cgroup namespace, isn't guessed: the Store uses the lowest visible limit and logs a warning, so pin the value there. The startup log shows the budget with its mode, physical limit and source, heap, direct memory, reserve and whether the cgroup view is complete. If neither the JVM nor the cgroup files report a usable memory size, the budget falls back to the JVM max heap, clamped to the same 256 MiB to 32 GB. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `auto` to auto-size. In PD mode PD may also keep a copy (KV key `S:FS`), uploaded by the first Store to start. A Store loads that copy over its local file only when it has no local file or the file is no newer than `S:Timestamp`; the upload sets no timestamp, and a missing one counts as 0, so changing `S:FS` alone changes nothing. To upgrade, set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` or `-Drocksdb.total_memory_size=auto` on each Store and restart it; the Store keeps this value above both its local file and PD's copy. Alternatively, change the value in `S:FS`, then put the current time in milliseconds into `S:Timestamp` through PD's KV service (gRPC `KvService.put`): every running Store then reloads PD's copy, writes it over its local file and restarts itself with `bin/restart-hugegraph-store.sh` within 60 seconds, and a stopped Store loads it at its next start. Any other value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | | `rocksdb.write_buffer_size` | `32000000` | Derived, see Meaning | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`, at least `1048576` (1 MiB). | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | Number of memtables merged together before a flush. | | `rocksdb.write_buffer_ratio` | not set | `0.66` | Share of `total_memory_size` given to the write cache. The rest becomes the block cache. | From 221df8ee53ac5acb0e8c265c827794e003188afb Mon Sep 17 00:00:00 2001 From: ayanx Date: Tue, 27 Oct 2026 14:05:09 +0530 Subject: [PATCH 17/23] docs(store): keep S:FS readable by earlier Stores and note the restart flags Recommend 0 or removing the key in S:FS until every Store supports auto, since an earlier release fails to start on auto. Note that the S:Timestamp self-restart drops -j and -g, including the Docker JAVA_OPTS, so only an exported JAVA_OPTIONS carries over. Both in EN and CN. --- content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- content/en/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index d641d642d4..de1b30aec1 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -208,7 +208,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ | 配置项 | 当前主线目录值 | 代码默认值 | 说明 | |--------|--------|------------|------| -| `rocksdb.total_memory_size` | `auto` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,取值为 `auto` 或字节数,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中;所有分区的 Raft 日志 memtable 共用一个 256 MiB 的写缓冲,并计入这 1 GiB 缓存,不会额外占用内存。该预算是容量目标,并非硬上限:`rocksdb.write_buffer_allow_stall` 默认为 `false`,写缓存写满时不会阻塞写入,因此在大量写入时本地内存可能超出预算,容器仍可能被 OOM 终止。将其设为 `true` 能更严格地守住上限,但会增加写入延迟。取值为 `auto` 时(缺失、空值或 `0` 与之等同),取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留(1 GiB Raft 日志缓存加 512 MiB 其他本地内存)后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。Store 看不到的限制(例如私有 cgroup 命名空间之上的限制)不会被猜测:Store 使用可见的最低限制并输出警告,此时请显式设置该值。启动日志会输出预算及其模式、物理内存限制及来源、堆、直接内存、预留,以及 cgroup 视图是否完整。JVM 和 cgroup 文件都拿不到可用的内存大小时,改用 JVM 最大堆,同样限制在 256 MiB 到 32 GB 之间。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `auto` 才会自动计算。PD 模式下 PD 还可能保存一份副本(KV 键 `S:FS`),由第一个启动的 Store 上传。Store 只有在没有本地文件,或本地文件不比 `S:Timestamp` 新时,才会加载这份副本并覆盖本地文件;上传时不会写入时间戳,缺失的时间戳按 0 处理,因此只修改 `S:FS` 不会生效。升级时,请在每个 Store 上设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` 或 `-Drocksdb.total_memory_size=auto` 并重启,Store 会让这个值优先于本地文件和 PD 的副本。也可以先修改 `S:FS` 中的值,再通过 PD 的 KV 服务(gRPC `KvService.put`)把当前时间(毫秒)写入 `S:Timestamp`:所有运行中的 Store 会重新加载 PD 的副本、覆盖本地文件,并在 60 秒内通过 `bin/restart-hugegraph-store.sh` 自行重启;已停止的 Store 会在下次启动时加载它。其他值按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | +| `rocksdb.total_memory_size` | `auto` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,取值为 `auto` 或字节数,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中;所有分区的 Raft 日志 memtable 共用一个 256 MiB 的写缓冲,并计入这 1 GiB 缓存,不会额外占用内存。该预算是容量目标,并非硬上限:`rocksdb.write_buffer_allow_stall` 默认为 `false`,写缓存写满时不会阻塞写入,因此在大量写入时本地内存可能超出预算,容器仍可能被 OOM 终止。将其设为 `true` 能更严格地守住上限,但会增加写入延迟。取值为 `auto` 时(缺失、空值或 `0` 与之等同),取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留(1 GiB Raft 日志缓存加 512 MiB 其他本地内存)后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。Store 看不到的限制(例如私有 cgroup 命名空间之上的限制)不会被猜测:Store 使用可见的最低限制并输出警告,此时请显式设置该值。启动日志会输出预算及其模式、物理内存限制及来源、堆、直接内存、预留,以及 cgroup 视图是否完整。JVM 和 cgroup 文件都拿不到可用的内存大小时,改用 JVM 最大堆,同样限制在 256 MiB 到 32 GB 之间。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `auto` 才会自动计算。PD 模式下 PD 还可能保存一份副本(KV 键 `S:FS`),由第一个启动的 Store 上传。Store 只有在没有本地文件,或本地文件不比 `S:Timestamp` 新时,才会加载这份副本并覆盖本地文件;上传时不会写入时间戳,缺失的时间戳按 0 处理,因此只修改 `S:FS` 不会生效。升级时,请在每个 Store 上设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` 或 `-Drocksdb.total_memory_size=auto` 并重启,Store 会让这个值优先于本地文件和 PD 的副本。也可以先在 `S:FS` 中把该值改为 `0` 或删除它,再通过 PD 的 KV 服务(gRPC `KvService.put`)把当前时间(毫秒)写入 `S:Timestamp`:所有运行中的 Store 会重新加载 PD 的副本、覆盖本地文件,并在 60 秒内通过 `bin/restart-hugegraph-store.sh` 自行重启;已停止的 Store 会在下次启动时加载它。只有所有 Store 都已运行支持 `auto` 的版本后,才能在 `S:FS` 中写入 `auto`:旧版本无法解析 `auto`,会启动失败,而 `0` 在旧版本中按原来的堆大小默认值处理。PD 中原本没有副本、由新版本 Store 上传的副本也是如此,因为发行的配置文件写的是 `auto`。自行重启时不带任何参数,最初启动时的 `-j` 和 `-g` 会丢失(包括 Docker 入口脚本以 `-j` 传入的 `JAVA_OPTS`),启动脚本改用默认堆大小;只有导出的 `JAVA_OPTIONS` 中的 JVM 设置会保留,因此对使用自定义堆大小启动的 Store,建议采用逐个 Store 覆盖的方式。其他值按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | | `rocksdb.write_buffer_size` | `32000000` | 自动推导,见说明 | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`,最小为 `1048576`(1 MiB)。 | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | 落盘前合并的 memtable 数量。 | | `rocksdb.write_buffer_ratio` | 未设置 | `0.66` | `total_memory_size` 中分配给写缓存的比例,其余作为 block cache。 | diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index 669c125541..f34d5cf685 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -210,7 +210,7 @@ This file contains only `rocksdb` and Actuator exposure settings. | Key | Master distribution value | Code default | Meaning | |-----|---------|--------------|---------| -| `rocksdb.total_memory_size` | `auto` | Auto-sized, see Meaning | Memory budget, `auto` or a number of bytes, shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top; the Raft log memtables of all partitions share a 256 MiB write buffer charged to that cache, so they don't add to it. The budget is a sizing target, not a hard cap: with `rocksdb.write_buffer_allow_stall` at its default `false`, writes are not stalled when the write cache is full, so native memory can exceed the budget under heavy writes and a container can still be OOM-killed. Setting it to `true` holds the cap more firmly at the cost of write latency. With `auto` (a missing, blank or `0` value means the same), the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve (the 1 GiB Raft log cache plus 512 MiB for other native memory), clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. A limit the Store can't see, such as one above a private cgroup namespace, isn't guessed: the Store uses the lowest visible limit and logs a warning, so pin the value there. The startup log shows the budget with its mode, physical limit and source, heap, direct memory, reserve and whether the cgroup view is complete. If neither the JVM nor the cgroup files report a usable memory size, the budget falls back to the JVM max heap, clamped to the same 256 MiB to 32 GB. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `auto` to auto-size. In PD mode PD may also keep a copy (KV key `S:FS`), uploaded by the first Store to start. A Store loads that copy over its local file only when it has no local file or the file is no newer than `S:Timestamp`; the upload sets no timestamp, and a missing one counts as 0, so changing `S:FS` alone changes nothing. To upgrade, set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` or `-Drocksdb.total_memory_size=auto` on each Store and restart it; the Store keeps this value above both its local file and PD's copy. Alternatively, change the value in `S:FS`, then put the current time in milliseconds into `S:Timestamp` through PD's KV service (gRPC `KvService.put`): every running Store then reloads PD's copy, writes it over its local file and restarts itself with `bin/restart-hugegraph-store.sh` within 60 seconds, and a stopped Store loads it at its next start. Any other value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | +| `rocksdb.total_memory_size` | `auto` | Auto-sized, see Meaning | Memory budget, `auto` or a number of bytes, shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top; the Raft log memtables of all partitions share a 256 MiB write buffer charged to that cache, so they don't add to it. The budget is a sizing target, not a hard cap: with `rocksdb.write_buffer_allow_stall` at its default `false`, writes are not stalled when the write cache is full, so native memory can exceed the budget under heavy writes and a container can still be OOM-killed. Setting it to `true` holds the cap more firmly at the cost of write latency. With `auto` (a missing, blank or `0` value means the same), the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve (the 1 GiB Raft log cache plus 512 MiB for other native memory), clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. A limit the Store can't see, such as one above a private cgroup namespace, isn't guessed: the Store uses the lowest visible limit and logs a warning, so pin the value there. The startup log shows the budget with its mode, physical limit and source, heap, direct memory, reserve and whether the cgroup view is complete. If neither the JVM nor the cgroup files report a usable memory size, the budget falls back to the JVM max heap, clamped to the same 256 MiB to 32 GB. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `auto` to auto-size. In PD mode PD may also keep a copy (KV key `S:FS`), uploaded by the first Store to start. A Store loads that copy over its local file only when it has no local file or the file is no newer than `S:Timestamp`; the upload sets no timestamp, and a missing one counts as 0, so changing `S:FS` alone changes nothing. To upgrade, set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` or `-Drocksdb.total_memory_size=auto` on each Store and restart it; the Store keeps this value above both its local file and PD's copy. Alternatively, set the value to `0` or remove it in `S:FS`, then put the current time in milliseconds into `S:Timestamp` through PD's KV service (gRPC `KvService.put`): every running Store then reloads PD's copy, writes it over its local file and restarts itself with `bin/restart-hugegraph-store.sh` within 60 seconds, and a stopped Store loads it at its next start. Put `auto` in `S:FS` only once every Store runs a release that supports it: an earlier release can't parse `auto` and fails to start, while it reads `0` as its old heap-sized default. The same holds for a copy uploaded by a newer Store when PD had none, since the shipped file says `auto`. The self-restart passes no arguments, so `-j` and `-g` from the original start are dropped, including the `JAVA_OPTS` the Docker entrypoint passes as `-j`, and the launcher's default heap applies; only JVM settings in an exported `JAVA_OPTIONS` carry over, so prefer the per-Store override on Stores started with a custom heap. Any other value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | | `rocksdb.write_buffer_size` | `32000000` | Derived, see Meaning | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`, at least `1048576` (1 MiB). | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | Number of memtables merged together before a flush. | | `rocksdb.write_buffer_ratio` | not set | `0.66` | Share of `total_memory_size` given to the write cache. The rest becomes the block cache. | From eab980446feedec03ff7fa65541ee779d779dc3d Mon Sep 17 00:00:00 2001 From: ayanx Date: Tue, 27 Oct 2026 23:05:55 +0530 Subject: [PATCH 18/23] docs(store): say a Docker Store stops on S:Timestamp instead of restarting --- content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- content/en/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index de1b30aec1..dff4fb294b 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -208,7 +208,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ | 配置项 | 当前主线目录值 | 代码默认值 | 说明 | |--------|--------|------------|------| -| `rocksdb.total_memory_size` | `auto` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,取值为 `auto` 或字节数,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中;所有分区的 Raft 日志 memtable 共用一个 256 MiB 的写缓冲,并计入这 1 GiB 缓存,不会额外占用内存。该预算是容量目标,并非硬上限:`rocksdb.write_buffer_allow_stall` 默认为 `false`,写缓存写满时不会阻塞写入,因此在大量写入时本地内存可能超出预算,容器仍可能被 OOM 终止。将其设为 `true` 能更严格地守住上限,但会增加写入延迟。取值为 `auto` 时(缺失、空值或 `0` 与之等同),取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留(1 GiB Raft 日志缓存加 512 MiB 其他本地内存)后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。Store 看不到的限制(例如私有 cgroup 命名空间之上的限制)不会被猜测:Store 使用可见的最低限制并输出警告,此时请显式设置该值。启动日志会输出预算及其模式、物理内存限制及来源、堆、直接内存、预留,以及 cgroup 视图是否完整。JVM 和 cgroup 文件都拿不到可用的内存大小时,改用 JVM 最大堆,同样限制在 256 MiB 到 32 GB 之间。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `auto` 才会自动计算。PD 模式下 PD 还可能保存一份副本(KV 键 `S:FS`),由第一个启动的 Store 上传。Store 只有在没有本地文件,或本地文件不比 `S:Timestamp` 新时,才会加载这份副本并覆盖本地文件;上传时不会写入时间戳,缺失的时间戳按 0 处理,因此只修改 `S:FS` 不会生效。升级时,请在每个 Store 上设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` 或 `-Drocksdb.total_memory_size=auto` 并重启,Store 会让这个值优先于本地文件和 PD 的副本。也可以先在 `S:FS` 中把该值改为 `0` 或删除它,再通过 PD 的 KV 服务(gRPC `KvService.put`)把当前时间(毫秒)写入 `S:Timestamp`:所有运行中的 Store 会重新加载 PD 的副本、覆盖本地文件,并在 60 秒内通过 `bin/restart-hugegraph-store.sh` 自行重启;已停止的 Store 会在下次启动时加载它。只有所有 Store 都已运行支持 `auto` 的版本后,才能在 `S:FS` 中写入 `auto`:旧版本无法解析 `auto`,会启动失败,而 `0` 在旧版本中按原来的堆大小默认值处理。PD 中原本没有副本、由新版本 Store 上传的副本也是如此,因为发行的配置文件写的是 `auto`。自行重启时不带任何参数,最初启动时的 `-j` 和 `-g` 会丢失(包括 Docker 入口脚本以 `-j` 传入的 `JAVA_OPTS`),启动脚本改用默认堆大小;只有导出的 `JAVA_OPTIONS` 中的 JVM 设置会保留,因此对使用自定义堆大小启动的 Store,建议采用逐个 Store 覆盖的方式。其他值按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | +| `rocksdb.total_memory_size` | `auto` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,取值为 `auto` 或字节数,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中;所有分区的 Raft 日志 memtable 共用一个 256 MiB 的写缓冲,并计入这 1 GiB 缓存,不会额外占用内存。该预算是容量目标,并非硬上限:`rocksdb.write_buffer_allow_stall` 默认为 `false`,写缓存写满时不会阻塞写入,因此在大量写入时本地内存可能超出预算,容器仍可能被 OOM 终止。将其设为 `true` 能更严格地守住上限,但会增加写入延迟。取值为 `auto` 时(缺失、空值或 `0` 与之等同),取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留(1 GiB Raft 日志缓存加 512 MiB 其他本地内存)后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。Store 看不到的限制(例如私有 cgroup 命名空间之上的限制)不会被猜测:Store 使用可见的最低限制并输出警告,此时请显式设置该值。启动日志会输出预算及其模式、物理内存限制及来源、堆、直接内存、预留,以及 cgroup 视图是否完整。JVM 和 cgroup 文件都拿不到可用的内存大小时,改用 JVM 最大堆,同样限制在 256 MiB 到 32 GB 之间。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `auto` 才会自动计算。PD 模式下 PD 还可能保存一份副本(KV 键 `S:FS`),由第一个启动的 Store 上传。Store 只有在没有本地文件,或本地文件不比 `S:Timestamp` 新时,才会加载这份副本并覆盖本地文件;上传时不会写入时间戳,缺失的时间戳按 0 处理,因此只修改 `S:FS` 不会生效。升级时,请在每个 Store 上设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` 或 `-Drocksdb.total_memory_size=auto` 并重启,Store 会让这个值优先于本地文件和 PD 的副本。也可以先在 `S:FS` 中把该值改为 `0` 或删除它,再通过 PD 的 KV 服务(gRPC `KvService.put`)把当前时间(毫秒)写入 `S:Timestamp`:所有运行中的 Store 会重新加载 PD 的副本、覆盖本地文件,并在 60 秒内运行 `bin/restart-hugegraph-store.sh`,因此所有 Store 会在同一分钟内重启;已停止的 Store 会在下次启动时加载它。只有所有 Store 都已运行支持 `auto` 的版本后,才能在 `S:FS` 中写入 `auto`:旧版本无法解析 `auto`,会启动失败,而 `0` 在旧版本中按原来的堆大小默认值处理。PD 中原本没有副本、由新版本 Store 上传的副本也是如此,因为发行的配置文件写的是 `auto`。自行重启时不带任何参数,最初启动时的 `-j` 和 `-g` 会丢失,启动脚本改用默认堆大小;只有导出的 `JAVA_OPTIONS` 中的 JVM 设置会保留,因此对使用自定义堆大小启动的 Store,建议采用逐个 Store 覆盖的方式。Docker 中的 Store 根本不会重启:Java 是入口脚本的最后一个进程,停止它会使容器以退出码 143 退出;除非有重启策略(例如 `restart: unless-stopped`)通过入口脚本重新启动容器,否则容器会一直处于停止状态。在 Docker 中请设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE`,不要写入 `S:Timestamp`。其他值按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | | `rocksdb.write_buffer_size` | `32000000` | 自动推导,见说明 | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`,最小为 `1048576`(1 MiB)。 | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | 落盘前合并的 memtable 数量。 | | `rocksdb.write_buffer_ratio` | 未设置 | `0.66` | `total_memory_size` 中分配给写缓存的比例,其余作为 block cache。 | diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index f34d5cf685..2604d229ed 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -210,7 +210,7 @@ This file contains only `rocksdb` and Actuator exposure settings. | Key | Master distribution value | Code default | Meaning | |-----|---------|--------------|---------| -| `rocksdb.total_memory_size` | `auto` | Auto-sized, see Meaning | Memory budget, `auto` or a number of bytes, shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top; the Raft log memtables of all partitions share a 256 MiB write buffer charged to that cache, so they don't add to it. The budget is a sizing target, not a hard cap: with `rocksdb.write_buffer_allow_stall` at its default `false`, writes are not stalled when the write cache is full, so native memory can exceed the budget under heavy writes and a container can still be OOM-killed. Setting it to `true` holds the cap more firmly at the cost of write latency. With `auto` (a missing, blank or `0` value means the same), the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve (the 1 GiB Raft log cache plus 512 MiB for other native memory), clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. A limit the Store can't see, such as one above a private cgroup namespace, isn't guessed: the Store uses the lowest visible limit and logs a warning, so pin the value there. The startup log shows the budget with its mode, physical limit and source, heap, direct memory, reserve and whether the cgroup view is complete. If neither the JVM nor the cgroup files report a usable memory size, the budget falls back to the JVM max heap, clamped to the same 256 MiB to 32 GB. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `auto` to auto-size. In PD mode PD may also keep a copy (KV key `S:FS`), uploaded by the first Store to start. A Store loads that copy over its local file only when it has no local file or the file is no newer than `S:Timestamp`; the upload sets no timestamp, and a missing one counts as 0, so changing `S:FS` alone changes nothing. To upgrade, set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` or `-Drocksdb.total_memory_size=auto` on each Store and restart it; the Store keeps this value above both its local file and PD's copy. Alternatively, set the value to `0` or remove it in `S:FS`, then put the current time in milliseconds into `S:Timestamp` through PD's KV service (gRPC `KvService.put`): every running Store then reloads PD's copy, writes it over its local file and restarts itself with `bin/restart-hugegraph-store.sh` within 60 seconds, and a stopped Store loads it at its next start. Put `auto` in `S:FS` only once every Store runs a release that supports it: an earlier release can't parse `auto` and fails to start, while it reads `0` as its old heap-sized default. The same holds for a copy uploaded by a newer Store when PD had none, since the shipped file says `auto`. The self-restart passes no arguments, so `-j` and `-g` from the original start are dropped, including the `JAVA_OPTS` the Docker entrypoint passes as `-j`, and the launcher's default heap applies; only JVM settings in an exported `JAVA_OPTIONS` carry over, so prefer the per-Store override on Stores started with a custom heap. Any other value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | +| `rocksdb.total_memory_size` | `auto` | Auto-sized, see Meaning | Memory budget, `auto` or a number of bytes, shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top; the Raft log memtables of all partitions share a 256 MiB write buffer charged to that cache, so they don't add to it. The budget is a sizing target, not a hard cap: with `rocksdb.write_buffer_allow_stall` at its default `false`, writes are not stalled when the write cache is full, so native memory can exceed the budget under heavy writes and a container can still be OOM-killed. Setting it to `true` holds the cap more firmly at the cost of write latency. With `auto` (a missing, blank or `0` value means the same), the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve (the 1 GiB Raft log cache plus 512 MiB for other native memory), clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. A limit the Store can't see, such as one above a private cgroup namespace, isn't guessed: the Store uses the lowest visible limit and logs a warning, so pin the value there. The startup log shows the budget with its mode, physical limit and source, heap, direct memory, reserve and whether the cgroup view is complete. If neither the JVM nor the cgroup files report a usable memory size, the budget falls back to the JVM max heap, clamped to the same 256 MiB to 32 GB. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `auto` to auto-size. In PD mode PD may also keep a copy (KV key `S:FS`), uploaded by the first Store to start. A Store loads that copy over its local file only when it has no local file or the file is no newer than `S:Timestamp`; the upload sets no timestamp, and a missing one counts as 0, so changing `S:FS` alone changes nothing. To upgrade, set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` or `-Drocksdb.total_memory_size=auto` on each Store and restart it; the Store keeps this value above both its local file and PD's copy. Alternatively, set the value to `0` or remove it in `S:FS`, then put the current time in milliseconds into `S:Timestamp` through PD's KV service (gRPC `KvService.put`): every running Store then reloads PD's copy, writes it over its local file and runs `bin/restart-hugegraph-store.sh` within 60 seconds, so all Stores restart within the same minute, and a stopped Store loads it at its next start. Put `auto` in `S:FS` only once every Store runs a release that supports it: an earlier release can't parse `auto` and fails to start, while it reads `0` as its old heap-sized default. The same holds for a copy uploaded by a newer Store when PD had none, since the shipped file says `auto`. The self-restart passes no arguments, so `-j` and `-g` from the original start are dropped and the launcher's default heap applies; only JVM settings in an exported `JAVA_OPTIONS` carry over, so prefer the per-Store override on Stores started with a custom heap. A Docker Store doesn't restart at all: Java is the entrypoint's last process, so stopping it ends the container with exit code 143, and the container stays down unless a restart policy such as `restart: unless-stopped` starts it again through the entrypoint. On Docker, set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` instead of publishing `S:Timestamp`. Any other value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | | `rocksdb.write_buffer_size` | `32000000` | Derived, see Meaning | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`, at least `1048576` (1 MiB). | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | Number of memtables merged together before a flush. | | `rocksdb.write_buffer_ratio` | not set | `0.66` | Share of `total_memory_size` given to the write cache. The rest becomes the block cache. | From 5c2761594bd9bad6b1cfe68e98196aa627f5652b Mon Sep 17 00:00:00 2001 From: ayanx Date: Wed, 28 Oct 2026 09:50:02 +0530 Subject: [PATCH 19/23] docs(store): say a restarted Docker Store keeps JAVA_OPTS and a plain docker run stays down --- content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- content/en/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index dff4fb294b..270a1d10eb 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -208,7 +208,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ | 配置项 | 当前主线目录值 | 代码默认值 | 说明 | |--------|--------|------------|------| -| `rocksdb.total_memory_size` | `auto` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,取值为 `auto` 或字节数,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中;所有分区的 Raft 日志 memtable 共用一个 256 MiB 的写缓冲,并计入这 1 GiB 缓存,不会额外占用内存。该预算是容量目标,并非硬上限:`rocksdb.write_buffer_allow_stall` 默认为 `false`,写缓存写满时不会阻塞写入,因此在大量写入时本地内存可能超出预算,容器仍可能被 OOM 终止。将其设为 `true` 能更严格地守住上限,但会增加写入延迟。取值为 `auto` 时(缺失、空值或 `0` 与之等同),取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留(1 GiB Raft 日志缓存加 512 MiB 其他本地内存)后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。Store 看不到的限制(例如私有 cgroup 命名空间之上的限制)不会被猜测:Store 使用可见的最低限制并输出警告,此时请显式设置该值。启动日志会输出预算及其模式、物理内存限制及来源、堆、直接内存、预留,以及 cgroup 视图是否完整。JVM 和 cgroup 文件都拿不到可用的内存大小时,改用 JVM 最大堆,同样限制在 256 MiB 到 32 GB 之间。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `auto` 才会自动计算。PD 模式下 PD 还可能保存一份副本(KV 键 `S:FS`),由第一个启动的 Store 上传。Store 只有在没有本地文件,或本地文件不比 `S:Timestamp` 新时,才会加载这份副本并覆盖本地文件;上传时不会写入时间戳,缺失的时间戳按 0 处理,因此只修改 `S:FS` 不会生效。升级时,请在每个 Store 上设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` 或 `-Drocksdb.total_memory_size=auto` 并重启,Store 会让这个值优先于本地文件和 PD 的副本。也可以先在 `S:FS` 中把该值改为 `0` 或删除它,再通过 PD 的 KV 服务(gRPC `KvService.put`)把当前时间(毫秒)写入 `S:Timestamp`:所有运行中的 Store 会重新加载 PD 的副本、覆盖本地文件,并在 60 秒内运行 `bin/restart-hugegraph-store.sh`,因此所有 Store 会在同一分钟内重启;已停止的 Store 会在下次启动时加载它。只有所有 Store 都已运行支持 `auto` 的版本后,才能在 `S:FS` 中写入 `auto`:旧版本无法解析 `auto`,会启动失败,而 `0` 在旧版本中按原来的堆大小默认值处理。PD 中原本没有副本、由新版本 Store 上传的副本也是如此,因为发行的配置文件写的是 `auto`。自行重启时不带任何参数,最初启动时的 `-j` 和 `-g` 会丢失,启动脚本改用默认堆大小;只有导出的 `JAVA_OPTIONS` 中的 JVM 设置会保留,因此对使用自定义堆大小启动的 Store,建议采用逐个 Store 覆盖的方式。Docker 中的 Store 根本不会重启:Java 是入口脚本的最后一个进程,停止它会使容器以退出码 143 退出;除非有重启策略(例如 `restart: unless-stopped`)通过入口脚本重新启动容器,否则容器会一直处于停止状态。在 Docker 中请设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE`,不要写入 `S:Timestamp`。其他值按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | +| `rocksdb.total_memory_size` | `auto` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,取值为 `auto` 或字节数,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中;所有分区的 Raft 日志 memtable 共用一个 256 MiB 的写缓冲,并计入这 1 GiB 缓存,不会额外占用内存。该预算是容量目标,并非硬上限:`rocksdb.write_buffer_allow_stall` 默认为 `false`,写缓存写满时不会阻塞写入,因此在大量写入时本地内存可能超出预算,容器仍可能被 OOM 终止。将其设为 `true` 能更严格地守住上限,但会增加写入延迟。取值为 `auto` 时(缺失、空值或 `0` 与之等同),取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留(1 GiB Raft 日志缓存加 512 MiB 其他本地内存)后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。Store 看不到的限制(例如私有 cgroup 命名空间之上的限制)不会被猜测:Store 使用可见的最低限制并输出警告,此时请显式设置该值。启动日志会输出预算及其模式、物理内存限制及来源、堆、直接内存、预留,以及 cgroup 视图是否完整。JVM 和 cgroup 文件都拿不到可用的内存大小时,改用 JVM 最大堆,同样限制在 256 MiB 到 32 GB 之间。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `auto` 才会自动计算。PD 模式下 PD 还可能保存一份副本(KV 键 `S:FS`),由第一个启动的 Store 上传。Store 只有在没有本地文件,或本地文件不比 `S:Timestamp` 新时,才会加载这份副本并覆盖本地文件;上传时不会写入时间戳,缺失的时间戳按 0 处理,因此只修改 `S:FS` 不会生效。升级时,请在每个 Store 上设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` 或 `-Drocksdb.total_memory_size=auto` 并重启,Store 会让这个值优先于本地文件和 PD 的副本。也可以先在 `S:FS` 中把该值改为 `0` 或删除它,再通过 PD 的 KV 服务(gRPC `KvService.put`)把当前时间(毫秒)写入 `S:Timestamp`:所有运行中的 Store 会重新加载 PD 的副本、覆盖本地文件,并在 60 秒内运行 `bin/restart-hugegraph-store.sh`,因此所有 Store 会在同一分钟内重启;已停止的 Store 会在下次启动时加载它。只有所有 Store 都已运行支持 `auto` 的版本后,才能在 `S:FS` 中写入 `auto`:旧版本无法解析 `auto`,会启动失败,而 `0` 在旧版本中按原来的堆大小默认值处理。PD 中原本没有副本、由新版本 Store 上传的副本也是如此,因为发行的配置文件写的是 `auto`。自行重启时不带任何参数,最初启动时的 `-j` 和 `-g` 会丢失,启动脚本改用默认堆大小;只有导出的 `JAVA_OPTIONS` 中的 JVM 设置会保留,因此对使用自定义堆大小启动的 Store,建议采用逐个 Store 覆盖的方式。Docker 中的 Store 根本不会重启:Java 是入口脚本的最后一个进程,停止它会使容器连同其中的重启脚本一起以退出码 143 退出。设置了重启策略的容器(Compose 文件使用 `restart: unless-stopped`)会通过入口脚本重新启动,并保留 `JAVA_OPTS`;像 3.3 节中独立运行的 `docker run -d` 示例那样未指定 `--restart` 启动的容器,则会一直处于停止状态。在 Docker 中请设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE`,不要写入 `S:Timestamp`。其他值按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | | `rocksdb.write_buffer_size` | `32000000` | 自动推导,见说明 | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`,最小为 `1048576`(1 MiB)。 | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | 落盘前合并的 memtable 数量。 | | `rocksdb.write_buffer_ratio` | 未设置 | `0.66` | `total_memory_size` 中分配给写缓存的比例,其余作为 block cache。 | diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index 2604d229ed..4a31b1de7c 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -210,7 +210,7 @@ This file contains only `rocksdb` and Actuator exposure settings. | Key | Master distribution value | Code default | Meaning | |-----|---------|--------------|---------| -| `rocksdb.total_memory_size` | `auto` | Auto-sized, see Meaning | Memory budget, `auto` or a number of bytes, shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top; the Raft log memtables of all partitions share a 256 MiB write buffer charged to that cache, so they don't add to it. The budget is a sizing target, not a hard cap: with `rocksdb.write_buffer_allow_stall` at its default `false`, writes are not stalled when the write cache is full, so native memory can exceed the budget under heavy writes and a container can still be OOM-killed. Setting it to `true` holds the cap more firmly at the cost of write latency. With `auto` (a missing, blank or `0` value means the same), the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve (the 1 GiB Raft log cache plus 512 MiB for other native memory), clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. A limit the Store can't see, such as one above a private cgroup namespace, isn't guessed: the Store uses the lowest visible limit and logs a warning, so pin the value there. The startup log shows the budget with its mode, physical limit and source, heap, direct memory, reserve and whether the cgroup view is complete. If neither the JVM nor the cgroup files report a usable memory size, the budget falls back to the JVM max heap, clamped to the same 256 MiB to 32 GB. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `auto` to auto-size. In PD mode PD may also keep a copy (KV key `S:FS`), uploaded by the first Store to start. A Store loads that copy over its local file only when it has no local file or the file is no newer than `S:Timestamp`; the upload sets no timestamp, and a missing one counts as 0, so changing `S:FS` alone changes nothing. To upgrade, set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` or `-Drocksdb.total_memory_size=auto` on each Store and restart it; the Store keeps this value above both its local file and PD's copy. Alternatively, set the value to `0` or remove it in `S:FS`, then put the current time in milliseconds into `S:Timestamp` through PD's KV service (gRPC `KvService.put`): every running Store then reloads PD's copy, writes it over its local file and runs `bin/restart-hugegraph-store.sh` within 60 seconds, so all Stores restart within the same minute, and a stopped Store loads it at its next start. Put `auto` in `S:FS` only once every Store runs a release that supports it: an earlier release can't parse `auto` and fails to start, while it reads `0` as its old heap-sized default. The same holds for a copy uploaded by a newer Store when PD had none, since the shipped file says `auto`. The self-restart passes no arguments, so `-j` and `-g` from the original start are dropped and the launcher's default heap applies; only JVM settings in an exported `JAVA_OPTIONS` carry over, so prefer the per-Store override on Stores started with a custom heap. A Docker Store doesn't restart at all: Java is the entrypoint's last process, so stopping it ends the container with exit code 143, and the container stays down unless a restart policy such as `restart: unless-stopped` starts it again through the entrypoint. On Docker, set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` instead of publishing `S:Timestamp`. Any other value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | +| `rocksdb.total_memory_size` | `auto` | Auto-sized, see Meaning | Memory budget, `auto` or a number of bytes, shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top; the Raft log memtables of all partitions share a 256 MiB write buffer charged to that cache, so they don't add to it. The budget is a sizing target, not a hard cap: with `rocksdb.write_buffer_allow_stall` at its default `false`, writes are not stalled when the write cache is full, so native memory can exceed the budget under heavy writes and a container can still be OOM-killed. Setting it to `true` holds the cap more firmly at the cost of write latency. With `auto` (a missing, blank or `0` value means the same), the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve (the 1 GiB Raft log cache plus 512 MiB for other native memory), clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. A limit the Store can't see, such as one above a private cgroup namespace, isn't guessed: the Store uses the lowest visible limit and logs a warning, so pin the value there. The startup log shows the budget with its mode, physical limit and source, heap, direct memory, reserve and whether the cgroup view is complete. If neither the JVM nor the cgroup files report a usable memory size, the budget falls back to the JVM max heap, clamped to the same 256 MiB to 32 GB. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `auto` to auto-size. In PD mode PD may also keep a copy (KV key `S:FS`), uploaded by the first Store to start. A Store loads that copy over its local file only when it has no local file or the file is no newer than `S:Timestamp`; the upload sets no timestamp, and a missing one counts as 0, so changing `S:FS` alone changes nothing. To upgrade, set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` or `-Drocksdb.total_memory_size=auto` on each Store and restart it; the Store keeps this value above both its local file and PD's copy. Alternatively, set the value to `0` or remove it in `S:FS`, then put the current time in milliseconds into `S:Timestamp` through PD's KV service (gRPC `KvService.put`): every running Store then reloads PD's copy, writes it over its local file and runs `bin/restart-hugegraph-store.sh` within 60 seconds, so all Stores restart within the same minute, and a stopped Store loads it at its next start. Put `auto` in `S:FS` only once every Store runs a release that supports it: an earlier release can't parse `auto` and fails to start, while it reads `0` as its old heap-sized default. The same holds for a copy uploaded by a newer Store when PD had none, since the shipped file says `auto`. The self-restart passes no arguments, so `-j` and `-g` from the original start are dropped and the launcher's default heap applies; only JVM settings in an exported `JAVA_OPTIONS` carry over, so prefer the per-Store override on Stores started with a custom heap. A Docker Store doesn't restart at all: Java is the entrypoint's last process, so stopping it ends the container with exit code 143 and the restart script with it. A container with a restart policy (the Compose files use `restart: unless-stopped`) comes back through the entrypoint and keeps `JAVA_OPTS`; one started like the standalone `docker run -d` example in section 3.3, which has no `--restart`, stays down. On Docker, set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` instead of publishing `S:Timestamp`. Any other value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | | `rocksdb.write_buffer_size` | `32000000` | Derived, see Meaning | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`, at least `1048576` (1 MiB). | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | Number of memtables merged together before a flush. | | `rocksdb.write_buffer_ratio` | not set | `0.66` | Share of `total_memory_size` given to the write cache. The rest becomes the block cache. | From 0cb48506cfba2349849bb489f07a617e87efa585 Mon Sep 17 00:00:00 2001 From: ayanx Date: Wed, 28 Oct 2026 14:50:35 +0530 Subject: [PATCH 20/23] docs(store): note that a JAVA_TOOL_OPTIONS or JDK_JAVA_OPTIONS cap is kept --- content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- content/en/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index 270a1d10eb..7970180dc8 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -208,7 +208,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ | 配置项 | 当前主线目录值 | 代码默认值 | 说明 | |--------|--------|------------|------| -| `rocksdb.total_memory_size` | `auto` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,取值为 `auto` 或字节数,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中;所有分区的 Raft 日志 memtable 共用一个 256 MiB 的写缓冲,并计入这 1 GiB 缓存,不会额外占用内存。该预算是容量目标,并非硬上限:`rocksdb.write_buffer_allow_stall` 默认为 `false`,写缓存写满时不会阻塞写入,因此在大量写入时本地内存可能超出预算,容器仍可能被 OOM 终止。将其设为 `true` 能更严格地守住上限,但会增加写入延迟。取值为 `auto` 时(缺失、空值或 `0` 与之等同),取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限时不再添加)和 1.5 GiB 预留(1 GiB Raft 日志缓存加 512 MiB 其他本地内存)后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。Store 看不到的限制(例如私有 cgroup 命名空间之上的限制)不会被猜测:Store 使用可见的最低限制并输出警告,此时请显式设置该值。启动日志会输出预算及其模式、物理内存限制及来源、堆、直接内存、预留,以及 cgroup 视图是否完整。JVM 和 cgroup 文件都拿不到可用的内存大小时,改用 JVM 最大堆,同样限制在 256 MiB 到 32 GB 之间。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `auto` 才会自动计算。PD 模式下 PD 还可能保存一份副本(KV 键 `S:FS`),由第一个启动的 Store 上传。Store 只有在没有本地文件,或本地文件不比 `S:Timestamp` 新时,才会加载这份副本并覆盖本地文件;上传时不会写入时间戳,缺失的时间戳按 0 处理,因此只修改 `S:FS` 不会生效。升级时,请在每个 Store 上设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` 或 `-Drocksdb.total_memory_size=auto` 并重启,Store 会让这个值优先于本地文件和 PD 的副本。也可以先在 `S:FS` 中把该值改为 `0` 或删除它,再通过 PD 的 KV 服务(gRPC `KvService.put`)把当前时间(毫秒)写入 `S:Timestamp`:所有运行中的 Store 会重新加载 PD 的副本、覆盖本地文件,并在 60 秒内运行 `bin/restart-hugegraph-store.sh`,因此所有 Store 会在同一分钟内重启;已停止的 Store 会在下次启动时加载它。只有所有 Store 都已运行支持 `auto` 的版本后,才能在 `S:FS` 中写入 `auto`:旧版本无法解析 `auto`,会启动失败,而 `0` 在旧版本中按原来的堆大小默认值处理。PD 中原本没有副本、由新版本 Store 上传的副本也是如此,因为发行的配置文件写的是 `auto`。自行重启时不带任何参数,最初启动时的 `-j` 和 `-g` 会丢失,启动脚本改用默认堆大小;只有导出的 `JAVA_OPTIONS` 中的 JVM 设置会保留,因此对使用自定义堆大小启动的 Store,建议采用逐个 Store 覆盖的方式。Docker 中的 Store 根本不会重启:Java 是入口脚本的最后一个进程,停止它会使容器连同其中的重启脚本一起以退出码 143 退出。设置了重启策略的容器(Compose 文件使用 `restart: unless-stopped`)会通过入口脚本重新启动,并保留 `JAVA_OPTS`;像 3.3 节中独立运行的 `docker run -d` 示例那样未指定 `--restart` 启动的容器,则会一直处于停止状态。在 Docker 中请设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE`,不要写入 `S:Timestamp`。其他值按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | +| `rocksdb.total_memory_size` | `auto` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,取值为 `auto` 或字节数,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中;所有分区的 Raft 日志 memtable 共用一个 256 MiB 的写缓冲,并计入这 1 GiB 缓存,不会额外占用内存。该预算是容量目标,并非硬上限:`rocksdb.write_buffer_allow_stall` 默认为 `false`,写缓存写满时不会阻塞写入,因此在大量写入时本地内存可能超出预算,容器仍可能被 OOM 终止。将其设为 `true` 能更严格地守住上限,但会增加写入延迟。取值为 `auto` 时(缺失、空值或 `0` 与之等同),取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限,或 `JAVA_TOOL_OPTIONS`、`JDK_JAVA_OPTIONS` 设置了该上限时不再添加)和 1.5 GiB 预留(1 GiB Raft 日志缓存加 512 MiB 其他本地内存)后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。Store 看不到的限制(例如私有 cgroup 命名空间之上的限制)不会被猜测:Store 使用可见的最低限制并输出警告,此时请显式设置该值。启动日志会输出预算及其模式、物理内存限制及来源、堆、直接内存、预留,以及 cgroup 视图是否完整。JVM 和 cgroup 文件都拿不到可用的内存大小时,改用 JVM 最大堆,同样限制在 256 MiB 到 32 GB 之间。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `auto` 才会自动计算。PD 模式下 PD 还可能保存一份副本(KV 键 `S:FS`),由第一个启动的 Store 上传。Store 只有在没有本地文件,或本地文件不比 `S:Timestamp` 新时,才会加载这份副本并覆盖本地文件;上传时不会写入时间戳,缺失的时间戳按 0 处理,因此只修改 `S:FS` 不会生效。升级时,请在每个 Store 上设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` 或 `-Drocksdb.total_memory_size=auto` 并重启,Store 会让这个值优先于本地文件和 PD 的副本。也可以先在 `S:FS` 中把该值改为 `0` 或删除它,再通过 PD 的 KV 服务(gRPC `KvService.put`)把当前时间(毫秒)写入 `S:Timestamp`:所有运行中的 Store 会重新加载 PD 的副本、覆盖本地文件,并在 60 秒内运行 `bin/restart-hugegraph-store.sh`,因此所有 Store 会在同一分钟内重启;已停止的 Store 会在下次启动时加载它。只有所有 Store 都已运行支持 `auto` 的版本后,才能在 `S:FS` 中写入 `auto`:旧版本无法解析 `auto`,会启动失败,而 `0` 在旧版本中按原来的堆大小默认值处理。PD 中原本没有副本、由新版本 Store 上传的副本也是如此,因为发行的配置文件写的是 `auto`。自行重启时不带任何参数,最初启动时的 `-j` 和 `-g` 会丢失,启动脚本改用默认堆大小;只有导出的 `JAVA_OPTIONS` 中的 JVM 设置会保留,因此对使用自定义堆大小启动的 Store,建议采用逐个 Store 覆盖的方式。Docker 中的 Store 根本不会重启:Java 是入口脚本的最后一个进程,停止它会使容器连同其中的重启脚本一起以退出码 143 退出。设置了重启策略的容器(Compose 文件使用 `restart: unless-stopped`)会通过入口脚本重新启动,并保留 `JAVA_OPTS`;像 3.3 节中独立运行的 `docker run -d` 示例那样未指定 `--restart` 启动的容器,则会一直处于停止状态。在 Docker 中请设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE`,不要写入 `S:Timestamp`。其他值按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | | `rocksdb.write_buffer_size` | `32000000` | 自动推导,见说明 | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`,最小为 `1048576`(1 MiB)。 | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | 落盘前合并的 memtable 数量。 | | `rocksdb.write_buffer_ratio` | 未设置 | `0.66` | `total_memory_size` 中分配给写缓存的比例,其余作为 block cache。 | diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index 4a31b1de7c..780672bd13 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -210,7 +210,7 @@ This file contains only `rocksdb` and Actuator exposure settings. | Key | Master distribution value | Code default | Meaning | |-----|---------|--------------|---------| -| `rocksdb.total_memory_size` | `auto` | Auto-sized, see Meaning | Memory budget, `auto` or a number of bytes, shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top; the Raft log memtables of all partitions share a 256 MiB write buffer charged to that cache, so they don't add to it. The budget is a sizing target, not a hard cap: with `rocksdb.write_buffer_allow_stall` at its default `false`, writes are not stalled when the write cache is full, so native memory can exceed the budget under heavy writes and a container can still be OOM-killed. Setting it to `true` holds the cap more firmly at the cost of write latency. With `auto` (a missing, blank or `0` value means the same), the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself) and a 1.5 GiB reserve (the 1 GiB Raft log cache plus 512 MiB for other native memory), clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. A limit the Store can't see, such as one above a private cgroup namespace, isn't guessed: the Store uses the lowest visible limit and logs a warning, so pin the value there. The startup log shows the budget with its mode, physical limit and source, heap, direct memory, reserve and whether the cgroup view is complete. If neither the JVM nor the cgroup files report a usable memory size, the budget falls back to the JVM max heap, clamped to the same 256 MiB to 32 GB. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `auto` to auto-size. In PD mode PD may also keep a copy (KV key `S:FS`), uploaded by the first Store to start. A Store loads that copy over its local file only when it has no local file or the file is no newer than `S:Timestamp`; the upload sets no timestamp, and a missing one counts as 0, so changing `S:FS` alone changes nothing. To upgrade, set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` or `-Drocksdb.total_memory_size=auto` on each Store and restart it; the Store keeps this value above both its local file and PD's copy. Alternatively, set the value to `0` or remove it in `S:FS`, then put the current time in milliseconds into `S:Timestamp` through PD's KV service (gRPC `KvService.put`): every running Store then reloads PD's copy, writes it over its local file and runs `bin/restart-hugegraph-store.sh` within 60 seconds, so all Stores restart within the same minute, and a stopped Store loads it at its next start. Put `auto` in `S:FS` only once every Store runs a release that supports it: an earlier release can't parse `auto` and fails to start, while it reads `0` as its old heap-sized default. The same holds for a copy uploaded by a newer Store when PD had none, since the shipped file says `auto`. The self-restart passes no arguments, so `-j` and `-g` from the original start are dropped and the launcher's default heap applies; only JVM settings in an exported `JAVA_OPTIONS` carry over, so prefer the per-Store override on Stores started with a custom heap. A Docker Store doesn't restart at all: Java is the entrypoint's last process, so stopping it ends the container with exit code 143 and the restart script with it. A container with a restart policy (the Compose files use `restart: unless-stopped`) comes back through the entrypoint and keeps `JAVA_OPTS`; one started like the standalone `docker run -d` example in section 3.3, which has no `--restart`, stays down. On Docker, set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` instead of publishing `S:Timestamp`. Any other value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | +| `rocksdb.total_memory_size` | `auto` | Auto-sized, see Meaning | Memory budget, `auto` or a number of bytes, shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top; the Raft log memtables of all partitions share a 256 MiB write buffer charged to that cache, so they don't add to it. The budget is a sizing target, not a hard cap: with `rocksdb.write_buffer_allow_stall` at its default `false`, writes are not stalled when the write cache is full, so native memory can exceed the budget under heavy writes and a container can still be OOM-killed. Setting it to `true` holds the cap more firmly at the cost of write latency. With `auto` (a missing, blank or `0` value means the same), the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself, or `JAVA_TOOL_OPTIONS` or `JDK_JAVA_OPTIONS` sets the cap) and a 1.5 GiB reserve (the 1 GiB Raft log cache plus 512 MiB for other native memory), clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. A limit the Store can't see, such as one above a private cgroup namespace, isn't guessed: the Store uses the lowest visible limit and logs a warning, so pin the value there. The startup log shows the budget with its mode, physical limit and source, heap, direct memory, reserve and whether the cgroup view is complete. If neither the JVM nor the cgroup files report a usable memory size, the budget falls back to the JVM max heap, clamped to the same 256 MiB to 32 GB. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `auto` to auto-size. In PD mode PD may also keep a copy (KV key `S:FS`), uploaded by the first Store to start. A Store loads that copy over its local file only when it has no local file or the file is no newer than `S:Timestamp`; the upload sets no timestamp, and a missing one counts as 0, so changing `S:FS` alone changes nothing. To upgrade, set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` or `-Drocksdb.total_memory_size=auto` on each Store and restart it; the Store keeps this value above both its local file and PD's copy. Alternatively, set the value to `0` or remove it in `S:FS`, then put the current time in milliseconds into `S:Timestamp` through PD's KV service (gRPC `KvService.put`): every running Store then reloads PD's copy, writes it over its local file and runs `bin/restart-hugegraph-store.sh` within 60 seconds, so all Stores restart within the same minute, and a stopped Store loads it at its next start. Put `auto` in `S:FS` only once every Store runs a release that supports it: an earlier release can't parse `auto` and fails to start, while it reads `0` as its old heap-sized default. The same holds for a copy uploaded by a newer Store when PD had none, since the shipped file says `auto`. The self-restart passes no arguments, so `-j` and `-g` from the original start are dropped and the launcher's default heap applies; only JVM settings in an exported `JAVA_OPTIONS` carry over, so prefer the per-Store override on Stores started with a custom heap. A Docker Store doesn't restart at all: Java is the entrypoint's last process, so stopping it ends the container with exit code 143 and the restart script with it. A container with a restart policy (the Compose files use `restart: unless-stopped`) comes back through the entrypoint and keeps `JAVA_OPTS`; one started like the standalone `docker run -d` example in section 3.3, which has no `--restart`, stays down. On Docker, set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` instead of publishing `S:Timestamp`. Any other value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | | `rocksdb.write_buffer_size` | `32000000` | Derived, see Meaning | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`, at least `1048576` (1 MiB). | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | Number of memtables merged together before a flush. | | `rocksdb.write_buffer_ratio` | not set | `0.66` | Share of `total_memory_size` given to the write cache. The rest becomes the block cache. | From b5d55e105df4b58c13ee36da4b6e98a25f66c7cb Mon Sep 17 00:00:00 2001 From: ayanx Date: Thu, 29 Oct 2026 11:52:59 +0530 Subject: [PATCH 21/23] docs(store): check the memory rows against the commit that merged hugegraph#3355 --- content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- content/en/docs/quickstart/hugegraph/hugegraph-hstore.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index 91cc57d7cd..3df61ed104 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -149,7 +149,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ #### 4.1 application.yml -以下配置参考表核对的是 server 主线提交 `2f827d6e8c9c62ae858f2fc122b3a192d015e2f4` 的发行目录值和 Java 默认注入值,其中 `rocksdb.total_memory_size`、`rocksdb.write_buffer_size` 和 `raft.disruptorBufferSize` 三行描述的是 apache/hugegraph#3355 的行为;1.7.0 发布包请使用其版本标签对应的配置文件,不要跨版本覆盖。 +以下配置参考表核对的是 server 主线提交 `2f827d6e8c9c62ae858f2fc122b3a192d015e2f4` 的发行目录值和 Java 默认注入值,其中 `rocksdb.total_memory_size`、`rocksdb.write_buffer_size` 和 `raft.disruptorBufferSize` 三行核对的是合入 apache/hugegraph#3355 的提交 `edd7cee77`;1.7.0 发布包请使用其版本标签对应的配置文件,不要跨版本覆盖。 #### 4.2 application-pd.yml diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index 7899847bf3..1be34ec4fc 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -150,7 +150,7 @@ Store reads two files under `conf/`. Release 1.7.0 sources: [application.yml](ht #### 4.1 application.yml -The following tables were checked against distribution values and Java defaults at Server master commit `2f827d6e8c9c62ae858f2fc122b3a192d015e2f4`, except the `rocksdb.total_memory_size`, `rocksdb.write_buffer_size` and `raft.disruptorBufferSize` rows, which describe apache/hugegraph#3355. Use tagged configuration for release 1.7.0 without replacing it with another version's files. +The following tables were checked against distribution values and Java defaults at Server master commit `2f827d6e8c9c62ae858f2fc122b3a192d015e2f4`, except the `rocksdb.total_memory_size`, `rocksdb.write_buffer_size` and `raft.disruptorBufferSize` rows, which were checked at commit `edd7cee77`, where apache/hugegraph#3355 was merged. Use tagged configuration for release 1.7.0 without replacing it with another version's files. #### 4.2 application-pd.yml From 2d4de64ee6903cfd7f7b540fc4a8f2c48c49a6cf Mon Sep 17 00:00:00 2001 From: dark Date: Fri, 30 Oct 2026 05:07:53 +0800 Subject: [PATCH 22/23] docs: streamline rocksdb memory budget and align cn/en docs --- content/cn/docs/guides/hugegraph-docker-cluster.md | 10 +++------- .../docs/quickstart/hugegraph/hugegraph-hstore.md | 12 ++++++++++-- content/en/docs/guides/hugegraph-docker-cluster.md | 14 ++++---------- .../docs/quickstart/hugegraph/hugegraph-hstore.md | 12 ++++++++++-- 4 files changed, 27 insertions(+), 21 deletions(-) diff --git a/content/cn/docs/guides/hugegraph-docker-cluster.md b/content/cn/docs/guides/hugegraph-docker-cluster.md index 617e99ea68..cb986bb7f2 100644 --- a/content/cn/docs/guides/hugegraph-docker-cluster.md +++ b/content/cn/docs/guides/hugegraph-docker-cluster.md @@ -201,13 +201,9 @@ curl -o /dev/null -w '%{http_code}\n' -u "admin:${HUGEGRAPH_ADMIN_PASSWORD}" \ > **已弃用的别名**:`PD_ADDRESS` → `HG_STORE_PD_ADDRESS`、`GRPC_HOST` → `HG_STORE_GRPC_HOST`、`RAFT_ADDRESS` → `HG_STORE_RAFT_ADDRESS` -RocksDB 缓存位于 Java 堆外。不设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` 时,将按容器内存限制自动计算;若容器未设置限制,则按宿主机(或 Docker Desktop 虚拟机)内存计算。 -前提是加载到的 `rocksdb.total_memory_size` 为 `auto`(或缺失、为 `0`):沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,会保留该预算,PD 保存并下发给各 Store 的副本也是如此, -请按 [HStore 配置参考](/cn/docs/quickstart/hugegraph/hugegraph-hstore/)中 `rocksdb.total_memory_size` 一行的升级说明处理。 -上面的 Compose 文件默认未设置内存限制,每个 Store 均会按整台宿主机估算预算;多 Store 共用单机时,建议为服务配置内存限制,或将该变量显式设为字节数。 -需要注意,容器限制仅约束 RocksDB 预算:Store 启动脚本仍会按宿主机空闲内存自动分配 `-Xmx`(最多 2 GiB),并将直接内存设为相同大小。 -因此请同步通过 `JAVA_OPTS`(镜像会以 `-j` 传给启动脚本)显式约束 JVM 内存,例如容器限制为 4 GiB 时可设置 `JAVA_OPTS="-Xmx1g -XX:MaxDirectMemorySize=512m"`; -若直接使用默认值,堆内存、直接内存、1.5 GiB 预留加上 256 MiB 最低预算总和已超过 2 GiB。详见 [HStore 配置参考](/cn/docs/quickstart/hugegraph/hugegraph-hstore/)中的 `rocksdb.total_memory_size`。 +RocksDB 缓存使用堆外内存。未设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` 时,系统将优先根据容器内存限制(cgroup)自动计算;未配置容器限制时则按宿主机内存推导。 +由于默认 Docker Compose 未限制容器内存,各 Store 均会以宿主机整机容量估算;若单机部署多个 Store,建议显式配置容器内存限制,或直接为该变量指定字节数。 +此外,Store 启动脚本默认会根据系统空闲内存分配 JVM 堆与直接内存(上限各 2 GiB)。建议结合容器规格,通过 `JAVA_OPTS` 显式约束 JVM 内存(例如容器为 4 GiB 时配置 `JAVA_OPTS="-Xmx1g -XX:MaxDirectMemorySize=512m"`)。详细推导公式见 [HStore 配置参考](/cn/docs/quickstart/hugegraph/hugegraph-hstore/#rocksdb-memory-budget)。 ### Server 变量 diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index 26760e73d5..a299835a51 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -152,7 +152,7 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ #### 4.1 application.yml -以下配置参考表核对的是 server 主线提交 `2f827d6e8c9c62ae858f2fc122b3a192d015e2f4` 的发行目录值和 Java 默认注入值,其中 `rocksdb.total_memory_size`、`rocksdb.write_buffer_size` 和 `raft.disruptorBufferSize` 三行核对的是合入 apache/hugegraph#3355 的提交 `edd7cee77`;1.7.0 发布包请使用其版本标签对应的配置文件,不要跨版本覆盖。 +以下配置参考表核对的是 Server 主线提交的发行目录值和 Java 默认注入值;1.7.0 发布包请使用其版本标签对应的配置文件,不要跨版本覆盖。 #### 4.2 application-pd.yml @@ -213,12 +213,20 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ | 配置项 | 当前主线目录值 | 代码默认值 | 说明 | |--------|--------|------------|------| -| `rocksdb.total_memory_size` | `auto` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的内存预算,取值为 `auto` 或字节数,按 `write_buffer_ratio` 分为写缓存和 block cache。两者都在堆外,不受 `-Xmx` 限制,另有固定 1 GiB 的 Raft 日志缓存不计入其中;所有分区的 Raft 日志 memtable 共用一个 256 MiB 的写缓冲,并计入这 1 GiB 缓存,不会额外占用内存。该预算是容量目标,并非硬上限:`rocksdb.write_buffer_allow_stall` 默认为 `false`,写缓存写满时不会阻塞写入,因此在大量写入时本地内存可能超出预算,容器仍可能被 OOM 终止。将其设为 `true` 能更严格地守住上限,但会增加写入延迟。取值为 `auto` 时(缺失、空值或 `0` 与之等同),取物理内存减去 JVM 最大堆、`MaxDirectMemorySize`(未设置时按堆大小计算;`bin/start-hugegraph-store.sh` 默认把它设为所选的堆大小;`-j` 自行设置了 `-Xmx`、`-XX:MaxHeapSize` 或该上限,或 `JAVA_TOOL_OPTIONS`、`JDK_JAVA_OPTIONS` 设置了该上限时不再添加)和 1.5 GiB 预留(1 GiB Raft 日志缓存加 512 MiB 其他本地内存)后剩余部分的一半,并限制在 256 MiB 到 32 GB 之间;容器内设置了内存限制时以 cgroup 内存限制为准,Store 能看到的父 cgroup 上的限制也会计入;否则以宿主机内存为准。Store 看不到的限制(例如私有 cgroup 命名空间之上的限制)不会被猜测:Store 使用可见的最低限制并输出警告,此时请显式设置该值。启动日志会输出预算及其模式、物理内存限制及来源、堆、直接内存、预留,以及 cgroup 视图是否完整。JVM 和 cgroup 文件都拿不到可用的内存大小时,改用 JVM 最大堆,同样限制在 256 MiB 到 32 GB 之间。沿用旧版本的 `application-pd.yml` 仍是 `32000000000`,需改为 `auto` 才会自动计算。PD 模式下 PD 还可能保存一份副本(KV 键 `S:FS`),由第一个启动的 Store 上传。Store 只有在没有本地文件,或本地文件不比 `S:Timestamp` 新时,才会加载这份副本并覆盖本地文件;上传时不会写入时间戳,缺失的时间戳按 0 处理,因此只修改 `S:FS` 不会生效。升级时,请在每个 Store 上设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` 或 `-Drocksdb.total_memory_size=auto` 并重启,Store 会让这个值优先于本地文件和 PD 的副本。也可以先在 `S:FS` 中把该值改为 `0` 或删除它,再通过 PD 的 KV 服务(gRPC `KvService.put`)把当前时间(毫秒)写入 `S:Timestamp`:所有运行中的 Store 会重新加载 PD 的副本、覆盖本地文件,并在 60 秒内运行 `bin/restart-hugegraph-store.sh`,因此所有 Store 会在同一分钟内重启;已停止的 Store 会在下次启动时加载它。只有所有 Store 都已运行支持 `auto` 的版本后,才能在 `S:FS` 中写入 `auto`:旧版本无法解析 `auto`,会启动失败,而 `0` 在旧版本中按原来的堆大小默认值处理。PD 中原本没有副本、由新版本 Store 上传的副本也是如此,因为发行的配置文件写的是 `auto`。自行重启时不带任何参数,最初启动时的 `-j` 和 `-g` 会丢失,启动脚本改用默认堆大小;只有导出的 `JAVA_OPTIONS` 中的 JVM 设置会保留,因此对使用自定义堆大小启动的 Store,建议采用逐个 Store 覆盖的方式。Docker 中的 Store 根本不会重启:Java 是入口脚本的最后一个进程,停止它会使容器连同其中的重启脚本一起以退出码 143 退出。设置了重启策略的容器(Compose 文件使用 `restart: unless-stopped`)会通过入口脚本重新启动,并保留 `JAVA_OPTS`;像 3.3 节中独立运行的 `docker run -d` 示例那样未指定 `--restart` 启动的容器,则会一直处于停止状态。在 Docker 中请设置 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE`,不要写入 `S:Timestamp`。其他值按原值使用,且必须是纯字节数,最小为 `1048576`(1 MiB);更小的值或 `4GB` 之类的值会导致启动失败。 | +| `rocksdb.total_memory_size` | `auto` | 自动计算,见说明 | 本节点所有 RocksDB 实例共享的堆外内存预算(取值为 `auto` 或具体字节数)。详细自动推导规则见下方说明。 | | `rocksdb.write_buffer_size` | `32000000` | 自动推导,见说明 | memtable 大小,单位字节。缺失或为 `0` 时取 `total_memory_size / 1000`,最小为 `1048576`(1 MiB)。 | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | 落盘前合并的 memtable 数量。 | | `rocksdb.write_buffer_ratio` | 未设置 | `0.66` | `total_memory_size` 中分配给写缓存的比例,其余作为 block cache。 | {.td-table--wide} +### RocksDB 内存预算自动计算规则 {#rocksdb-memory-budget} + +> [!NOTE] +> 当 `rocksdb.total_memory_size` 设置为 `auto`(或未设置、为 `0`)时,系统自动计算堆外 RocksDB 缓存预算: +> - **推导公式**:以物理内存(容器环境下优先取可见的 cgroup 限制)减去 JVM 最大堆(`-Xmx`)、直接内存(`MaxDirectMemorySize`)和 1.5 GiB 系统预留后,取剩余量的一半,范围钳制在 256 MiB 至 32 GiB 之间。 +> - **写缓存与 Block Cache**:根据 `rocksdb.write_buffer_ratio`(默认 0.66)切分。该预算为软目标容量;大量并发写入时若需严格限制内存可将 `rocksdb.write_buffer_allow_stall` 设为 `true`。 +> - **升级与覆盖**:历史配置若保留了旧固定值 `32000000000`,可通过设置环境变量 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` 或启动参数 `-Drocksdb.total_memory_size=auto` 启用自动推导。 + `org/apache/hugegraph/rocksdb/access/RocksDBOptions.java` 中定义的其他选项都可以加在同一个 `rocksdb:` 块下,例如 `rocksdb.max_background_jobs`、`rocksdb.level0_file_num_compaction_trigger` 或 `rocksdb.bloom_filter_bits_per_key`。 **线程池** diff --git a/content/en/docs/guides/hugegraph-docker-cluster.md b/content/en/docs/guides/hugegraph-docker-cluster.md index c8f5a8514c..b17f7e7236 100644 --- a/content/en/docs/guides/hugegraph-docker-cluster.md +++ b/content/en/docs/guides/hugegraph-docker-cluster.md @@ -201,16 +201,10 @@ The following tables describe current master entrypoints and Compose files, not > **Deprecated aliases**: `PD_ADDRESS` → `HG_STORE_PD_ADDRESS`, `GRPC_HOST` → `HG_STORE_GRPC_HOST`, `RAFT_ADDRESS` → `HG_STORE_RAFT_ADDRESS`. -RocksDB caches live outside the Java heap. Leave `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` unset to size them from the container memory limit, or from the -memory of the Docker host (such as the Docker Desktop VM) when the container has no limit. That needs the loaded -`rocksdb.total_memory_size` to be `auto` (or missing, or `0`): an `application-pd.yml` kept from an earlier release still sets `32000000000` -and keeps that budget, as does the copy PD keeps and hands to every Store, so follow the upgrade note under `rocksdb.total_memory_size` in the -[HStore configuration reference](/docs/quickstart/hugegraph/hugegraph-hstore/). The Compose files above set no limit, so each Store sizes its budget -from the whole host; set a memory limit on Store services or set this variable to a plain number of bytes when several Stores share one host. -A container limit bounds only the RocksDB budget: the Store launcher still picks `-Xmx` from the host's free memory (up to 2 GiB) and sets direct memory to the -same size. Size those for the limit too through `JAVA_OPTS` (passed to the launcher as `-j`), for example `JAVA_OPTS="-Xmx1g -XX:MaxDirectMemorySize=512m"` for -a 4 GiB container limit; with defaults, heap plus direct memory, the 1.5 GiB reserve, and the 256 MiB minimum budget already exceed a 2 GiB limit. -See `rocksdb.total_memory_size` in the [HStore configuration reference](/docs/quickstart/hugegraph/hugegraph-hstore/). +RocksDB caches use off-heap memory. When `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` is unset, the system sizes the cache automatically from the container memory limit (cgroup), or falls back to the host memory if no container limit is configured. +Because the default Docker Compose file does not set a memory limit, each Store sizes its budget using the host's full capacity. When deploying multiple Stores on a single host, specify a container memory limit or set this variable directly to an explicit byte value. +Additionally, the Store startup script sizes the JVM heap and direct memory based on host free memory by default (up to 2 GiB each). Constrain JVM memory explicitly via `JAVA_OPTS` to match your container budget (for example, `JAVA_OPTS="-Xmx1g -XX:MaxDirectMemorySize=512m"` for a 4 GiB container). See the [HStore configuration reference](/docs/quickstart/hugegraph/hugegraph-hstore/#rocksdb-memory-budget) for the detailed calculation rules. + ### Server Variables diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index 4f943cfbcb..12086e840c 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -153,7 +153,7 @@ Store reads two files under `conf/`. Release 1.7.0 sources: [application.yml](ht #### 4.1 application.yml -The following tables were checked against distribution values and Java defaults at Server master commit `2f827d6e8c9c62ae858f2fc122b3a192d015e2f4`, except the `rocksdb.total_memory_size`, `rocksdb.write_buffer_size` and `raft.disruptorBufferSize` rows, which were checked at commit `edd7cee77`, where apache/hugegraph#3355 was merged. Use tagged configuration for release 1.7.0 without replacing it with another version's files. +The following tables were checked against distribution values and Java defaults at Server master. Use tagged configuration for release 1.7.0 without replacing it with another version's files. #### 4.2 application-pd.yml @@ -215,12 +215,20 @@ This file contains only `rocksdb` and Actuator exposure settings. | Key | Master distribution value | Code default | Meaning | |-----|---------|--------------|---------| -| `rocksdb.total_memory_size` | `auto` | Auto-sized, see Meaning | Memory budget, `auto` or a number of bytes, shared by all RocksDB instances on this node, split into the write cache and the block cache by `write_buffer_ratio`. Both caches are off-heap, so `-Xmx` does not limit them, and a fixed 1 GiB Raft log cache comes on top; the Raft log memtables of all partitions share a 256 MiB write buffer charged to that cache, so they don't add to it. The budget is a sizing target, not a hard cap: with `rocksdb.write_buffer_allow_stall` at its default `false`, writes are not stalled when the write cache is full, so native memory can exceed the budget under heavy writes and a container can still be OOM-killed. Setting it to `true` holds the cap more firmly at the cost of write latency. With `auto` (a missing, blank or `0` value means the same), the node uses half of the memory left after the JVM max heap, `MaxDirectMemorySize` (counted as the heap size when unset; `bin/start-hugegraph-store.sh` sets it to the heap it picks unless `-j` sets `-Xmx`, `-XX:MaxHeapSize` or the cap itself, or `JAVA_TOOL_OPTIONS` or `JDK_JAVA_OPTIONS` sets the cap) and a 1.5 GiB reserve (the 1 GiB Raft log cache plus 512 MiB for other native memory), clamped to 256 MiB to 32 GB. In a container the starting point is the cgroup memory limit when one is set, including a limit set only on a parent cgroup the Store can see; otherwise it is the host's memory. A limit the Store can't see, such as one above a private cgroup namespace, isn't guessed: the Store uses the lowest visible limit and logs a warning, so pin the value there. The startup log shows the budget with its mode, physical limit and source, heap, direct memory, reserve and whether the cgroup view is complete. If neither the JVM nor the cgroup files report a usable memory size, the budget falls back to the JVM max heap, clamped to the same 256 MiB to 32 GB. An `application-pd.yml` kept from an earlier release still sets `32000000000`; set it to `auto` to auto-size. In PD mode PD may also keep a copy (KV key `S:FS`), uploaded by the first Store to start. A Store loads that copy over its local file only when it has no local file or the file is no newer than `S:Timestamp`; the upload sets no timestamp, and a missing one counts as 0, so changing `S:FS` alone changes nothing. To upgrade, set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` or `-Drocksdb.total_memory_size=auto` on each Store and restart it; the Store keeps this value above both its local file and PD's copy. Alternatively, set the value to `0` or remove it in `S:FS`, then put the current time in milliseconds into `S:Timestamp` through PD's KV service (gRPC `KvService.put`): every running Store then reloads PD's copy, writes it over its local file and runs `bin/restart-hugegraph-store.sh` within 60 seconds, so all Stores restart within the same minute, and a stopped Store loads it at its next start. Put `auto` in `S:FS` only once every Store runs a release that supports it: an earlier release can't parse `auto` and fails to start, while it reads `0` as its old heap-sized default. The same holds for a copy uploaded by a newer Store when PD had none, since the shipped file says `auto`. The self-restart passes no arguments, so `-j` and `-g` from the original start are dropped and the launcher's default heap applies; only JVM settings in an exported `JAVA_OPTIONS` carry over, so prefer the per-Store override on Stores started with a custom heap. A Docker Store doesn't restart at all: Java is the entrypoint's last process, so stopping it ends the container with exit code 143 and the restart script with it. A container with a restart policy (the Compose files use `restart: unless-stopped`) comes back through the entrypoint and keeps `JAVA_OPTS`; one started like the standalone `docker run -d` example in section 3.3, which has no `--restart`, stays down. On Docker, set `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE` instead of publishing `S:Timestamp`. Any other value is used as given and must be plain bytes, at least `1048576` (1 MiB); a smaller value or one such as `4GB` stops startup. | +| `rocksdb.total_memory_size` | `auto` | Auto-sized, see note | Off-heap memory budget shared across all RocksDB instances on this node (`auto` or byte count). See below for derivation rules. | | `rocksdb.write_buffer_size` | `32000000` | Derived, see Meaning | Memtable size in bytes. When absent or `0`, the node uses `total_memory_size / 1000`, at least `1048576` (1 MiB). | | `rocksdb.min_write_buffer_number_to_merge` | `16` | `16` | Number of memtables merged together before a flush. | | `rocksdb.write_buffer_ratio` | not set | `0.66` | Share of `total_memory_size` given to the write cache. The rest becomes the block cache. | {.td-table--wide} +### RocksDB memory budget sizing rules {#rocksdb-memory-budget} + +> [!NOTE] +> When `rocksdb.total_memory_size` is set to `auto` (or absent / `0`), the node sizes the off-heap RocksDB budget automatically: +> - **Calculation Formula**: Takes physical memory (using the visible cgroup limit in containers) minus JVM maximum heap (`-Xmx`), direct memory (`MaxDirectMemorySize`), and a 1.5 GiB system reserve, then allocates half the remainder, clamped between 256 MiB and 32 GiB. +> - **Write Cache & Block Cache**: Budget is divided according to `rocksdb.write_buffer_ratio` (default 0.66). This is a soft capacity target; under heavy concurrent write loads, set `rocksdb.write_buffer_allow_stall` to `true` to enforce strict memory bounds. +> - **Upgrading Existing Deployments**: Deployments carrying legacy `32000000000` can enable auto-sizing by setting environment variable `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` or JVM property `-Drocksdb.total_memory_size=auto`. + Any other option defined in `org/apache/hugegraph/rocksdb/access/RocksDBOptions.java` can be added under the same `rocksdb:` block, for example `rocksdb.max_background_jobs`, `rocksdb.level0_file_num_compaction_trigger` or `rocksdb.bloom_filter_bits_per_key`. **Thread pools** From 06e75d3d732ce662ed3763abb73bcde6438c2bbe Mon Sep 17 00:00:00 2001 From: ayanx Date: Fri, 30 Oct 2026 12:43:45 +0530 Subject: [PATCH 23/23] docs(store): give the auto-sizing ceiling as 32 GB and say what allow-stall bounds --- content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md | 4 ++-- content/en/docs/quickstart/hugegraph/hugegraph-hstore.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index a299835a51..bb0c9e9496 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -223,8 +223,8 @@ Store 从 `conf/` 读取两个配置文件。1.7.0 发布标签的源文件见 [ > [!NOTE] > 当 `rocksdb.total_memory_size` 设置为 `auto`(或未设置、为 `0`)时,系统自动计算堆外 RocksDB 缓存预算: -> - **推导公式**:以物理内存(容器环境下优先取可见的 cgroup 限制)减去 JVM 最大堆(`-Xmx`)、直接内存(`MaxDirectMemorySize`)和 1.5 GiB 系统预留后,取剩余量的一半,范围钳制在 256 MiB 至 32 GiB 之间。 -> - **写缓存与 Block Cache**:根据 `rocksdb.write_buffer_ratio`(默认 0.66)切分。该预算为软目标容量;大量并发写入时若需严格限制内存可将 `rocksdb.write_buffer_allow_stall` 设为 `true`。 +> - **推导公式**:以物理内存(容器环境下优先取可见的 cgroup 限制)减去 JVM 最大堆(`-Xmx`)、直接内存(`MaxDirectMemorySize`)和 1.5 GiB 系统预留后,取剩余量的一半,范围钳制在 256 MiB 至 32 GB(`32000000000` 字节)之间。 +> - **写缓存与 Block Cache**:根据 `rocksdb.write_buffer_ratio`(默认 0.66)切分。该预算为软目标容量;大量并发写入时可将 `rocksdb.write_buffer_allow_stall` 设为 `true`,以增加写入延迟为代价把写缓存限制在其份额内。Block cache 并非严格上限,本地内存仍可能超出预算,容器仍可能被 OOM 终止。 > - **升级与覆盖**:历史配置若保留了旧固定值 `32000000000`,可通过设置环境变量 `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` 或启动参数 `-Drocksdb.total_memory_size=auto` 启用自动推导。 `org/apache/hugegraph/rocksdb/access/RocksDBOptions.java` 中定义的其他选项都可以加在同一个 `rocksdb:` 块下,例如 `rocksdb.max_background_jobs`、`rocksdb.level0_file_num_compaction_trigger` 或 `rocksdb.bloom_filter_bits_per_key`。 diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index 12086e840c..14a3bab6f7 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -225,8 +225,8 @@ This file contains only `rocksdb` and Actuator exposure settings. > [!NOTE] > When `rocksdb.total_memory_size` is set to `auto` (or absent / `0`), the node sizes the off-heap RocksDB budget automatically: -> - **Calculation Formula**: Takes physical memory (using the visible cgroup limit in containers) minus JVM maximum heap (`-Xmx`), direct memory (`MaxDirectMemorySize`), and a 1.5 GiB system reserve, then allocates half the remainder, clamped between 256 MiB and 32 GiB. -> - **Write Cache & Block Cache**: Budget is divided according to `rocksdb.write_buffer_ratio` (default 0.66). This is a soft capacity target; under heavy concurrent write loads, set `rocksdb.write_buffer_allow_stall` to `true` to enforce strict memory bounds. +> - **Calculation Formula**: Takes physical memory (using the visible cgroup limit in containers) minus JVM maximum heap (`-Xmx`), direct memory (`MaxDirectMemorySize`), and a 1.5 GiB system reserve, then allocates half the remainder, clamped between 256 MiB and 32 GB (`32000000000` bytes). +> - **Write Cache & Block Cache**: Budget is divided according to `rocksdb.write_buffer_ratio` (default 0.66). This is a soft capacity target; under heavy concurrent write loads, set `rocksdb.write_buffer_allow_stall` to `true` to hold the write cache to its share at the cost of write latency. The block cache is not a strict limit, so native memory can still exceed the budget and a container can still be OOM-killed. > - **Upgrading Existing Deployments**: Deployments carrying legacy `32000000000` can enable auto-sizing by setting environment variable `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto` or JVM property `-Drocksdb.total_memory_size=auto`. Any other option defined in `org/apache/hugegraph/rocksdb/access/RocksDBOptions.java` can be added under the same `rocksdb:` block, for example `rocksdb.max_background_jobs`, `rocksdb.level0_file_num_compaction_trigger` or `rocksdb.bloom_filter_bits_per_key`.