feat: detect NUMA region capacity and size host spaces from it by aminaramoon · Pull Request #173 · NVIDIA/cuCascade

aminaramoon · 2026-07-29T15:48:46Z

Topology discovery now enumerates NUMA nodes with their total and free
memory, parsed from /sys/devices/system/node/node/meminfo, and
num_numa_nodes is derived from that list. system_topology_info gains
get_numa_memory_capacity(), get_numa_free_memory(), and
get_total_numa_memory_capacity() lookups.

reservation_manager_configurator gains set_usage_limit_ratio_per_host(),
which sizes each host memory space as a fraction of the capacity of the
NUMA node backing it, mirroring set_usage_limit_ratio_per_gpu(). Absolute
per-host and total-host capacities keep their existing behavior; build()
throws when a fraction is requested and the NUMA capacity is unknown.

Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com

Topology discovery now enumerates NUMA nodes with their total and free memory, parsed from /sys/devices/system/node/node<id>/meminfo, and num_numa_nodes is derived from that list. system_topology_info gains get_numa_memory_capacity(), get_numa_free_memory(), and get_total_numa_memory_capacity() lookups. reservation_manager_configurator gains set_usage_limit_ratio_per_host(), which sizes each host memory space as a fraction of the capacity of the NUMA node backing it, mirroring set_usage_limit_ratio_per_gpu(). Absolute per-host and total-host capacities keep their existing behavior; build() throws when a fraction is requested and the NUMA capacity is unknown. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

copy-pr-bot · 2026-07-29T15:48:50Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

The machine used to develop this has a single NUMA node and both GPUs report numa_node 0, so the numa_node == -1 path was never exercised. Add cases where the GPU has no NUMA affinity: absolute per-host and total-host capacities are still honored verbatim, and a requested fraction throws because it has nothing to resolve against. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

bwyogatama · 2026-07-29T17:12:54Z

/ok to test 9436141

felipeblazing

In general I think we should change most of our host nomenclature to numa_region so that this is clear what we are talking about. This would prevent downstream users from getting confused between the entire host capacity and a numa region

felipeblazing · 2026-07-30T12:45:15Z

+  ///       if the capacity of a NUMA node backing a host space is unknown.
+  /// @note The fraction applies per host space. With `use_host_per_gpu()` and several GPUs
+  ///       on the same NUMA node, each space gets that fraction of the shared node.
+  builder_reference& set_usage_limit_ratio_per_host(double fraction);


Suggested change

builder_reference& set_usage_limit_ratio_per_host(double fraction);

builder_reference& set_usage_limit_ratio_per_numa_region(double fraction);

felipeblazing · 2026-07-30T12:48:15Z

+   * @return Free memory of the node in bytes; 0 if the node is unknown or its free
+   *         memory could not be determined.
+   */
+  [[nodiscard]] std::size_t get_numa_free_memory(int numa_id) const


it feels like if the user sends in the wrong numa_id this and get_numa_memory_capacity will silently fail

felipeblazing · 2026-07-30T12:48:38Z

  return *this;
 }

+builder_reference& reservation_manager_configurator::set_usage_limit_ratio_per_host(double fraction)


Suggested change

builder_reference& reservation_manager_configurator::set_usage_limit_ratio_per_host(double fraction)

builder_reference& reservation_manager_configurator::set_usage_limit_ratio_per_numa_region(double fraction)

felipeblazing · 2026-07-30T13:06:16Z

 std::vector<reservation_manager_configurator::host_info>
-reservation_manager_configurator::extract_host_ids(
-  const std::vector<gpu_info>& gpus, [[maybe_unused]] const system_topology_info& topology) const
+reservation_manager_configurator::extract_host_ids(const std::vector<gpu_info>& gpus,


I might consider renaming this to numa_region_ids to be clearer.

felipeblazing · 2026-07-30T13:07:01Z

-      : (_is_capacity_per_space ? _host_capacity : _host_capacity / host_infos.size());
+  // Absolute capacities are either per-space or split evenly across spaces; a fraction is
+  // always relative to the capacity of the NUMA node backing the space.
+  auto host_capacity_of = [&](const host_info& info) -> std::size_t {


Suggested change

auto host_capacity_of = [&](const host_info& info) -> std::size_t {

auto numa_region_capacity_of = [&](const host_info& info) -> std::size_t {

felipeblazing · 2026-07-30T13:08:42Z

+ *
+ * @return NUMA node information sorted by node id; empty if unavailable.
+ */
+std::vector<numa_topology_info> discover_numa_nodes()


Is there a way that we can figure out during numa_node discovery whether or not a numa_node is actually gpu memory as happens on the dgx station for example where the gpu memory shows up as numa region 1?

Bump cucascade to the NVIDIA/cuCascade#173 head, which discovers per-NUMA memory capacity and adds set_usage_limit_ratio_per_host(). Host capacity becomes a fraction/bytes variant: capacity_fraction sizes each host space as a fraction of its backing NUMA node's RAM (validated (0,1]), while capacity_bytes keeps the absolute per-node form and wins if both are set. The built-in default changes from 8 GiB per node to 0.8 of each node. Hosts that expose no NUMA meminfo now fail initialization with a clear error instead of silently getting 8 GiB; set capacity_bytes there.

Bump cucascade to the NVIDIA/cuCascade#173 head, which discovers per-NUMA memory capacity and adds set_usage_limit_ratio_per_host(). Host capacity becomes a fraction/bytes variant: capacity_fraction sizes each host space as a fraction of its backing NUMA node's RAM (validated (0,1]), while capacity_bytes keeps the absolute per-node form and wins if both are set. The built-in default changes from 8 GiB per node to 0.9 of each node. Hosts that expose no NUMA meminfo now fail initialization with a clear error instead of silently getting 8 GiB; set capacity_bytes there.

A host memory space covers a single NUMA region, not the whole host, so the per-space builder settings are renamed to say so: set_per_host_capacity -> set_per_numa_region_capacity set_usage_limit_ratio_per_host -> set_usage_limit_ratio_per_numa_region set_reservation_limit_per_host -> set_reservation_limit_per_numa_region set_reservation_fraction_per_host -> set_reservation_fraction_per_numa_region set_downgrade_fractions_per_host -> set_downgrade_fractions_per_numa_region set_total_host_capacity keeps its name: it is the one host-wide setting, divided evenly across the host spaces. The host creation policies are renamed for what they actually select -- the id a host space is identified by, not how many spaces exist: use_host_per_gpu -> use_gpu_id_as_host_id use_host_per_numa -> use_numa_id_as_host_id (default) Under use_gpu_id_as_host_id the space is still bound to that GPU's NUMA node for allocation; only the space id differs. get_numa_memory_capacity() and get_numa_free_memory() now return std::optional so an unknown node id no longer resolves to a silent 0, and find_numa_node() exposes the node itself. NUMA discovery reads node<id>/cpulist and flags a node that has memory but no CPUs as is_device_memory. DGX Station and Grace-Hopper expose GPU HBM as such a node, and CXL expanders do the same; sizing a host space from one would hand out device memory as host memory. Those nodes are excluded from get_total_numa_memory_capacity(), and requesting a capacity fraction against one now throws. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

bwyogatama added breaking feature request New feature or request labels Jul 29, 2026

aminaramoon marked this pull request as ready for review July 29, 2026 18:02

felipeblazing requested changes Jul 30, 2026

View reviewed changes

bwyogatama mentioned this pull request Jul 30, 2026

fix(config): defaults — memory fractions, thread pools, host capacity_fraction sirius-db/sirius#1337

Open

aminaramoon requested a review from felipeblazing July 30, 2026 23:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: detect NUMA region capacity and size host spaces from it - #173

feat: detect NUMA region capacity and size host spaces from it#173
aminaramoon wants to merge 3 commits into
NVIDIA:mainfrom
aminaramoon:numa_capacity

aminaramoon commented Jul 29, 2026

Uh oh!

copy-pr-bot Bot commented Jul 29, 2026

Uh oh!

bwyogatama commented Jul 29, 2026

Uh oh!

felipeblazing left a comment

Uh oh!

felipeblazing Jul 30, 2026

Uh oh!

felipeblazing Jul 30, 2026

Uh oh!

felipeblazing Jul 30, 2026

Uh oh!

felipeblazing Jul 30, 2026

Uh oh!

felipeblazing Jul 30, 2026

Uh oh!

felipeblazing Jul 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	builder_reference& set_usage_limit_ratio_per_host(double fraction);
	builder_reference& set_usage_limit_ratio_per_numa_region(double fraction);

	builder_reference& reservation_manager_configurator::set_usage_limit_ratio_per_host(double fraction)
	builder_reference& reservation_manager_configurator::set_usage_limit_ratio_per_numa_region(double fraction)

	auto host_capacity_of = [&](const host_info& info) -> std::size_t {
	auto numa_region_capacity_of = [&](const host_info& info) -> std::size_t {

Uh oh!

Conversation

aminaramoon commented Jul 29, 2026

Uh oh!

copy-pr-bot Bot commented Jul 29, 2026

Uh oh!

bwyogatama commented Jul 29, 2026

Uh oh!

felipeblazing left a comment

Choose a reason for hiding this comment

Uh oh!

felipeblazing Jul 30, 2026

Choose a reason for hiding this comment

Uh oh!

felipeblazing Jul 30, 2026

Choose a reason for hiding this comment

Uh oh!

felipeblazing Jul 30, 2026

Choose a reason for hiding this comment

Uh oh!

felipeblazing Jul 30, 2026

Choose a reason for hiding this comment

Uh oh!

felipeblazing Jul 30, 2026

Choose a reason for hiding this comment

Uh oh!

felipeblazing Jul 30, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants