Skip to content

Commit ae06ced

Browse files
committed
Coarsen the SUMMA (non-BLAS) axis in mixed retile
The mixed (plain x arena-ToT -> arena-ToT) auto-retile coarsened only the BLAS axes (plain external + contracted K). Extend it to also coarsen the leftover SUMMA external (the arena-ToT operand's external). mixed_retile_config gains tot_external_target (default 16): the SUMMA external is coarsened toward it, coarsen-only (an axis already coarser is kept intact; refine is unsupported). Detection stays automatic and mixed-only, so ToT*ToT still reverts to the stock SUMMA path. Adds g_summa_coarse_{m,n,k}_grid witnesses and tests for both orientations, the default auto path, and the ToT*ToT no-retile guard.
1 parent a65acd4 commit ae06ced

4 files changed

Lines changed: 305 additions & 27 deletions

File tree

src/TiledArray/dist_eval/contraction_eval.h

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,20 @@ inline std::atomic<std::size_t> g_summa_result_merge_count{0};
8181
/// tests assert it is > 0 to prove the grouped active distribution engaged.
8282
/// gop.sum it across ranks for np-correctness.
8383
inline std::atomic<std::size_t> g_summa_proc_h_grouped_calls{0};
84+
85+
/// witness: the COARSE SUMMA-M / -N / -K grid tile counts (the number of T
86+
/// tiles the operands are retiled into on each role), recorded by
87+
/// `ContEngine::init_distribution` whenever a retile plan is active. These
88+
/// directly expose the "retiled operand trange" tile count on each SUMMA axis:
89+
/// coarsening SUMMA-N from 64 fine tiles to 4 coarse tiles records
90+
/// `g_summa_coarse_n_grid == 4`, even though the per-row strided BLAS GEMM count
91+
/// is INVARIANT (a SUMMA external is not a BLAS axis). STORE (not accumulate) --
92+
/// the last active contraction's coarse grid; reset to 0 before the contraction
93+
/// under test. They stay at the reset value on the inactive (stock SUMMA) path.
94+
/// np=1 or symmetric np>1, so the per-rank store agrees across ranks.
95+
inline std::atomic<std::size_t> g_summa_coarse_m_grid{0};
96+
inline std::atomic<std::size_t> g_summa_coarse_n_grid{0};
97+
inline std::atomic<std::size_t> g_summa_coarse_k_grid{0};
8498
#endif
8599

86100
/// \brief Distributed contraction evaluator implementation

src/TiledArray/expressions/cont_engine.h

Lines changed: 44 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -595,19 +595,27 @@ class ContEngine : public BinaryEngine<Derived> {
595595
/// inactive plan. The plan is consumed by later phases; in this phase it is
596596
/// only stored and threaded to the Summa ctor.
597597
/// Synthesize coarse retile targets for the mixed (plain x arena-ToT ->
598-
/// arena-ToT) shape with the env-gated auto-retile. Collapse the PLAIN
599-
/// operand's external -- role M when the plain operand is LEFT, role N when it
600-
/// is RIGHT -- and the contracted (K) axis toward the element sizes in
601-
/// detail::mixed_retile_config (0 => single tile / full collapse). The fused
602-
/// (H) axes and the arena-ToT operand's external (the other of M/N) are left
603-
/// EMPTY, i.e. kept intact at U. The role axes are extracted by the SAME
604-
/// positional H/M/N/K partition that make_retile_plan consults (nf leading
605-
/// fused modes; the last nc left axes are K; the remaining left axes are M;
606-
/// the right axes past nf+nc are N), so the synthesized targets are in
607-
/// canonical H/M/N/K space -- identical to what a .retile() caller supplies.
608-
/// This reproduces the verified mixed_T_x_ToT_coarsen_MK (plain left ->
609-
/// coarsen M+K) and mixed_ToT_x_T_coarsen_NK (plain right -> coarsen N+K)
610-
/// target strategy.
598+
/// arena-ToT) shape with the env-gated auto-retile. Per
599+
/// detail::mixed_retile_config, coarsen THREE roles:
600+
/// - the PLAIN operand's external (role M when the plain operand is LEFT,
601+
/// role N when it is RIGHT) toward plain_external_target (default 0 =>
602+
/// single tile / full collapse);
603+
/// - the contracted (K) axis toward contracted_target (default 0 => collapse);
604+
/// - the arena-ToT operand's external -- the LEFTOVER SUMMA axis (role N
605+
/// when the plain operand is LEFT, role M when it is RIGHT) -- toward
606+
/// tot_external_target (default 16).
607+
/// The fused (H) axes are left EMPTY (kept intact at U). Every target is fed
608+
/// through coarsen_tr1, which COARSENS ONLY (merges U tiles onto existing U
609+
/// boundaries; never refines), so a leftover-SUMMA axis already coarser than
610+
/// the target is kept intact rather than refined (refine is unsupported).
611+
/// The role axes are extracted by the SAME positional H/M/N/K partition that
612+
/// make_retile_plan consults (nf leading fused modes; the last nc left axes
613+
/// are K; the remaining left axes are M; the right axes past nf+nc are N), so
614+
/// the synthesized targets are in canonical H/M/N/K space -- identical to what
615+
/// a .retile() caller supplies. This is the default verified by
616+
/// mixed_T_x_ToT_coarsen_MK_retile_N (plain left -> collapse M+K, coarsen N to
617+
/// 16) and mixed_ToT_x_T_coarsen_NK_retile_M (plain right -> collapse N+K,
618+
/// coarsen M to 16).
611619
void synthesize_mixed_targets_(bool left_is_plain,
612620
const math::GemmHelper& outer_gh,
613621
std::vector<TiledRange1>& tH,
@@ -624,14 +632,24 @@ class ContEngine : public BinaryEngine<Derived> {
624632
const std::size_t ext_target =
625633
detail::mixed_retile_config.plain_external_target;
626634
const std::size_t k_target = detail::mixed_retile_config.contracted_target;
635+
const std::size_t tot_ext_target =
636+
detail::mixed_retile_config.tot_external_target;
627637
if (left_is_plain) {
628-
// plain operand is LEFT => its external is role M (left outer axes).
638+
// plain operand is LEFT => its external is role M (left outer axes),
639+
// collapsed; the arena-ToT external is role N (right outer axes),
640+
// coarsened to tot_ext_target.
629641
for (unsigned int i = nf; i + nc < left_rank; ++i)
630642
tM.push_back(coarsen_tr1(left_U.dim(i), ext_target));
643+
for (unsigned int i = nf + nc; i < right_rank; ++i)
644+
tN.push_back(coarsen_tr1(right_U.dim(i), tot_ext_target));
631645
} else {
632-
// plain operand is RIGHT => its external is role N (right outer axes).
646+
// plain operand is RIGHT => its external is role N (right outer axes),
647+
// collapsed; the arena-ToT external is role M (left outer axes),
648+
// coarsened to tot_ext_target.
633649
for (unsigned int i = nf + nc; i < right_rank; ++i)
634650
tN.push_back(coarsen_tr1(right_U.dim(i), ext_target));
651+
for (unsigned int i = nf; i + nc < left_rank; ++i)
652+
tM.push_back(coarsen_tr1(left_U.dim(i), tot_ext_target));
635653
}
636654
// K (contracted) axes are the last nc axes of the left operand.
637655
for (unsigned int i = left_rank - nc; i < left_rank; ++i)
@@ -1098,6 +1116,17 @@ class ContEngine : public BinaryEngine<Derived> {
10981116
// owner is rank 0, identical to the old fine-u_grid behavior. (Refine
10991117
// at np>1 was rejected above; only coarsen/identity reaches here.)
11001118
const size_type K_coarse = coarse_K_(K_);
1119+
#ifdef TA_STRIDED_DGEMM_COUNT
1120+
// Witness: expose the coarse SUMMA grid (the retiled operand trange tile
1121+
// counts on each role). A coarsened SUMMA external shows up here even
1122+
// though the per-row strided BLAS GEMM count is invariant.
1123+
TiledArray::detail::g_summa_coarse_m_grid.store(
1124+
static_cast<std::size_t>(M_grid), std::memory_order_relaxed);
1125+
TiledArray::detail::g_summa_coarse_n_grid.store(
1126+
static_cast<std::size_t>(N_grid), std::memory_order_relaxed);
1127+
TiledArray::detail::g_summa_coarse_k_grid.store(
1128+
static_cast<std::size_t>(K_coarse), std::memory_order_relaxed);
1129+
#endif
11011130
// left U layout = [M-axes..., K-axes...]; phase rows = M_grid, cols =
11021131
// K_coarse.
11031132
auto left_phase = proc_grid_.make_row_phase_pmap(K_coarse);

src/TiledArray/expressions/mixed_retile_config.h

Lines changed: 22 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -36,24 +36,39 @@ namespace TiledArray::expressions::detail {
3636
/// "coarsen this role to a single tile spanning the full axis extent".
3737
///
3838
/// HARDCODED ON PURPOSE: the optimal coarsening tracks the physical system
39-
/// size, so a maintainer edits these two constants per target system size and
40-
/// rebuilds. Only the plain-operand BLAS externals and the contracted (K) axis
41-
/// are coarsened; the fused (H) axes and the arena-ToT operand's externals are
42-
/// left at the operands' own tiling. The plain operand's external lands on
43-
/// SUMMA role M when the plain operand is the LEFT argument, and on role N when
44-
/// it is the RIGHT argument; K is always shared and always coarsened. Hence a
45-
/// single "plain external" number covers both orientations.
39+
/// size, so a maintainer edits these constants per target system size and
40+
/// rebuilds.
41+
///
42+
/// The default strategy coarsens THREE roles:
43+
/// 1. the plain-operand BLAS external (SUMMA role M when the plain operand is
44+
/// LEFT, role N when it is RIGHT) -> a single tile (target 0);
45+
/// 2. the contracted (SUMMA-K) axis (always shared) -> a single tile;
46+
/// 3. the arena-ToT operand's external -- the leftover SUMMA axis (role N when
47+
/// the plain operand is LEFT, role M when it is RIGHT) -> tile size 16.
48+
/// The fused (H) axes are left at the operands' own tiling (empty target).
49+
///
50+
/// COARSEN-ONLY (never refine): every target is fed through `coarsen_tr1`,
51+
/// which only ever MERGES consecutive user (U) tiles onto existing U
52+
/// boundaries; it never splits a tile. So a leftover-SUMMA axis whose tiles are
53+
/// ALREADY >= the target (16) is kept intact (no refine -- refine is
54+
/// unsupported here), while a finer axis is merged up toward 16.
4655
struct MixedRetileConfig {
4756
/// target tile size for the plain operand's external (SUMMA-M if the plain
4857
/// operand is LEFT, SUMMA-N if it is RIGHT). 0 => single tile.
4958
std::size_t plain_external_target = 0;
5059
/// target tile size for the contracted (SUMMA-K) axis. 0 => single tile.
5160
std::size_t contracted_target = 0;
61+
/// target tile size for the arena-ToT operand's external -- the leftover
62+
/// SUMMA axis (role N if the plain operand is LEFT, role M if it is RIGHT).
63+
/// Coarsen-only: an axis already coarser than this is left intact (never
64+
/// refined). 0 => single tile.
65+
std::size_t tot_external_target = 16;
5266
};
5367

5468
inline constexpr MixedRetileConfig mixed_retile_config{
5569
/*plain_external_target=*/0, // collapse M (or N) on the plain operand
5670
/*contracted_target=*/0, // collapse K
71+
/*tot_external_target=*/16, // coarsen the leftover SUMMA axis to 16
5772
};
5873

5974
/// Mutable enable flag for the mixed auto-retile gate. Default-initialized once

0 commit comments

Comments
 (0)