Skip to content

Fix Eigen::Ref<MatrixXd> Jacobian output args in MATLAB wrapper#189

Merged
dellaert merged 1 commit into
borglab:masterfrom
thatdudegrantt:fix/matlab-eigen-ref-jacobians
Jul 22, 2026
Merged

Fix Eigen::Ref<MatrixXd> Jacobian output args in MATLAB wrapper#189
dellaert merged 1 commit into
borglab:masterfrom
thatdudegrantt:fix/matlab-eigen-ref-jacobians

Conversation

@thatdudegrantt

Copy link
Copy Markdown
Contributor

Fixes borglab/gtsam#2492

Problem

When calling methods like Pose3.transformFrom with Jacobian output
arguments from MATLAB, every overload using Eigen::Ref<Eigen::MatrixXd>
was permanently unreachable, producing:

Error using gtsam.Pose3/transformFrom
Arguments do not match any overload of function gtsam.Pose3.transformFrom

Root cause: Eigen::Ref<Eigen::MatrixXd> is parsed as a TemplatedType
with typename.name == 'Ref', not a plain Type with is_ref set. The
wrapper codegen had no special handling for this pattern and emitted:

isa(varargin{2}, 'Eigen.RefMatrixXd')

Eigen.RefMatrixXd is not a MATLAB class, so this check always returns
false. On the C++ MEX side, the wrapper also incorrectly tried to
unwrap_shared_ptr<MatrixXd> from in[], which would segfault even
if dispatch somehow succeeded.

Fix

gtwrap/matlab_wrapper/mixins.py

  • Add eigen_ref_types tuple ('MatrixXd')
  • Add is_eigen_ref() to detect Ref<MatrixXd> TemplatedType args

gtwrap/matlab_wrapper/wrapper.py

  • _wrap_method_check_statement: exclude Ref args from length(varargin)
    count and isa() checks; add nargout == N gate instead
  • _unwrap_argument: allocate Eigen::MatrixXd locals for Ref args
    instead of reading from in[]
  • _wrapper_unwrap_arguments: don't advance arg_id for Ref args
    (they don't consume an in[] slot); capture eigen_ref_args before
    method can be reassigned to a string for templated methods
  • wrap_collector_function_return: write Jacobian locals to out[1],
    out[2], ... after the primary return value
  • generate_collector_function: exclude Ref args from checkArguments
    nargin count

Generated output (before/after)

Before (broken so Eigen.RefMatrixXd never matches):

if length(varargin) == 3 && isa(varargin{1},'double') && ... && isa(varargin{2},'Eigen.RefMatrixXd')

After (correct, now dispatches on real inputs + nargout):

if length(varargin) == 1 && isa(varargin{1},'double') && size(varargin{1},1)==3 && size(varargin{1},2)==1 && nargout == 3

C++ MEX after:

Eigen::MatrixXd Hself = Eigen::MatrixXd();
Eigen::MatrixXd Hpoint = Eigen::MatrixXd();
out[0] = wrap< Point3 >(obj->transformFrom(point, Hself, Hpoint));
out[1] = wrap< Eigen::MatrixXd >(Hself);
out[2] = wrap< Eigen::MatrixXd >(Hpoint);

Testing

Added test_eigen_ref_jacobians to tests/test_matlab_wrapper.py with
fixture tests/fixtures/eigen_ref.i covering four cases:

  • Instance method with primitive input + 2 Ref args (transformFrom)
  • Instance method with zero real inputs + 1 Ref arg (inverse)
  • Instance method with class-type input + 2 Ref args (between)
  • Static method with 1 Ref arg (Expmap)

All 11 tests pass.

Environment

  • OS: Windows 11 x64
  • Python: 3.14.5
  • wrap commit: (your current branch tip)

Eigen::Ref<MatrixXd> arguments are Jacobian output parameters in C++
but were previously treated as inputs in the MATLAB wrapper, making
all Jacobian overloads permanently unreachable with the error:
'Arguments do not match any overload of function gtsam.Pose3.transformFrom'

Root cause: Ref<MatrixXd> parses as a TemplatedType with typename.name
'Ref', not a plain Type with is_ref set. The wrapper emitted bogus
isa(varargin{N},'Eigen.RefMatrixXd') checks that always return false,
and tried to unwrap_shared_ptr<MatrixXd> from in[] on the C++ side.

Fix:
- Add is_eigen_ref() to CheckMixin to detect Ref<MatrixXd> TemplatedType
- Exclude Ref args from varargin count and isa() checks in .m dispatch
- Add nargout == N gate so Jacobian overload is selected when caller
  requests extra outputs
- Allocate Eigen::MatrixXd locals in MEX instead of reading from in[]
- Write Jacobian locals to out[1], out[2], ... after the primary return
- Exclude Ref args from checkArguments count (nargin validation)

Fixes borglab/gtsam#2492
@dellaert

Copy link
Copy Markdown
Member

Awesome, @thatdudegrantt !!!

@dellaert
dellaert merged commit 1249158 into borglab:master Jul 22, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot Retrieve Jacobians from Pose3 Methods in MATLAB Wrapper

2 participants