Skip to content

Arrow key inputs are ignored on ENABLE_VIRTUAL_TERMINAL_INPUT enabled windows console #914

Description

@YO4

Symptoms similar to those described in #903 also occur in mswin/mingw.
ENABLE_VIRTUAL_TERMINAL_INPUT is normally disabled, but it may be enabled by other processes sharing the console, and it is possible that it is not reset due to a crash or other issue.

Step to reproduce

in irb

Reline::IOGate.instance_exec do
  m = "\0"*4; @GetConsoleMode.call(@hConsoleInputHandle, m);
  @SetConsoleMode.call(@hConsoleInputHandle, m.unpack1("L") | 0x200)
end

Or, for more flexibility,

irb(main):001> Reline::IOGate.class
=> Reline::Windows
irb(main):002* def set_vt_input(f)
irb(main):003*   Reline::IOGate.instance_exec do
irb(main):004*     m = "\0"*4; @GetConsoleMode.call(@hConsoleInputHandle, m);
irb(main):005*     mode = m.unpack1("L")
irb(main):006*     puts "old mode: #{mode.to_s(16)}"
irb(main):007*     new_mode = (mode & ~0x200) | (f ? 0x200 : 0)
irb(main):008*     @SetConsoleMode.call(@hConsoleInputHandle, new_mode)
irb(main):009*     puts "new mode: #{new_mode.to_s(16)}"
irb(main):010*   end
irb(main):011> end
=> :set_vt_input
irb(main):012> set_vt_input(true)
old mode: 1f7
new mode: 3f7
=> nil
irb(main):013> # => cursor movement using the arrow keys is disabled
irb(main):014> set_vt_input(false)
old mode: 3f7
new mode: 1f7
=> nil
irb(main):015> # => cursor movement using the arrow keys is enabled

Under the hood

Reline::Windows converts input key events to the output of getwch() rather than VT sequences (presumably to preserve the specifications from the early stages of development).
When ENABLE_VIRTUAL_TERMINAL_INPUT is enabled, Reline receives input that can be represented as VT sequences as characters rather than events, so no conversion to getwch() code takes place.
Since Reline::Windows bases its key bindings on getwch() codes, it cannot recognize input in the form of VT sequences.

Options for a Solution

  1. Add key bindings for VT sequences to Reline::Windows
  2. Stop using the getwch() code and have Reline::Windows convert key events to VT sequences

Although option 2 introduces more changes, since WSL coexists in Windows environments, this approach is preferable because it allows for the use of a common configuration.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions