Ebs: Consider EC2 block device mappings for initiator device names - #509
Open
mateo-moon wants to merge 1 commit into
Open
Ebs: Consider EC2 block device mappings for initiator device names#509mateo-moon wants to merge 1 commit into
mateo-moon wants to merge 1 commit into
Conversation
On instances built on the AWS Nitro system, attached EBS volumes are
exposed by the NVMe driver as /dev/nvme*n1, so pruning the candidate
device letters from lsblk kernel names with "sd"/"xvd" prefixes never
removes anything and every attach requests /dev/sdz. The first attach
per instance succeeds; any further attach fails permanently with
InvalidParameterValue ("Attachment point /dev/sdz is already in use"),
limiting EBS_INIT nodes to a single attached volume.
Additionally prune the letters of device names that the EC2 API reports
as used in the instance's block device mappings, which is authoritative
regardless of guest-side device naming.
Fixes LINBIT#508
Author
|
E2E-validated live on a 3-AZ Kubernetes cluster (arm64/Nitro, v1.34.1 base + all four of our EBS fix PRs): full validation report in #506 (comment) (latest comment). Highlights relevant to this PR are listed there; the four fixes together make native EBS work end-to-end on latest. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #508.
Problem
On instances built on the AWS Nitro system, attached EBS volumes are exposed by the NVMe driver as
/dev/nvme*n1.EbsInitiatorProvider.findUnusedDevice()prunes candidate device letters only from lsblk kernel names withsd/xvdprefixes, so on Nitro nothing is ever pruned, the descending scan always returnsz, and every attach requests/dev/sdz. The first attach per instance succeeds; every further concurrent attach fails permanently withInvalidParameterValue ("Attachment point /dev/sdz is already in use")— limitingEBS_INITnodes to a single attached volume. Details and live transcript in #508.Fix
Additionally prune the letters of device names that EC2 itself reports as used in the instance's block device mappings (
DescribeInstances→blockDeviceMappings[].deviceName). This is authoritative regardless of guest-side device naming, and also covers device names EC2 returns without the/dev/prefix. The lsblk-based pruning is kept for the pre-Nitro/Xen case.Testing
:satellite:compileJava+:satellite:checkstyleMainclean (no new warnings).m6g.xlarge, i.e. Nitro): planned by cherry-picking onto v1.30.4, since the EBS path on current master is not exercisable end-to-end until Controller never dispatches resources to EBS_TARGET special satellites on v1.33.3/v1.34.1 (works on v1.30.4) — native EBS volumes never created #506 (no resource dispatch to EBS_TARGET satellites) is resolved — the fix applies to master cleanly and the touched code is identical between v1.30.4 and master. Happy to report results in this PR.