Bump minimum Java version to 17 and remove pre-17 workarounds by iemejia · Pull Request #3579 · apache/parquet-java

iemejia · 2026-05-23T08:47:25Z

Summary

Raise the minimum build/runtime requirement to Java 17 and remove workarounds and tooling that are no longer needed. The Hadoop 3.4.x bump and the vectored-I/O buffer fix that previously lived here have been split out (the Hadoop upgrade will come via #3559).

Java 17 baseline

Set maven.compiler.release to 17 and add an enforcer rule requiring Java 17+ to build.
Remove the extraJavaTestArgs (--add-opens / -XX flags) that were needed to run the tests on newer JDKs from a Java 11 baseline.
Drop the per-module maven.compiler.release=17 override in parquet-encoding-vector; it is now the project-wide default.
Simplify the CleanUtil javadoc.
CI: build only on JDK 17 (drop the JDK 11 matrix entry) and stop exporting extraJavaTestArgs.
Update the README to state the Java 17 requirement.

Spotless upgrade (2.46.1 -> 3.8.0)

Spotless 2.46.1 calls com.sun.tools.javac.util.Log methods that were removed in JDK 25, causing NoSuchMethodError during formatting. Spotless 3.8.0 is compatible with JDK 25. The switch/case comment indentation changes are produced by the new formatter version.

Remove obsolete japicmp exclude for `IndexIterator`

The exclude worked around a false-positive source incompatibility reported by japicmp 0.23.1 on the synthetic/bridge methods of PrimitiveIterator.OfInt (#3388). japicmp was upgraded to 0.26.1 (#3592), which handles those methods correctly, so the workaround is no longer needed. Verified that japicmp:cmp (with breakBuildOnSourceIncompatibleModifications=true) passes with the exclude removed.

Split out (deferred to #3559)

The Hadoop 3.3.0 -> 3.4.3 bump and the related vectored-I/O ByteBuffer leak fix have been removed from this PR. The Java 17 build passes without them (vectored-I/O tests simply skip on Hadoop 3.3.x, with no buffer leaks), so they are not required here and are better handled alongside #3559.

Testing

Verified on JDK 17: full spotless:check, japicmp:cmp (with break-on-incompatible), and parquet-hadoop compile/tests all pass.
Verified the Java 17 build passes with Hadoop 3.3.0.

nastra · 2026-07-01T13:34:16Z

    <shade.prefix>shaded.parquet</shade.prefix>
    <!-- Guarantees no newer classes/methods/constants are used by parquet. -->
-    <hadoop.version>3.3.0</hadoop.version>
+    <hadoop.version>3.4.3</hadoop.version>


we should probably get the hadoop version bump merged first. There's already a PR for that: #3559

Good call. I dropped the Hadoop bump from this PR — I verified the Java 17 build (compile + tests) passes fine on Hadoop 3.3.0 (vectored-I/O tests just skip, no buffer leaks), so it isn't needed here. Let's have the bump land via #3559.

nastra · 2026-07-01T13:35:36Z

    <japicmp.version>0.25.7</japicmp.version>
    <javax.annotation.version>1.3.2</javax.annotation.version>
-    <spotless.version>2.46.1</spotless.version>
+    <spotless.version>3.5.1</spotless.version>


Suggested change

<spotless.version>3.5.1</spotless.version>

<spotless.version>3.8.0</spotless.version>

Done — bumped to 3.8.0 and ran spotless:apply (no formatting changes beyond the version bump). Thanks for the suggestion.

nastra · 2026-07-01T13:36:19Z

-    for (ConsecutivePartList consecutivePart : allParts) {
-      ParquetFileRange currRange = ranges.get(k++);
-      consecutivePart.readFromVectoredRange(currRange, builder);
+    // Use a capturing allocator to track all buffers allocated by Hadoop during vectored reads.


#3559 has an alternative approach for this.

Removed from this PR as well — it's only relevant once vectored I/O is actually exercised (Hadoop 3.4.x). Deferring to your approach in #3559.

Thanks 👍 I don't think we should add code to Parquet to patch open issues on Hadoop. Just cleaned up the PR, I think it is good to go now.

kevinjqliu

Thanks for the PR!

Looks like we can split out some some changes in other PRs, that would be helpful for reviews. I think we should keep this PR as light as possible and only deals changes required for java 17. Anything else we can either split into separate PR and merge first, or drop as a follow up

kevinjqliu · 2026-07-17T05:04:49Z

            <excludes>
              <exclude>${shade.prefix}</exclude>
-              <!-- JDK 11 adds interface methods/bridges on PrimitiveIterator.OfInt; ignore japicmp source incompatibility -->
+              <!-- JDK adds interface methods/bridges on PrimitiveIterator.OfInt; ignore japicmp source incompatibility -->


is this comment still correct without the 11? or is it a java 11 specific comment?

Good question — I dug into it. It turned out not to be JDK-11-specific behavior at all: the exclude was a workaround for a false-positive source incompatibility that japicmp 0.23.1 reported on the synthetic/bridge methods of PrimitiveIterator.OfInt (added in #3388). japicmp was upgraded to 0.26.1 in #3592 (already on master), which handles those methods correctly. I verified that japicmp:cmp (with breakBuildOnSourceIncompatibleModifications=true) passes with the exclude removed, so I removed the exclude and the comment entirely rather than reword it.

Raise the minimum build/runtime requirement to Java 17 and clean up workarounds and tooling that are no longer needed. Java 17 baseline: - Set maven.compiler.release to 17 and add an enforcer rule that requires Java 17+ to build. - Remove the extraJavaTestArgs (--add-opens / -XX flags) that were needed to run the tests on newer JDKs from a Java 11 baseline. - Drop the per-module maven.compiler.release=17 override in parquet-encoding-vector; it is now the project-wide default. - Simplify the CleanUtil javadoc. - CI: build only on JDK 17 (drop the JDK 11 matrix entry) and stop exporting extraJavaTestArgs. - Update README to state the Java 17 requirement. Spotless upgrade (2.46.1 -> 3.8.0): - Spotless 2.46.1 calls com.sun.tools.javac.util.Log methods that were removed in JDK 25, causing NoSuchMethodError at format time. Spotless 3.8.0 is compatible with JDK 25. The switch/case comment indentation changes are produced by the new formatter version. Remove obsolete japicmp exclude for IndexIterator: - The exclude worked around a false-positive source incompatibility reported by japicmp 0.23.1 on the synthetic/bridge methods of PrimitiveIterator.OfInt (apache#3388). japicmp was upgraded to 0.26.1 (apache#3592), which handles those methods correctly, so the workaround is no longer needed. Verified that japicmp:cmp (with breakBuildOnSourceIncompatibleModifications=true) passes with the exclude removed. Note: the Hadoop 3.3.0 -> 3.4.3 bump and the related vectored-I/O ByteBuffer leak fix that previously lived here have been split out; the Hadoop upgrade is tracked separately and will come via apache#3559.

iemejia changed the title ~~Upgrade Hadoop 3.3.0 -> 3.4.3, Spotless 2.46.1 -> 3.5.1, fix vectored I/O buffer leak~~ Bump minimum Java version to 17 and remove pre-17 workarounds May 23, 2026

nastra reviewed Jul 1, 2026

View reviewed changes

iemejia force-pushed the java17-extras branch 3 times, most recently from 16ddc39 to 530f1ea Compare July 14, 2026 21:44

kevinjqliu reviewed Jul 17, 2026

View reviewed changes

iemejia force-pushed the java17-extras branch from 530f1ea to d975fb0 Compare July 17, 2026 11:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Bump minimum Java version to 17 and remove pre-17 workarounds#3579

Bump minimum Java version to 17 and remove pre-17 workarounds#3579
iemejia wants to merge 1 commit into
apache:masterfrom
iemejia:java17-extras

iemejia commented May 23, 2026 •

edited

Loading

Uh oh!

nastra Jul 1, 2026

Uh oh!

iemejia Jul 17, 2026

Uh oh!

nastra Jul 1, 2026

Uh oh!

iemejia Jul 17, 2026

Uh oh!

nastra Jul 1, 2026 •

edited

Loading

Uh oh!

iemejia Jul 17, 2026

Uh oh!

Fokko Jul 17, 2026

Uh oh!

kevinjqliu left a comment

Uh oh!

kevinjqliu Jul 17, 2026

Uh oh!

iemejia Jul 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	<spotless.version>3.5.1</spotless.version>
	<spotless.version>3.8.0</spotless.version>

Uh oh!

Conversation

iemejia commented May 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Java 17 baseline

Spotless upgrade (2.46.1 -> 3.8.0)

Remove obsolete japicmp exclude for IndexIterator

Split out (deferred to #3559)

Testing

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nastra Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kevinjqliu left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

iemejia commented May 23, 2026 •

edited

Loading

Remove obsolete japicmp exclude for `IndexIterator`

nastra Jul 1, 2026 •

edited

Loading