Skip to content

Conversation

@steveloughran
Copy link
Contributor

Rationale for this change

As discussed on the mailing list; hadoop 3.3.0 is the minimum
support hadoop version for 1.15

What changes are included in this PR?

pom changes

  • remove the hadoop-2 profile
  • set base hadoop 3 version to 3.3.0

Are these changes tested?

locally, yes.

Are there any user-facing changes?

The changed minimum hadoop version needs to go into the release notes

Closes #2943

@Fokko
Copy link
Contributor

Fokko commented Nov 15, 2024

@steveloughran I think we want to include the Hadoop 2 Github Actions as well

@steveloughran
Copy link
Contributor Author

I think we want to include the Hadoop 2 Github Actions as well

I wanted to see what what happened there. Let me work on that.

@steveloughran steveloughran marked this pull request as draft November 15, 2024 11:48
@steveloughran
Copy link
Contributor Author

Looking at this I note that thift is download from apache archives and built every time. Apache infra might be unhappy about this; archive isn't replicated the way others are and they have "expressed concerns" about it being used elsewhere.

If he CI target OS/cpu is the same across builds, what about just sticking a precompiled thrift binary into dev/thift; if the cpu varies then it could be split for x86-64 and arm with the path set up appropriately.

@steveloughran steveloughran marked this pull request as ready for review November 19, 2024 17:56
@steveloughran
Copy link
Contributor Author

hadoop 3 builds are all happy; no hadoop 2 build any more

@steveloughran
Copy link
Contributor Author

no, vector failed. draft again

@steveloughran steveloughran marked this pull request as draft November 19, 2024 18:58
- remove the hadoop-2 profile
- set base hadoop 3 version to 3.3.0
- remove hadoop 2 github action profile
@steveloughran steveloughran force-pushed the GH-2943-remove-hadoop-2 branch from 8d22343 to 9d0454c Compare November 20, 2024 15:57
@steveloughran steveloughran marked this pull request as ready for review November 21, 2024 17:30
@steveloughran
Copy link
Contributor Author

@Fokko thanks; merge when ready

@Fokko Fokko merged commit 586a354 into apache:master Nov 26, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove support for Hadoop <3.3

2 participants