Fix some bugs in reshard bulk request splitting #139394
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jxie-1
added
>non-issue
:Distributed Indexing/Distributed
Collaborator
|
Pinging @elastic/es-distributed-indexing (Team:Distributed Indexing) |
lkts
approved these changes
Dec 11, 2025
| } | ||
| }); | ||
| // Item responses should match the order of the original item requests | ||
| BulkItemResponse[] bulkItemResponses = Arrays.stream(originalRequest.items()) |
Contributor
There was a problem hiding this comment.
Streams allocate and given that we don't win much in readability here (IMO of course) i would go a for loop.
| * @param <E> the type of anticipated exception | ||
| */ | ||
| @FunctionalInterface | ||
| public interface CheckedTriConsumer<S, T, U, E extends Exception> { |
Contributor
There was a problem hiding this comment.
I don't know if i like this. IMO when we are at this number of parameters we could benefit from names and introduce a specific interface where it is needed (so in this case SplitRequestExecutor or something). It can still be a @FunctionalInterface.
jxie-1
added a commit
to jxie-1/elasticsearch
that referenced
this pull request
Dec 12, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR addresses two issues:
When combining bulk shard responses, item ids were used the index to an array. This only works when there's one shard, as a
BulkOperationstarts with sequential item ids [0, 1, 2...] but gets split into multipleBulkShardRequests.For instance, in a scenario with 2 shards, a
BulkOperationwith 10 item requests could get split into twoBulkShardRequests with item ids [0, 1, 3, 7, 9] and [2, 4, 5, 6, 8] respectively. After splitting the firstBulkShardRequestfurther for resharding and receiving responses, the current implementation to combine the responses would create an array of length 5 and run into an index out of bounds exception when indexing using item id 7.The proposed fix is to combine the bulk responses into a map of item id -> item response. Then we can use the original request, mapping each item request to its corresponding item response to create an item response list.
The second issue is that the source shard executes the original bulk shard request instead of the split one. I added a parameter to
doPrimaryRequestfor the request to execute.