C++ bindings to the Batch API #220

madsbk · 2023-05-16T12:59:45Z

An example use of the Batch API (requires CUDA v12.1+):

    // Here we use the batch API to read "/tmp/test-file" into `b_dev` by
    // submitting 4 batch operations.
    constexpr int num_ops_in_batch = 4;
    constexpr int batchsize        = SIZE / num_ops_in_batch;
    kvikio::DriverProperties props;
    check(num_ops_in_batch < props.get_max_batch_io_size());

    // We open the file as usual.
    kvikio::FileHandle f("/tmp/test-file", "r");

    // Then we create a batch
    auto batch = kvikio::BatchHandle(num_ops_in_batch);

    // And submit 4 operations each with its own offset
    std::vector<kvikio::BatchOp> ops;
    for (int i = 0; i < num_ops_in_batch; ++i) {
      ops.push_back(kvikio::BatchOp{.file_handle   = f,
                                    .devPtr_base   = b_dev,
                                    .file_offset   = i * batchsize,
                                    .devPtr_offset = i * batchsize,
                                    .size          = batchsize,
                                    .opcode        = CUFILE_READ});
    }
    batch.submit(ops);

    // Finally, we wait on all 4 operations to be finished and check the result
    auto statuses = batch.status(num_ops_in_batch, num_ops_in_batch);
    check(statuses.size() == num_ops_in_batch);
    size_t total_read = 0;
    for (auto status : statuses) {
      check(status.status == CUFILE_COMPLETE);
      check(status.ret == batchsize);
      total_read += status.ret;
    }
    check(cudaMemcpy(b, b_dev, SIZE, cudaMemcpyDeviceToHost) == cudaSuccess);
    for (int i = 0; i < NELEM; ++i) {
      check(a[i] == b[i]);
    }
    cout << "Batch read using 4 operations: " << total_read << endl;

KiranModukuri · 2023-05-17T16:49:37Z

An example use of the Batch API (requires CUDA v12.1+):

  
    // Here we use the batch API to read "/tmp/test-file" into `b_dev` by
    // submitting 4 batch operations.
    constexpr int num_of_batches = 4;

may be it will be clear to say this is "num_ops_in_batch".

…o cufile_batch

madsbk · 2023-05-22T06:40:46Z

An example use of the Batch API (requires CUDA v12.1+):
  
    // Here we use the batch API to read "/tmp/test-file" into `b_dev` by
    // submitting 4 batch operations.
    constexpr int num_of_batches = 4;
may be it will be clear to say this is "num_ops_in_batch".

Good point, fixed

wence-

A few places where we can do error checking I think.

Also, I don't like the name-mangled lookup, is there not a better way?!

wence- · 2023-05-30T16:10:01Z

cpp/CMakeLists.txt

+  file(READ "${cuFile_INCLUDE_DIRS}/cufile.h" CUFILE_H_STR)
+  string(FIND "${CUFILE_H_STR}" "cuFileBatchIOSetUp" cuFileBatchIOSetUp_location)
+  if(cuFileBatchIOSetUp_location EQUAL "-1")
+    set(cuFile_BATCH_API_FOUND FALSE)
+  else()
+    set(cuFile_BATCH_API_FOUND TRUE)
+  endif()
+  message(STATUS "Found cuFile's Batch API: ${cuFile_BATCH_API_FOUND}")


The right thing to do is to attempt to compile and link a call to this function, surely this is something that cmake supports? To ensure that both the header file and the library actually offer the function.

This is fine, I guess, but there must be a canonical way.

I tried using check_symbol_exists() with no luck. Got irritated and just rolled my own :)

This doesn't work for some reason:

include(CheckSymbolExists) check_symbol_exists( cuFileBatchIOSetUp "${cuFile_INCLUDE_DIRS}/cufile.h" cuFile_BATCH_API_FOUND )

check_symbol_exists will try to compile and link a small executable that verifies that symbol exists. For this to work properly for libcufile you would also need to tell check_symbol_exists all the extra include directories, and link requirements.

The examples in the docs are easy since they are testing for values found in headers provided by the compiler/runtime and therefore no extra information is required.

Using CMake's try_compile would be easier in comparison but would still require setting up a code snippet and specifying the include directory for the libcufile and cuda rutime headers

wence- · 2023-05-30T16:16:58Z

cpp/include/kvikio/batch.hpp

+    CUFILE_TRY(
+      cuFileAPI::instance().BatchIOSubmit(_handle, io_batch_params.size(), &io_batch_params[0], 0));
+  }


This will produce a generic error if io_batch_params.size() is larger than the requested max batch size. Does it make more sense to check if operations.size() > _max_num_events and provide a useful message?

Good point, done.

cpp/include/kvikio/batch.hpp

wence- · 2023-05-30T16:24:52Z

cpp/include/kvikio/shim/cufile.hpp

+    // both the batch and async API.
+    try {
+      void* s{};
+      get_symbol(s, lib, "_ZTS23cuFileBatchContextState");


Can we check the CTK version somehow? This is unstable wrt compilers (since name mangling is not standardised).

No, not AFAIK :/
I think this will suffice for now. When CUDA v12.2 is released, I think we should make the Batch API require v12.2 and then use cuFileReadAsync to determine the availability of both the batch and async API.

wence- · 2023-05-30T16:29:02Z

cpp/include/kvikio/shim/cufile_h_wrapper.hpp

+typedef enum CUfileOpcode { CUFILE_READ = 0, CUFILE_WRITE } CUfileOpcode_t;
+
+typedef enum CUFILEStatus_enum {
+  CUFILE_WAITING  = 0x000001,  /* required value prior to submission */
+  CUFILE_PENDING  = 0x000002,  /* once enqueued */
+  CUFILE_INVALID  = 0x000004,  /* request was ill-formed or could not be enqueued */
+  CUFILE_CANCELED = 0x000008,  /* request successfully canceled */
+  CUFILE_COMPLETE = 0x0000010, /* request successfully completed */
+  CUFILE_TIMEOUT  = 0x0000020, /* request timed out */
+  CUFILE_FAILED   = 0x0000040  /* unable to complete */
+} CUfileStatus_t;
+
+typedef struct CUfileIOEvents {
+  void* cookie;
+  CUfileStatus_t status; /* status of the operation */
+  size_t ret;            /* -ve error or amount of I/O done. */
+} CUfileIOEvents_t;
+


These are just copied from cufile.h I guess. Do you endeavour to be ABI compatible with the cufile definitions or is this just "anything, as long as it has the appopriate fields/names"?

It is "anything, as long as it has the appopriate fields/names". Added a comment.

Co-authored-by: Lawrence Mitchell <[email protected]>

madsbk · 2023-06-01T08:04:22Z

Thanks @wence-, I think I have addressed all of your issues?

wence-

Thanks @madsbk

madsbk · 2023-06-01T10:31:11Z

Awesome, thanks for the review @wence-. Just in time for the code freeze :)

madsbk · 2023-06-01T10:32:05Z

/merge

madsbk added improvement Improves an existing functionality non-breaking Introduces a non-breaking change labels May 16, 2023

madsbk force-pushed the cufile_batch branch from 3d49f43 to af33d94 Compare May 16, 2023 13:00

cmake: define CUFILE_BATCH_API_FOUND

4d44f16

madsbk force-pushed the cufile_batch branch 4 times, most recently from 0aea697 to 0026f80 Compare May 17, 2023 07:38

impl. of BatchHandle

49369ac

madsbk force-pushed the cufile_batch branch from 0026f80 to 49369ac Compare May 17, 2023 08:05

use cuFileBatchContextState to determine if the batch API is available

67a386b

madsbk added 2 commits May 22, 2023 08:37

rename num_of_batches => num_ops_in_batch

1ea346d

Merge branch 'branch-23.06' of https://github.com/rapidsai/kvikio int…

24073f8

…o cufile_batch

madsbk added 2 commits May 22, 2023 10:36

Impl. batch cancel

1b1fe50

doc

7d9d4f4

madsbk marked this pull request as ready for review May 22, 2023 15:18

madsbk self-assigned this May 24, 2023

Merge branch 'branch-23.06' into cufile_batch

16fd312

wence- reviewed May 30, 2023

View reviewed changes

madsbk and others added 4 commits June 1, 2023 08:11

clean up

e762b10

Co-authored-by: Lawrence Mitchell <[email protected]>

is_batch_available: use CUFILE_BATCH_API_FOUND

94f208b

check CUFILE_BATCH_API_FOUND before defining CUfileOpcode and Co.

72164ff

submit: check max_num_events

ac3955d

madsbk requested a review from wence- June 1, 2023 08:04

wence- approved these changes Jun 1, 2023

View reviewed changes

rapids-bot bot merged commit b28298f into rapidsai:branch-23.06 Jun 1, 2023

madsbk deleted the cufile_batch branch June 1, 2023 10:36

jakirkham mentioned this pull request Jun 27, 2023

Tracking support of new cuFile features #204

Open

4 tasks

jakirkham mentioned this pull request May 1, 2024

Adding multiline/IOV APIs #77

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

C++ bindings to the Batch API #220

C++ bindings to the Batch API #220

madsbk commented May 16, 2023 •

edited

Loading

KiranModukuri commented May 17, 2023

madsbk commented May 22, 2023

wence- left a comment

wence- May 30, 2023

madsbk Jun 1, 2023 •

edited

Loading

robertmaynard Jun 8, 2023

wence- May 30, 2023

madsbk Jun 1, 2023

wence- May 30, 2023

madsbk Jun 1, 2023

wence- May 30, 2023

madsbk Jun 1, 2023

madsbk commented Jun 1, 2023

wence- left a comment

madsbk commented Jun 1, 2023

madsbk commented Jun 1, 2023

C++ bindings to the Batch API #220

C++ bindings to the Batch API #220

Conversation

madsbk commented May 16, 2023 • edited Loading

KiranModukuri commented May 17, 2023

madsbk commented May 22, 2023

wence- left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

madsbk Jun 1, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

madsbk commented Jun 1, 2023

wence- left a comment

Choose a reason for hiding this comment

madsbk commented Jun 1, 2023

madsbk commented Jun 1, 2023

madsbk commented May 16, 2023 •

edited

Loading

madsbk Jun 1, 2023 •

edited

Loading