Skip to content

Commit e66b83b

Browse files
authored
Docs cleanup (#480)
Authors: - Mads R. B. Kristensen (https://github.com/madsbk) - Lawrence Mitchell (https://github.com/wence-) Approvers: - Lawrence Mitchell (https://github.com/wence-) URL: #480
1 parent 03fbb45 commit e66b83b

File tree

2 files changed

+92
-2
lines changed

2 files changed

+92
-2
lines changed

README.md

Lines changed: 85 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
## Summary
44

5-
KvikIO is a Python and C++ library for high performance file IO. It provides C++ and Python
5+
KvikIO (pronounced "kuh-VICK-eye-oh", see [here](https://ordnet.dk/ddo_en/dict?query=kvik) for pronunciation of kvik) is a Python and C++ library for high performance file IO. It provides C++ and Python
66
bindings to [cuFile](https://docs.nvidia.com/gpudirect-storage/api-reference-guide/index.html),
77
which enables [GPUDirect Storage (GDS)](https://developer.nvidia.com/blog/gpudirect-storage/).
88
KvikIO also works efficiently when GDS isn't available and can read/write both host and device data seamlessly.
@@ -15,9 +15,92 @@ The C++ library is header-only making it easy to include in [existing projects](
1515
* A Python [Zarr](https://zarr.readthedocs.io/en/stable/) backend for reading and writing GPU data to file seamlessly.
1616
* Concurrent reads and writes using an internal thread pool.
1717
* Non-blocking API.
18-
* Handle both host and device IO seamlessly.
18+
* Transparently handles reads and writes to/from memory on both host and device.
1919
* Provides Python bindings to [nvCOMP](https://github.com/NVIDIA/nvcomp).
2020

21+
2122
### Documentation
2223
* Python: <https://docs.rapids.ai/api/kvikio/nightly/>
2324
* C++: <https://docs.rapids.ai/api/libkvikio/nightly/>
25+
26+
27+
### Examples
28+
29+
#### Python
30+
```python
31+
import cupy
32+
import kvikio
33+
34+
def main(path):
35+
a = cupy.arange(100)
36+
f = kvikio.CuFile(path, "w")
37+
# Write whole array to file
38+
f.write(a)
39+
f.close()
40+
41+
b = cupy.empty_like(a)
42+
f = kvikio.CuFile(path, "r")
43+
# Read whole array from file
44+
f.read(b)
45+
assert all(a == b)
46+
f.close()
47+
48+
# Use contexmanager
49+
c = cupy.empty_like(a)
50+
with kvikio.CuFile(path, "r") as f:
51+
f.read(c)
52+
assert all(a == c)
53+
54+
# Non-blocking read
55+
d = cupy.empty_like(a)
56+
with kvikio.CuFile(path, "r") as f:
57+
future1 = f.pread(d[:50])
58+
future2 = f.pread(d[50:], file_offset=d[:50].nbytes)
59+
# Note: must wait for futures before exiting block
60+
# at which point the file is closed.
61+
future1.get() # Wait for first read
62+
future2.get() # Wait for second read
63+
assert all(a == d)
64+
65+
66+
if __name__ == "__main__":
67+
main("/tmp/kvikio-hello-world-file")
68+
```
69+
70+
#### C++
71+
```c++
72+
#include <cstddef>
73+
#include <cuda_runtime.h>
74+
#include <kvikio/file_handle.hpp>
75+
using namespace std;
76+
77+
int main()
78+
{
79+
// Create two arrays `a` and `b`
80+
constexpr std::size_t size = 100;
81+
void *a = nullptr;
82+
void *b = nullptr;
83+
cudaMalloc(&a, size);
84+
cudaMalloc(&b, size);
85+
86+
// Write `a` to file
87+
kvikio::FileHandle fw("test-file", "w");
88+
size_t written = fw.write(a, size);
89+
fw.close();
90+
91+
// Read file into `b`
92+
kvikio::FileHandle fr("test-file", "r");
93+
size_t read = fr.read(b, size);
94+
fr.close();
95+
96+
// Read file into `b` in parallel using 16 threads
97+
kvikio::default_thread_pool::reset(16);
98+
{
99+
// FileHandles have RAII semantics
100+
kvikio::FileHandle f("test-file", "r");
101+
future<size_t> future = f.pread(b_dev, sizeof(a), 0); // Non-blocking
102+
size_t read = future.get(); // Blocking
103+
// Notice, `f` closes automatically on destruction.
104+
}
105+
}
106+
```

docs/source/conf.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,13 @@
8181
# The name of the Pygments (syntax highlighting) style to use.
8282
pygments_style = None
8383

84+
autodoc_default_options = {
85+
'members': True,
86+
'member-order': 'bysource',
87+
'special-members': '__init__',
88+
'undoc-members': True,
89+
'exclude-members': '__weakref__'
90+
}
8491

8592
# -- Options for HTML output -------------------------------------------------
8693

0 commit comments

Comments
 (0)