Skip to content

ggml-quants : weighted rounding algorithms with cumulative search #12557

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
compilade wants to merge 13 commits into
base: master
Choose a base branch
from
+804 −93
Draft

ggml-quants : weighted rounding algorithms with cumulative search #12557

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
dd6b840
ggml-quants : improve IQ4_NL, IQ4_XS, and Q3_K
compilade Feb 21, 2025
d0060fc
ggml-quants : better and faster make_qkxs_quants
compilade Feb 21, 2025
6f7fe74
ggml-quants : improve imatrix behavior for TQ1_0, TQ2_0, Q4_0, Q5_0
compilade Feb 21, 2025
f27c1af
ggml-quants : improve TQ2_0 imatrix
compilade Mar 7, 2025
0c9e442
ggml-quants : remove some commented code
compilade Mar 15, 2025
30ad9c2
ggml-quants : faster exhaustive IQ4_NL rounding with k_heap
compilade Mar 15, 2025
3be1151
ggml-quants : use a max-heap for linear quants like Q3_K
compilade Mar 20, 2025
f86b8ff
ggml-quants : use qkxh in more places
compilade Mar 21, 2025
3e4b675
ggml-quants : use a max-heap for TQ1_0 and TQ2_0 quantization
compilade Mar 22, 2025
af23abd
ggml-quants : remove slower qsort-based cumulative search
compilade Mar 22, 2025
a411397
Merge branch 'master' into compilade/optimal-rounding
compilade Mar 22, 2025
8b8b88f
ggml-quants : restore Q2_K use of make_qp_quants
compilade Mar 22, 2025
a5b1943
ggml-quants : fix some edge cases in make_qkxh_nl_quants
compilade Mar 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
ggml-quants : remove some commented code
  • Loading branch information
@compilade
compilade committed Mar 15, 2025
commit 0c9e44248996f3bcd08ca2c1acf0ef5548428f41
2 changes: 0 additions & 2 deletions ggml/src/ggml-quants.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5173,7 +5173,6 @@ static void quantize_row_iq4_nl_impl(const int super_block_size, const int block
for (int j = 0; j < block_size; ++j) weight[j] = qw[j] * sqrtf(sigma2 + xb[j]*xb[j]);
} else {
for (int j = 0; j < block_size; ++j) weight[j] = sqrtf(sigma2 + xb[j]*xb[j]);
// for (int j = 0; j < block_size; ++j) weight[j] = 1;
}
float amax = 0;
for (int j = 0; j < block_size; ++j) {
Expand Down Expand Up @@ -5258,7 +5257,6 @@ size_t quantize_iq4_nl(const float * restrict src, void * restrict dst, int64_t
return nrow * nblock * sizeof(block_iq4_nl);
}

//void quantize_row_iq4_nl_ref(const float * restrict x, void * restrict vy, int64_t k) {
void quantize_row_iq4_nl_ref(const float * restrict x, block_iq4_nl * restrict y, int64_t k) {
GGML_ASSERT(k%QK4_NL == 0);
int64_t nblock = k/QK4_NL;
Expand Down