Nick Terrell
68dfd14a8c
[linux] Opt out of row based match finder for the kernel
...
The row based match finder is slower without SIMD. We used to detect the
presence of SIMD to set the lower bound to 17, but that breaks
determinism. Instead, specifically opt into it for the kernel, because
it is one of the rare cases that doesn't have SIMD support.
2025-03-11 16:18:59 -04:00
Yann Collet
22b2fd2517
Merge pull request #4317 from hirohira9119/fix-function-signature
...
Fix function signature mismatch for ZSTD_convertBlockSequences
2025-02-27 13:03:03 -08:00
Yann Collet
db2d205ada
fixed -Wconversion for lib/decompress/zstd_decompress_block.c
2025-02-26 10:01:05 -08:00
hirohira
2840631dc1
Fix function signature mismatch for ZSTD_convertBlockSequences
2025-02-26 08:23:48 +09:00
Yann Collet
d2c562b803
update hrlog comment
2025-02-10 10:48:56 -08:00
Yann Collet
67fad95f79
derive hashratelog from hashlog when only hashlog is set
2025-02-10 10:46:37 -08:00
Yann Collet
09d7e34ed8
adjust mml
2025-02-10 10:46:37 -08:00
Yann Collet
d5e4698267
fix boundary condition
2025-02-10 10:46:37 -08:00
Yann Collet
72406b71c3
update hrlog rule to favor compression ratio a bit more at low levels
2025-02-10 10:46:37 -08:00
Yann Collet
f26cc54f37
dynamic bucket sizes
2025-02-10 10:46:37 -08:00
Yann Collet
4609a40b89
dynamically adjust hratelog and ldmml based on strategy
2025-02-10 10:46:37 -08:00
Yann Collet
23e5f80390
Revert "pass dictionary loading method as parameter"
...
This reverts commit 821fc567f93a415e9fbe856271ccd452ee7acf07.
2025-02-05 18:47:26 -08:00
Yann Collet
c7cd7dc04b
better MT fluidity
...
--patch-from no longer blocked on first job dictionary loading
2025-02-05 18:42:00 -08:00
Yann Collet
f11bd19c7f
ensure cdict is properly reset to NULL
2025-02-05 18:42:00 -08:00
Yann Collet
7406d2b6eb
skips the need to create a temporary cdict for --patch-from
...
thus saving a bit of memory and a little bit of cpu time
2025-02-05 18:42:00 -08:00
Yann Collet
220abe6da8
reduced memory usage
...
by avoiding to duplicate in memory
a dictionary that was passed by reference.
2025-02-05 18:42:00 -08:00
Yann Collet
85a44b233a
always free .cdictLocal
2025-02-05 18:41:59 -08:00
Yann Collet
e637fc64c5
update type naming convention
2025-02-05 18:41:59 -08:00
Yann Collet
34ba14437a
minor boundary change
...
improves compression ratio at low levels
2025-02-05 18:41:59 -08:00
Yann Collet
ffa66a6971
fix speed of --patch-from at high compression mode
2025-02-05 18:41:59 -08:00
Yann Collet
e117d79e22
fix minor alignment warning
2025-02-05 16:13:58 -08:00
Yann Collet
c39424ea87
fix minor alignment warning
...
this is a prototype definition error:
`_mm_storeu_si128()` should accept a `void*` pointer,
since it explicitly states that it accepts unaligned addresses
yet requiring a `__m128i*` tells otherwise, and requires the compiler the enforce this alignment.
2025-02-05 16:11:54 -08:00
Yann Collet
32dff04d32
fix one minor alignment warning
...
seems like a prototype interface error:
input parameter should have been `const void*`,
since the documentation is explicit that input doesn't have to be aligned,
but `const __m256i*` makes the compiler enforce it.
2025-02-05 15:46:44 -08:00
Yann Collet
f0b5f65bca
fixed minor static function declaration issue
...
in AVX2 mode only
2025-01-18 22:49:16 -08:00
Yann Collet
19025f3da0
Merge pull request #4238 from szsam/patch-1
...
fix out-of-bounds array index access
2025-01-15 17:56:41 -08:00
Yann Collet
87f0a4fbe0
restore full equation
...
do not solve the equation, even though some members cancel each other,
this is done for clarity,
we'll let the compiler do the resolution at compile time.
2025-01-15 17:11:27 -08:00
Yann Collet
8bff69af86
Alignment instruction ZSTD_ALIGNED() in common/compiler.h
2025-01-15 17:11:27 -08:00
Yann Collet
2f3ee8b530
changed code compilation test to employ ZSTD_ARCH_X86_AVX2
2025-01-15 17:11:27 -08:00
Yann Collet
debe3d20d9
removed unused branch
2025-01-15 17:11:27 -08:00
Yann Collet
e3181cfd32
minor code doc update
2025-01-15 17:11:27 -08:00
Yann Collet
aa2cdf964f
added compilation-time checks to ensure AVX2 code is valid
...
since it depends on a specific definition of ZSTD_Sequence structure.
2025-01-15 17:11:27 -08:00
Yann Collet
57a4554192
removed unused variable
2025-01-15 17:11:27 -08:00
Yann Collet
4aaf9cefe9
fix minor conversion warning
2025-01-15 17:11:27 -08:00
Yann Collet
db3d48823a
no need for specialized variant
...
the branch is not in the hot loop
2025-01-15 17:11:27 -08:00
Yann Collet
cd53924eff
removed erroneous #includes
...
that were automatically added by the editor without notification
2025-01-15 17:11:27 -08:00
Yann Collet
ed0a8b8be1
AVX2 version of ZSTD_get1BlockSummary()
2025-01-15 17:11:27 -08:00
Yann Collet
b6a4d5a8ba
minor +10% speed improvement for scalar ZSTD_get1BlockSummary()
2025-01-15 17:11:27 -08:00
Yann Collet
8eb2587432
added benchmark for get1BlockSummary()
2025-01-15 17:11:27 -08:00
Yann Collet
8d62164589
control long length within AVX2 implementation
2025-01-15 17:11:27 -08:00
Yann Collet
d1f0e5fb97
fullbench can run a verification function
...
compressSequencesAndLiterals: fixed long lengths in scalar mode
2025-01-15 17:11:27 -08:00
Yann Collet
886720442f
initial implementation (incomplete)
...
needs to take care of long lengths > 65535
2025-01-15 17:11:27 -08:00
Mingjie Shen
afff3d2cce
return error if block delimiter is not found
2025-01-13 20:52:06 -05:00
Mingjie Shen
e490be895c
fix out-of-bounds array index access
2025-01-13 16:39:34 -05:00
Victor Zhang
a610550e2c
Merge pull request #4218 from facebook/externC
...
Move #includes out of `extern "C"` blocks
2025-01-07 10:06:08 -08:00
Victor Zhang
d88651e604
Do not vary row matchfinder selection based on availability of SSE2/Neon
...
Move towards a stronger guarantee of reproducibility by removing this small difference for machines without SSE2/Neon.
The SIMD behavior is now the default for all platforms.
2025-01-03 09:35:18 -08:00
Nick Terrell
1548bfc349
[opt] Fix too short of match getting generated
...
The optimal parser with LDM enabled using minMatch > 3 could generate a match
length of 3 when minMatch >= 4. This is not allowed.
1. Fix the bug
2. Add validation logic to `ZSTD_buildSeqStore()` in debug mode for all block
compressors that checks we never generate too short a match. This way we don't
rely on the `generate_sequences` fuzzer to find this issue.
Credit to OSS-Fuzz
2025-01-03 11:38:41 -05:00
Yann Collet
47cbfc87a9
restore invocation of ZSTD_entropyCompressSeqStore()
...
in the ZSTD_compressSequences() pipeline
2024-12-20 10:37:01 -08:00
Yann Collet
522adc34eb
minor: use MEM_writeLE24()
...
so that an empty frame needs only 3 bytes of dstCapacity.
2024-12-20 10:37:01 -08:00
Yann Collet
b7a9e69d8d
added parameter litCapacity
...
to ZSTD_compressSequencesAndLiterals()
to enforce the litCapacity >= litSize+8 condition.
2024-12-20 10:37:01 -08:00
Yann Collet
76445bb379
add a check, to return an error if Sequence validation is enabled
...
since ZSTD_compressSequencesAndLiterals() doesn't support it.
2024-12-20 10:37:01 -08:00
