sharpetronics/zstd
1
0
Fork 0
mirror of https://github.com/facebook/zstd.git synced 2025-10-06 00:04:13 -04:00
Code Issues Projects Releases Wiki Activity
10,906 Commits 38 Branches 77 Tags
Commit Graph

2001 Commits

Author SHA1 Message Date
Yann Collet
ab0f1798e8 ensure that srcSize is controlled 2024-12-20 10:37:00 -08:00
Yann Collet
a80f55f47d added a test for ZSTD_compressSequencesAndLiterals 
checks that srcSize is present in the frame header
and bounds the window size.
 2024-12-20 10:37:00 -08:00
Yann Collet
0a54f6f288 ZSTD_compressSequencesAndLiterals requires srcSize as parameter 
this makes it possible to adjust windowSize to its tightest.
 2024-12-20 10:37:00 -08:00
Yann Collet
ad023b392f fixed minor error in one benchmark scenario 2024-12-20 10:37:00 -08:00
Yann Collet
12c47d3262 improved speed of the Sequences converter 2024-12-20 10:37:00 -08:00
Yann Collet
95ad9e47ff added benchmark for ZSTD_convertBlockSequences_wBlockDelim() 2024-12-20 10:37:00 -08:00
Yann Collet
61ac8311e0 attempt to silence Visual Studio warning about fopen() 2024-12-20 10:37:00 -08:00
Yann Collet
f617e86b71 fixed incorrect assert 2024-12-20 10:37:00 -08:00
Yann Collet
f0d0d95234 added tests 
check that ZSTD_compressAndLiterals() also controls that the `srcSize` field is exact.
 2024-12-20 10:37:00 -08:00
Yann Collet
31b5ef2539 ZSTD_compressSequencesAndLiterals() now supports multi-blocks frames. 2024-12-20 10:36:59 -08:00
Yann Collet
5164d44dab change advanced parameter name: ZSTD_c_repcodeResolution 
and updated its documentation.
Note: older name ZSTD_c_searchForExternalRepcodes remains supported via #define
 2024-12-20 10:36:59 -08:00
Yann Collet
f176514467 minor doc update 2024-12-20 10:36:59 -08:00
Yann Collet
f281497aef fullbench: new scenario: compressSequencesAndLiterals() 2024-12-20 10:36:59 -08:00
Yann Collet
ac05ea89a5 fullbench: switch default generator to lorem ipsum 
which creates more "realistic" scenarios than former compressible noise.

The legacy data generator remains accessible,
it is triggered when requesting an explicit compressibility factor (-P#).
 2024-12-20 10:36:59 -08:00
Yann Collet
8ab04097ed add the compressSequences() benchmark scenario 2024-12-20 10:36:59 -08:00
Yann Collet
c050ae4fb8 fullbench: preparation functions are now in charge of allocating buffers 2024-12-20 10:36:59 -08:00
Yann Collet
8b7e1b795d minor variable renaming 2024-12-20 10:36:59 -08:00
Yann Collet
c540976a4b minor: test reordering 2024-12-20 10:36:59 -08:00
Yann Collet
4c097b4939 refactor fullbench 
to make it easier to add new scenarios
 2024-12-20 10:36:58 -08:00
Yann Collet
09964c6276 prepare fullbench for a refactor 2024-12-20 10:36:58 -08:00
Yann Collet
0b013b2688 added unit tests to ZSTD_compressSequencesAndLiterals() 
seems to work as expected,
correctly control that `litSize` and `srcSize` are exactly correct.
 2024-12-20 10:36:58 -08:00
Yann Collet
c97522f7fb codemod: ZSTD_sequenceFormat_e -> ZSTD_SequenceFormat_e 
since it's a type name.

Note: in contrast with previous names, this one is on the Public API side.
So there is a #define, so that existing programs using ZSTD_sequenceFormat_e still work.
 2024-12-20 10:36:56 -08:00
Yann Collet
477a01067f codemod: symbolEncodingType_e -> SymbolEncodingType_e 2024-12-20 10:36:56 -08:00
Yann Collet
a2245721ca codemod: seqStore_t -> SeqStore_t 
same idea, SeqStore_t is a type name, it should start with a Capital letter.
 2024-12-20 10:36:55 -08:00
Yann Collet
9671813375 codemod: seqDef -> SeqDef 
SeqDef is a type name, so it should start with a Capital letter.
It's an internal symbol, no impact on public API.
 2024-12-20 10:36:55 -08:00
Diego Nieto Cid
4a4786bef0 tests: fix non-regular file test on GNU/Hurd 
Since commit b21b03ca6 [1] the behaviour of writes to /dev/zero has been
fixed and now the non-regular file removal test no longer needs to be
done on /dev/random (which no longer works as random is not writable now).

[1] https://git.savannah.gnu.org/cgit/hurd/hurd.git/commit/?id=b21b03ca624b89caeedfe58430cea4b40317d39f
 2024-12-10 10:30:52 -03:00
Yann Collet
bbaba45589 change experimental parameter name 
from ZSTD_c_useBlockSplitter to ZSTD_c_splitAfterSequences.
 2024-10-31 13:43:40 -07:00
Yann Collet
4f93206d62 changed variable name to ZSTD_c_blockSplitterLevel 
suggested by @terrelln
 2024-10-29 11:12:09 -07:00
Yann Collet
f593ccda04 removed trace left over 2024-10-28 16:57:01 -07:00
Yann Collet
37706a677c added a test 
test both that the new parameter works as intended,
and that the over-split protection works as intended
 2024-10-28 16:31:15 -07:00
Yann Collet
4e1a87976f update regression results 2024-10-25 16:33:00 -07:00
Yann Collet
c63b5d2294 update regression results 2024-10-24 14:47:55 -07:00
Yann Collet
ea082692aa added block splitter variant for greedy & lazy (levels 5 to 7) 2024-10-24 13:27:01 -07:00
Yann Collet
70c77d20d6 update regression results 
first block is no longer splitted since adding the @savings over-split protection
 2024-10-24 11:43:16 -07:00
Yann Collet
bbda1acf85 update regression results 2024-10-23 15:56:56 -07:00
Yann Collet
d2eeed53dc updated compression results 
due to integration of `sample5` strategy, leading to better compression ratios on a range of levels
 2024-10-23 11:50:57 -07:00
Yann Collet
dac26eaeac updated regression test results 2024-10-23 11:50:57 -07:00
Yann Collet
433f4598ad fixed minor conversion warnings on Visual 2024-10-23 11:50:56 -07:00
Yann Collet
80a912dec1 fixed zstreamtest 2024-10-23 11:50:56 -07:00
Yann Collet
f83ed087f6 fixed RLE detection test 2024-10-23 11:50:56 -07:00
Yann Collet
41d870fbbf updated regression tests results 2024-10-17 11:06:26 -07:00
Yann Collet
61d08b0e42 fix test 
a margin of 4 is insufficient to guarantee compression success.
 2024-10-17 09:37:23 -07:00
Yann Collet
730d2dce41 fix test 2024-10-15 18:44:40 -07:00
Yann Collet
de6cc98e07 fix incorrect pointer manipulation 
fix #4155
 2024-10-01 09:25:26 -07:00
Pawel Czarnecki
1f5df587fa tests/decodecorpus: add more advanced options 
- add option to force a specific block type
- add option to force a specific literal tyle
- add option to generate only frame headers
- add option to skip generating magic numbers

Co-authored-by: Maciej Dudek <mdudek@antmicro.com>
Co-authored-by: Pawel Czarnecki <pczarnecki@antmicro.com>
Co-authored-by: Robert Winkler <rwinkler@antmicro.com>
Co-authored-by: Roman Dobrodii <rdobrodii@antmicro.com>
 2024-07-26 09:47:16 +02:00
Yann Collet
3de0541aef
Merge pull request #4079 from elasota/truncated-huff-state-error 
Throw error if Huffman weight initial states are truncated
 2024-06-30 16:17:03 -07:00
elasota
0938308ff6 Throw error if Huffman weight initial states are truncated 2024-06-20 17:46:16 -04:00
Dimitri Papadopoulos
2d736d9c50
Fix new typos found by codespell 2024-06-20 20:12:16 +02:00
Quentin Boswank
f19c98228f Fix $filter and Msys/Cygwin 
- switched the patter and input of $filter into the right places
- added pattern wildcard to MSYS_NT & CYGWIN_NT as they change with windows versions
- correctly identify MSYS2, even in an env like MINGW64
 2024-06-05 18:37:27 +02:00
Elliot Gorokhovsky
be6a182006
Unit test for external sequence producer + static CCtx + streaming (#4063) 2024-06-03 12:42:27 -04:00