sharpetronics/zstd
1
0
Fork 0
mirror of https://github.com/facebook/zstd.git synced 2025-11-04 00:02:59 -05:00
Code Issues Projects Releases Wiki Activity
9,853 Commits 37 Branches 77 Tags
Commit Graph

1782 Commits

Author SHA1 Message Date
senhuang42
23554ff25f Force CCtx minmatch to be same as generated minmatch 2020-11-23 13:29:20 -05:00
senhuang42
c502cd33e5 Fix generating 1 too few characters in random string generator 2020-11-20 16:58:25 -05:00
senhuang42
5b0c8f0a7c Add appropriate bound to matchlengths, and reduce srcSize max 2020-11-20 16:58:25 -05:00
senhuang42
a73a07b189 Add a bound for matchlength dependent on window size 2020-11-20 16:58:25 -05:00
senhuang42
5c68c5e31e Variety of minor fixups, reduce allocation, make deterministic 2020-11-20 16:58:25 -05:00
senhuang42
59c021f501 Add built binary to .gitignore 2020-11-20 16:58:25 -05:00
senhuang42
26bc0bfdf6 Add new fuzzer to build targets 2020-11-20 16:58:25 -05:00
senhuang42
ed575963c5 Implement new fuzzer for sequence compression 2020-11-20 16:58:25 -05:00
senhuang42
7742f076b4 Add experimental param for sequence validation 2020-11-20 11:57:41 -05:00
senhuang42
05c0229668 Clean up visual conversion warnings 2020-11-18 15:36:29 -05:00
senhuang42
d6d7ba2a1f Modification to offset validation to include entire sequence 2020-11-17 10:13:22 -05:00
senhuang42
55b90ef010 Fix unit tests to agree with new changes 2020-11-16 11:36:37 -05:00
senhuang42
3d26615c84 Adjust unit tests to agree with new sequence generation API 2020-11-16 10:49:17 -05:00
senhuang42
2db8441245 Add RLE support 2020-11-16 10:49:17 -05:00
senhuang42
2bbdddf24e Add test case to roundtrip using ZSTD_getSequences() and ZSTD_compressSequences() 2020-11-16 10:49:16 -05:00
senhuang42
9d936d61d2 Reduce number of memcpy() calls 2020-11-13 19:43:30 -05:00
senhuang42
1a8af0de73 Improve unit test 2020-11-12 11:09:09 -05:00
sen
f62edf0fe9
Merge pull request #2381 from senhuang42/expand_sequence_extraction_api 
Add enum to define ZSTD_Sequence type and update sequence extraction API
 2020-11-06 13:00:31 -05:00
senhuang42
7d1dea070c Update unit tests 2020-11-06 11:10:37 -05:00
senhuang42
51abd58208 Rename getSequences() to generateSequences() 2020-11-06 10:53:22 -05:00
Luke Pitt
eac309c71b Add ZSTD_getDictID_fromCDict function to experimental section 2020-11-04 11:37:37 +00:00
senhuang42
c54a25b666 Revert compressibility change 2020-11-02 11:38:58 -05:00
senhuang42
d4d0346b40 Update name of enum, clarify documentation 2020-11-02 11:38:17 -05:00
senhuang42
9102f30dbf Update unit test 2020-11-02 11:30:31 -05:00
senhuang42
3327932609 Update ZSTD_getSequences function signature 2020-11-02 10:17:59 -05:00
Nick Terrell
37d546c445
Merge pull request #2379 from terrelln/regression-test 
[regression] Updates results.csv & add README
 2020-10-30 15:09:38 -07:00
Nick Terrell
7205e609a9
Merge pull request #2354 from terrelln/stable-buffer 
Add ZSTD_c_stable{In,Out}Buffer and optimize when set
 2020-10-30 15:06:56 -07:00
Nick Terrell
a446fa33dc [regression] Add README explaining the test 2020-10-30 13:55:52 -07:00
Nick Terrell
222916a5d3 [regression] Update results.csv 
https://github.com/facebook/zstd/pull/2339 removes the single-pass zstdmt API.
This changes the compressed size, because we no longer take the # of threads into
account when deciding the job size.
 2020-10-30 13:54:30 -07:00
sen
c37c714ef1
Merge pull request #2376 from senhuang42/clarify_sequence_extraction_api 
Refine external ZSTD_Sequence API
 2020-10-30 15:47:25 -04:00
Nick Terrell
2ebf6d5588 [test] Add unit tests for ZSTD_c_stable{In,Out}Buffer 2020-10-30 10:55:34 -07:00
sen
ff93440fc6
Merge pull request #2375 from senhuang42/ldm_oss_fuzz_testcase 
Add a test case for LDM + opt parser with small uncompressible block
 2020-10-29 09:32:05 -04:00
senhuang42
7198ebb213 Un-mix declarations and code 2020-10-28 18:51:03 -04:00
senhuang42
60a52c29e6 Add check for allocation 2020-10-28 16:22:22 -04:00
Nick Terrell
599ff58e08
Merge pull request #2339 from terrelln/zstdmt-stability 
Fix zstdmt stability issues and clean up the zstdmt code
 2020-10-27 19:43:13 -07:00
senhuang42
169fc07aa1 Move test to appropriate location 2020-10-27 16:59:43 -04:00
senhuang42
db0b5d7d1e Add test to fuzzer.c 2020-10-27 16:57:24 -04:00
sen
17b700d78a
Merge pull request #2366 from senhuang42/enable_ldm_by_default 
Enable LDM by default if window size >= 128MB and strategy uses opt parser
 2020-10-27 14:59:28 -04:00
senhuang42
dc448563e9 Add test compatibility with last literals in sequences 2020-10-27 12:35:28 -04:00
Yann Collet
d3f1a9b5bd fix partial-build test 
sometimes, the scope difference is solely determined by the list of source files,
not by the flags.
 2020-10-22 21:36:09 -07:00
Yann Collet
91a8cb9559 fix DEBUGLEVEL redefinition from tests/ 2020-10-22 00:20:40 -07:00
Yann Collet
494f7169ed fix directory creation for Windows' libzstd 2020-10-22 00:15:31 -07:00
Yann Collet
ca75da8fa3 fix test 
DEBUGLEVEL redefinition
 2020-10-21 23:51:13 -07:00
Nick Terrell
d6dae2000b
Merge pull request #2365 from senhuang42/move_opt_parser_test_to_long_tests 
Move ldm + opt parser no regression test to long tests
 2020-10-20 11:34:36 -04:00
senhuang42
81a2c02d8f Move ldm no regression test to fuzzer longtests 2020-10-19 15:28:46 -04:00
senhuang42
df470e176b Add unit test for no cctx requested params change 2020-10-19 10:52:41 -04:00
senhuang42
42d037bdba Add libregression build target, also fix make clean and .gitignore 2020-10-15 10:34:50 -04:00
Yann Collet
f5d5cd3b40
Merge pull request #2341 from senhuang42/ldm_optimized_for_opt_parser 
Integrate long distance matches into optimal parser
 2020-10-13 13:09:07 -07:00
Nick Terrell
ede4f97153 [zstdmt] Fix bug where extra empty blocks are emitted 
When zstdmt cannot get a buffer and `ZSTD_e_end` is passed an empty
compression job can be created. Additionally, `mtctx->frameEnded` can be
set to 1, which could potentially cause problems like unterminated blocks.

The fix is to adjust to `ZSTD_e_flush` even when we can't get a buffer.
 2020-10-12 12:55:17 -07:00
Nick Terrell
9ab9229e11 [zstreamtest] Add compression determinism tests 
* Run compression twice and check the compressed data is byte-identical.
  The compression loop had to be rewritten to ensure deteriminism. It is
  guaranteed by always making maximal forward progress.
* When nbWorkers > 0, change the number of workers 1/8 of the time.
* Run in single-pass mode 1/4 of the time.

I've run a few hundred thousand iterations of zstreamtest and have seen
no deteriminism issues so far. Before the zstdmt fix that skips the
single-pass shortcut non-determinism showed up in a few hundred
iterations.
 2020-10-12 12:55:17 -07:00