W. Felix Handte
fe96e98f81
Support a Separate Hash Log in ZSTD_fast
2018-09-28 17:12:54 -07:00
W. Felix Handte
bc880ebe8f
Stop Passing in hashLog and stepSize to ZSTD_compressBlock_fast_generic
2018-09-28 17:12:54 -07:00
W. Felix Handte
b3107c7799
Temporary Commit to Retain Requested Hash and Chain Logs During Dict Attach
2018-09-28 17:12:54 -07:00
W. Felix Handte
34e0193129
Allow Setting Hash and Chain Logs on Contexts with Attached CDict
2018-09-28 17:12:54 -07:00
W. Felix Handte
eae8232f50
For Supported Strategies, Attach Dict Even When Params Don't Match
2018-09-28 17:12:54 -07:00
W. Felix Handte
01ff945eae
Split Attach and Copy Reset Strategies into Separate Implementation Functions
2018-09-28 17:12:54 -07:00
W. Felix Handte
a6d6bbeae1
Pull Attachment Decision into Separate Function
2018-09-28 17:12:54 -07:00
W. Felix Handte
b7fba599ae
And Then Avoid the Unused Parameter Warning
2018-09-28 17:12:54 -07:00
W. Felix Handte
1f188ae655
Move Asserts into Function to Avoid Unused Function Warning
2018-09-28 17:12:54 -07:00
W. Felix Handte
7212b5e5c2
Move Match State CParams Setting into resetCCtx and continueCCtx
2018-09-28 17:12:54 -07:00
W. Felix Handte
01e34d365b
Strengthen Assertion to Assert Equality
2018-09-28 17:12:53 -07:00
W. Felix Handte
50cc1cf4d5
Remove CParams Arg from ZSTD_ldm_blockCompress
2018-09-28 17:12:53 -07:00
W. Felix Handte
14764de49f
Stop Separately Passing CParams in ZSTD_lazy Internal Functions
2018-09-28 17:12:53 -07:00
W. Felix Handte
97149f22c3
Stop Separately Passing CParams in ZSTD_opt Internal Functions
2018-09-28 17:10:42 -07:00
W. Felix Handte
dcdf437fed
Also Remove CParams from Table Filling Functions' Args
2018-09-28 17:10:42 -07:00
W. Felix Handte
3483f89101
Also Assert Equivalency When Filling MatchState with Prefix
2018-09-28 17:10:42 -07:00
W. Felix Handte
6cb2454646
Remove CParams from Block Compressor Functions' Args
2018-09-28 17:10:42 -07:00
W. Felix Handte
03103269de
Assert ctx and ms cparams Equivalency
2018-09-28 17:10:42 -07:00
W. Felix Handte
4e3ecee9ed
Remove cParams from CDict
2018-09-28 17:10:42 -07:00
W. Felix Handte
76ef87ed9d
Add ZSTD_compressionParameters to ZSTD_matchState_t
2018-09-28 17:10:42 -07:00
Yann Collet
73773c6b6a
fixed legacy compilation tests
...
for some reason, these tests started failing recently on CircleCI
2018-09-27 18:15:14 -07:00
Nick Terrell
a180ea07c4
Restore ZSTD_noCompressBlock() for clarity
2018-09-27 16:06:02 -07:00
Nick Terrell
aec1a3ec58
Change byte to value to avoid a GRUB typedef
2018-09-27 15:24:48 -07:00
Nick Terrell
109bd37474
Include stddef.h for size_t
2018-09-27 15:24:48 -07:00
Nick Terrell
f2d6db45cd
[zstd] Add -Wmissing-prototypes
2018-09-27 15:24:48 -07:00
Yann Collet
2a5cd8535a
Merge pull request #1342 from facebook/fixcatyd
...
fix : huge (>4GB) chain of blocks
2018-09-27 10:20:14 -07:00
Yann Collet
404a7bfed0
moved again overflow correction
...
cannot work from within ZSTD_compressBlock()
2018-09-26 18:06:53 -07:00
Yann Collet
0e2dbac18a
changed overflow correction place
...
keep one in compress_frameChunk(),
so that it's tested at every loop
in case some user simply some large mulit-GB input in a single invocation.
Add one in ZSTD_compressBlock(),
since compressBlock() explicitly skips frameChunk().
2018-09-26 15:35:38 -07:00
Yann Collet
e74eade251
Merge pull request #1339 from facebook/grep_colors
...
fixed usage of grep in Makefile
2018-09-26 14:39:20 -07:00
Yann Collet
8883af6a1e
Merge pull request #1327 from facebook/adapt
...
Adaptive compression
2018-09-26 14:39:08 -07:00
Yann Collet
f98c69d77c
fix : huge (>4GB) stream of blocks
...
experimental function ZSTD_compressBlock() is designed for very small data in mind,
for situation where saving the ~12 bytes of frame header can actually make a difference.
Some systems though may have to deal with small and large data entangled.
If it's larger than a block (> 128KB), compressBlock() cannot compress them in one round.
That's why it's possible to compress in multiple rounds.
This is a chain of compressed blocks.
Some users push this capability to the limit, encoding gigantic chain of blocks.
On crossing the 4GB limit, some internal overflow occurs.
This fix moves the overflow correction mechanism higher in the call chain,
so that it's applied also to gigantic chains of blocks.
Added a test case in fuzzer.c, which crashes before the fix, and pass now.
2018-09-26 14:24:28 -07:00
Yann Collet
8ff17a6a09
Merge pull request #1329 from facebook/v04isout
...
Changed default legacy support to v0.5+
2018-09-26 13:39:05 -07:00
Yann Collet
08f68d83c5
fixed usage of grep in Makefile
...
when terminal uses colors
as suggested by @danielshir (#1294 )
2018-09-25 16:56:53 -07:00
Yann Collet
04f47bbdd2
Merge branch 'dev' into adapt
2018-09-24 16:56:45 -07:00
Yann Collet
9bb6c15f79
Merge pull request #1332 from facebook/minclevel
...
defined a minimum negative level
2018-09-24 16:01:13 -07:00
Yann Collet
292d8e4a83
added some tests based on limits.h
...
in order to ensure proper type mapping
when not using stdint.h
2018-09-23 23:57:30 -07:00
Yann Collet
71a5210617
avoid recompiling dll every time under mingw
2018-09-21 17:40:30 -07:00
Yann Collet
c484345a82
Merge branch 'mingw' into adapt
2018-09-21 16:00:46 -07:00
Yann Collet
bfff4f4809
ensure all writes to job->cSize are mutex protected
...
even when reporting errors,
using a macro for code brevity, as suggested by @terrelln,
2018-09-21 16:00:39 -07:00
Yann Collet
32b7cf1bcf
fixed tautological tests
...
involving ZSTD_TARGETLENGTH_MIN (== 0)
2018-09-21 15:04:43 -07:00
Yann Collet
c044345f8f
Merge branch 'mingw' into minclevel
2018-09-21 14:56:57 -07:00
Yann Collet
de6c75e4e5
Merge pull request #1318 from felixhandte/shadow-dict-matches
...
Don't Search Dictionary Context When Working Context Search Resulted in Mismatch
2018-09-21 12:15:33 -07:00
Yann Collet
a54c86cfc6
defined a minimum negative level
...
which can be probed using new function ZSTD_minCLevel().
Also : redefined ZSTD_TARGETLENGTH_MIN/MAX for consistency
used the opportunity to bump version number to v1.3.6
2018-09-20 16:52:03 -07:00
Yann Collet
b2939163e1
Changed default legacy support to v0.5+
...
thus dropping read support for v0.4.
It's always possible to re-enable it, by changing build macro ZSTD_LEGACY_SUPPORT to 4.
2018-09-20 14:30:20 -07:00
Yann Collet
7992942d66
fixed complex tsan issue
...
when job->consumed == job->src.size , compression job is presumed completed,
so it must be the very last action done in worker thread.
2018-09-20 13:47:31 -07:00
Yann Collet
6b07a66aec
fixed minor reporting discrepancy in MT mode
2018-09-19 16:30:55 -07:00
Yann Collet
ca02ebee07
removed static variables
...
so that --adapt can work on multiple input files too
2018-09-19 15:25:50 -07:00
Yann Collet
89bc309d90
error out when --adapt is associated with --single-thread
...
since they are not compatible
2018-09-19 14:49:13 -07:00
Yann Collet
2f78228f65
Merge branch 'dev' into adapt
2018-09-19 12:43:42 -07:00
Yann Collet
005f000aed
updated documentation of *refPrefix()
...
indicating the equivalence with `diff` operation.
2018-09-18 13:07:08 -07:00
