Yann Collet
9b5b47ac93
ensure adjustCParams adjust hLog and cLog even without srcSize
...
It would previously exit when srcSize is unknown.
But in the case of custom parameters,
hLog and cLog can still be too large in comparison with windowLog.
Reduces maximum memory allocated during zstreamtest --newapi
2017-09-28 01:25:40 -07:00
Nick Terrell
c233bdbaee
Increase maximum window size
...
* Maximum window size in 32-bit mode is 1GB, since allocations for 2GB fail
on my Mac.
* Maximum window size in 64-bit mode is 2GB, since that is the largest
power of 2 that works with the overflow prevention.
* Allow `--long=windowLog` to set the window log, along with
`--zstd=wlog=#`. These options also set the window size during
decompression, but don't override `--memory=#` if it is set.
* Present a helpful error message when the window size is too large during
decompression.
* The long range matcher defaults to a hash log 7 less than the window log,
which keeps it at 20 for window log 27.
* Keep the default long range matcher window size and the default maximum
window size at 27 for the API and CLI.
* Add tests that use the maximum window size and hash size for compression
and decompression.
2017-09-26 14:00:01 -07:00
Nick Terrell
d6abb28951
Prepare for ZSTD_WINDOWLOG_MAX == 31
2017-09-21 17:18:41 -07:00
Yann Collet
f1571dad8f
Merge pull request #838 from stellamplau/ldm-mergeDev
...
Add long distance matcher
2017-09-13 13:24:08 -07:00
Stella Lau
eb3327c10a
Merge branch 'dev' of https://github.com/facebook/zstd into ldm-mergeDev
2017-09-11 15:00:01 -07:00
Yann Collet
ce31004f20
fix following suggestions by @terrelln
2017-09-11 13:12:52 -07:00
Yann Collet
058ed2ad33
ZSTD_decodingBufferSize_min()
...
supporting function for bufferless streaming API (ZSTD_decompressContinue())
makes it possible to correctly size a round buffer for decoding using this API.
also : added field blockSizeMax within ZSTD_frameHeader,
as it's a necessary information to know when to restart at beginning of decoding buffer.
2017-09-09 01:03:29 -07:00
Yann Collet
3128e03be6
updated license header
...
to clarify dual-license meaning as "or"
2017-09-08 00:09:23 -07:00
Stella Lau
9e4060200b
Add tests and fix pointer alignment
2017-09-06 09:14:05 -07:00
Stella Lau
c706de5395
Rename and add short ldm parameters in cli
2017-09-05 21:11:18 -07:00
Stella Lau
643d28c701
Add ldm options to 'man zstd'
2017-09-05 11:27:15 -07:00
Stella Lau
67d4a6161c
Add ldmBucketSizeLog param
2017-09-02 21:55:29 -07:00
Stella Lau
a1f04d518d
Move hashEveryLog to cctxParams and update cli
2017-09-01 15:05:47 -07:00
Stella Lau
767a0b3be1
Move ldm hashLog, bucketLog, and mml to cctxParams
2017-09-01 12:24:59 -07:00
Stella Lau
17d8e0bdcc
Merge remote-tracking branch 'upstream/longRangeMatcher' into ldm-integrate
2017-09-01 10:19:38 -07:00
Stella Lau
8081becadc
Add long distance matching as a CCtxParam
2017-09-01 09:18:58 -07:00
Yann Collet
d7ad99b2ab
Merge branch 'longRangeMatcher' into dev
2017-08-31 18:08:37 -07:00
Stella Lau
6a546efb8c
Add long distance matcher
...
Move last literals section to ZSTD_block_internal
2017-08-31 12:53:19 -07:00
Stella Lau
90a31bfa16
Pass dictMode to ZSTDMT_initCStream; fix nits
...
- Return error code in estimate{CCtx,CStream}Size functions
2017-08-30 16:19:07 -07:00
Stella Lau
82d636b76a
Rename applyCCtxParams()
2017-08-29 18:03:06 -07:00
Stella Lau
c7a18b7c21
Localize 'dictMode' from cctx to function param
2017-08-29 15:52:24 -07:00
Stella Lau
c88fb9267f
Replace 'byReference' with enum
2017-08-29 11:55:02 -07:00
Stella Lau
b5b9275e67
Rename estimateCCtxSize_advanced() and estimateCStreamSize_advanced()
2017-08-29 10:49:29 -07:00
Stella Lau
18224608ff
Remove ZSTD_setCCtxParameter()
2017-08-25 13:58:41 -07:00
Stella Lau
0744592d38
Add function initializing cctxParams from clevel
2017-08-25 13:36:47 -07:00
Stella Lau
eb7bbab36a
Remove ZSTD_p_refDictContent and dictContentByRef
2017-08-25 11:11:45 -07:00
Stella Lau
15fdeb9e41
Enforce nbThreads<=1 for estimateCCtxSize
2017-08-24 16:28:49 -07:00
Stella Lau
5bc2c1e982
Add prototype support for customMem with cctxParams
2017-08-23 12:03:30 -07:00
Stella Lau
6f1a21c7e9
Remove formatting-only changes
2017-08-23 10:24:19 -07:00
Stella Lau
5b956f4753
Comment out CCtx_param versions of CDict functions
2017-08-21 14:49:16 -07:00
Stella Lau
1c0dbe81b1
Add documentation for CCtx_params
2017-08-21 13:18:00 -07:00
Stella Lau
939f954285
Pass ZSTD_CCtx_params as const ptr when possible
2017-08-21 12:57:18 -07:00
Stella Lau
73c73bf16a
Reduce code duplication in zstreamtest
2017-08-21 12:41:19 -07:00
Stella Lau
91b30dbe84
Remove test parameter
2017-08-21 10:09:06 -07:00
Stella Lau
f181f33bdf
Disable tests and refactor
2017-08-21 01:59:08 -07:00
Stella Lau
023b24e6d4
Add cctx param tests
2017-08-20 22:55:07 -07:00
Yann Collet
32fb407c9d
updated a bunch of headers
...
for the new license
2017-08-18 16:52:05 -07:00
Stella Lau
2300c58a6f
Move dictContentByRef to cctx params
2017-08-18 12:03:16 -07:00
Stella Lau
b6cb2ed8cb
Move dictMode to cctxParams
2017-08-18 11:43:31 -07:00
Stella Lau
97e27affcb
Move compression level to cctx params
2017-08-18 11:20:08 -07:00
Stella Lau
c0221124d5
Add function to set opaque parameters
2017-08-17 19:30:22 -07:00
Stella Lau
4169f49171
Add initialization/allocation functions for opaque params
2017-08-17 18:45:04 -07:00
Stella Lau
ade95b8bed
Add opaque interfaces for static initialization
2017-08-17 18:13:08 -07:00
Stella Lau
699f11b4f7
Create opaque parameter structure
2017-08-17 17:33:46 -07:00
Yann Collet
e1222544be
Merge pull request #753 from paulcruz74/adapt-approach-3
...
adaptive compression v1
2017-07-27 10:00:10 -07:00
Paul Cruz
6945b3c43d
removed previous version of completion for compression
2017-07-19 11:51:50 -07:00
Paul Cruz
50ce4eaeb6
added error detection for pthread initialization, added compression completion measurement, fixed const values
2017-07-17 10:12:44 -07:00
Yann Collet
4e77f7761d
clarified comment on ZSTD_p_contentSizeFlag
2017-07-13 02:09:07 -07:00
Yann Collet
40156a4967
bumped version nb to v1.3.1
2017-07-08 04:55:09 -07:00
Yann Collet
ead4dd48f6
new field frameHeader.headerSize
2017-07-07 15:51:24 -07:00
