Yann Collet
6f1377301f
circleci: removed -m32 tests
...
installing dependencies costs ~2mn, making it the longest test
(test itself was < 15sec)
2017-12-01 18:34:22 -08:00
Yann Collet
272c142330
circleci : balance some tests
...
try to group them by "equivalent" duration
since next job will not start as long as previous one is not completed on both VM
2017-12-01 18:11:34 -08:00
Yann Collet
6127592391
circleci : removed cmaketest
...
strange error on pthread detection
(seems a bug in cmake detection, since
`make zstd` works perfectly with `-pthread` on same conf)
2017-12-01 17:56:16 -08:00
Yann Collet
71f012e5bf
zstdcli: fixed minor warning when bench module not enabled
...
one variable defined but not used
2017-12-01 17:42:46 -08:00
Yann Collet
c6f5bf071c
fixed powerpc test
...
misspelled CFLAGS
2017-12-01 17:35:29 -08:00
Yann Collet
74991c2764
circleci: removed gccX/clang tests
...
does no longer need to install these large packages
2017-12-01 17:28:24 -08:00
Yann Collet
30eeb248a8
added environment variable through circleci dashboard
...
check if it works better
2017-12-01 17:22:04 -08:00
Yann Collet
363355448b
circleci: removed one test node
2017-12-01 17:02:55 -08:00
Yann Collet
519cf3b400
circleci: add CFLAGS=-O0 environment variable
...
for faster compilation
2017-12-01 16:31:40 -08:00
Yann Collet
4413ed27e4
circleci : make libzstd-nomt had its own test
2017-12-01 16:27:24 -08:00
Yann Collet
453fb5e830
removed -ftrapv from tests/ debug flags
...
-ftrapv is apparently buggy for gcc.
versions >= 5 are supposed to work better,
but even then, some complaints say it's still flaky when optimizations are enabled.
I even saw a post saying it only works if one creates its own signal handler,
which would make this flag no longer transparent.
on clang, it seems to work correctly.
But we would need to add a method to selectively add flags depending on compiler.
That's too much troubles for the intended benefit
(just catch integer overflows, which we can also do using ubsan).
2017-12-01 10:30:53 -08:00
Yann Collet
b3d76e0a94
Merge branch 'dev' of github.com:facebook/zstd into dev
2017-11-28 15:03:38 -08:00
Yann Collet
f5747afe0f
add integer overflow trapping to debug builds
...
doc: https://gcc.gnu.org/onlinedocs/gcc-4.0.2/gcc/Code-Gen-Options.html
2017-11-28 15:02:48 -08:00
Yann Collet
be73f8a749
Merge pull request #893 from felixhandte/fix-lz4-compression-bug
...
Fix LZ4 Compression Bug
2017-11-28 11:34:22 -08:00
W. Felix Handte
baff9dd15e
Fix LZ4 Compression Buffer Overflow
...
Fixes issue where, when `zstd --format=lz4` is fed an input larger than 128KB,
the read overruns the input buffer. This changes Zstd to use LZ4 with chained
64KB blocks. This is technically a breaking change in that some third party
LZ4 implementations may not support linked blocks. However, progress should not
be allowed to be stopped by such petty concerns as backwards compatibility!
2017-11-28 12:07:26 -05:00
W. Felix Handte
62c746dcf9
Add Test on LZ4 Format Input Buffer Overrun
2017-11-28 12:06:48 -05:00
Yann Collet
743b23878e
install: changed variable MANDIR into MAN1DIR
...
MANDIR still exists, and is now the parent of MAN1DIR
2017-11-27 13:47:35 -08:00
Yann Collet
bd88f633ac
zstreamtest : in -T#s, s considered a suffix meaning "seconds"
...
avoid unintentionnally triggering `seedset`,
so that seed gets automatically determined when not set.
2017-11-27 12:15:23 -08:00
Yann Collet
2fd765498a
updated man page
...
following patch #931 by @scottchiefbaker
2017-11-24 17:20:54 -08:00
Yann Collet
c857ee850a
minor update
2017-11-24 16:44:28 -08:00
Yann Collet
73a6c6ca5a
Merge pull request #931 from scottchiefbaker/documentation
...
Include information about the benchmark output/methodology
2017-11-24 00:01:34 -08:00
Scott Baker
31a191b178
Include information about the benchmark output/methodology
...
Addresses #930
2017-11-22 20:34:25 -08:00
Yann Collet
4451b213c0
Merge pull request #924 from facebook/opt2
...
faster btopt variant
2017-11-21 16:34:37 -08:00
Yann Collet
f8d5c478af
fixed comment, reported by @gyscos
2017-11-21 10:36:14 -08:00
Yann Collet
4154aec679
fixed comment, as suggested by @terrelln
2017-11-21 10:26:17 -08:00
Yann Collet
851383e7a4
Merge pull request #925 from dusta/patch-1
...
Missing dot
2017-11-20 14:34:03 -08:00
Dusta
287db17c8c
Missing dot
2017-11-20 21:35:28 +01:00
Yann Collet
899f2a29f6
strategy ZSTD_btopt pinned to (0) variant (faster one)
2017-11-20 11:53:20 -08:00
Yann Collet
3f457264d1
slightly improved compression speed
2017-11-19 14:40:21 -08:00
Yann Collet
42c1e64270
slightly improved ratio at -22
...
merging of repcode search into btsearch introduced a small compression ratio regressio at max level :
1.3.2 : 52728769
after repMerge patch : 52760789 (+32020)
A few minor changes have produced this difference.
They can be hard to spot.
This patch buys back about half of the difference,
by no longer inserting position at hc3 when a long match is found there.
It feels strangely counter-intuitive, but works :
after this patch : 52742555 (-18234)
2017-11-19 14:00:55 -08:00
Yann Collet
99435dbbab
minor : search early-out on sufficient_len for hc3 and rep
...
very very small speed and ratio increases
2017-11-19 12:58:04 -08:00
Yann Collet
d100670045
btopt0 : a bit faster and weaker
2017-11-19 10:38:02 -08:00
Yann Collet
e6da37c430
created (hidden) new strategy btopt0
...
about ~+10% faster but losing ~0.01 compression ratio
(note : amplitude vary a lot depending on files, but direction remains the same)
2017-11-19 10:21:21 -08:00
Yann Collet
e717a5b0dd
zstd_opt: minor speed optimization
...
Calculate reference log2sums only once per serie of sequence
(as opposed to once per sequence)
Also: improved code comments
2017-11-18 16:24:02 -08:00
Yann Collet
daebc7fe26
bench: slightly adjusted display format
...
adapt accuracy depending on value.
makes it possible to have higher accuracy for small value,
notably small compression speed.
This capability is expected to be useful while modifying optimal parser.
2017-11-18 15:54:32 -08:00
Yann Collet
fccb46fbe0
minor spelling fixes
2017-11-18 11:28:00 -08:00
Yann Collet
d11661c3ec
fix ZSTD_COMPRESSBOUND() macro
...
It was using macro `KB`, which is not defined in `zstd.h`.
2017-11-18 11:16:39 -08:00
Yann Collet
22ff60b3aa
Merge pull request #922 from terrelln/signal
...
[zstd] Fix rare bug with signal handler
2017-11-17 18:50:02 -08:00
Yann Collet
9708c46879
Merge pull request #921 from facebook/ubfix
...
Fixes some arithmetic pointer operations
2017-11-17 17:11:18 -08:00
Nick Terrell
a6052af0e8
[zstd] Fix rare bug with signal handler
2017-11-17 16:38:56 -08:00
Yann Collet
a4a20a4b2f
fix un-initialized memory warning
...
harmless, but cleaner
2017-11-17 15:51:52 -08:00
Yann Collet
c8b3e08535
Merge pull request #920 from facebook/benchSeparate
...
Bench multiple files
2017-11-17 13:25:12 -08:00
Yann Collet
23767e950a
fix one UB pointer arithmetic in encoder
...
Instead of calculating distance between 2 memory objects, which is UB,
we extract the offset from object 1, and transfer it into object 2.
2017-11-17 13:24:51 -08:00
Yann Collet
cdade555ee
fixed one UB pointer arithmetic
2017-11-17 11:40:08 -08:00
Yann Collet
6e8573a90a
Merge pull request #916 from facebook/optMergeRep
...
Optimal parser part 2 : Merge normal and _extDict parsers
2017-11-17 11:29:47 -08:00
Yann Collet
49a445e65d
fixed circle-ci script
2017-11-17 02:14:16 -08:00
Yann Collet
5b957ba899
minor interface adjustments
2017-11-17 01:21:40 -08:00
Yann Collet
d898fb7ba6
bench: added cli command -S to benchmark multiple files separately
...
Currently, all files are joined by default,
they are compressed separately but benchmarked together,
providing a single final result.
Benchmarking files separately make it possible to accurately measure difference for each file.
This is expected to be useful while tuning optimal parser.
2017-11-17 00:22:55 -08:00
Yann Collet
8accfa7fcc
bench: realTime is a global parameter
...
like most parameters not directly related to compression
2017-11-17 00:02:37 -08:00
Yann Collet
11e58d9ba4
fixed minor warning
...
warning: void function returning a value
(even if the return value is void)
2017-11-16 15:21:30 -08:00
