9550 Commits

Author SHA1 Message Date
Yann Collet
86bd977a79
Merge pull request #3117 from cuishuang/dev
fix some typos
2022-04-26 10:02:04 -07:00
cuishuang
05796796fd fix some typos
Signed-off-by: cuishuang <imcusg@gmail.com>
2022-04-26 17:40:23 +08:00
Yann Collet
66633f9386
Merge pull request #3039 from eli-schwartz/meson
Meson fixups for Windows
2022-04-19 15:51:19 -07:00
Yann Collet
f1faab6720
Merge pull request #3112 from facebook/man2
updated man page, providing more details for --train mode
2022-04-19 15:36:47 -07:00
Yann Collet
eb726c6a20 updated man pages
had to run the conversion script on Ubuntu, as it doesn't run correctly on macos anymore.
2022-04-13 18:57:27 -07:00
Yann Collet
0df2fd6088 updated man page, providing more details for --train mode
following questions from #3111.

Note : only the source markdown has been updated,
the actual man page zstd.1 still need to be processed.
2022-04-13 18:51:59 -07:00
Yann Collet
460780f804
Merge pull request #3094 from dirkmueller/usage_cleanup
Split help in long and short version, cleanup formatting
2022-04-05 07:09:54 -07:00
Yann Collet
e4cd9bbd88
Merge pull request #3108 from paulmenzel/fix-typo-in-zstd.1
Remove superfluous *not* in description of `--long[=#]` in zstd(1)
2022-04-03 23:12:06 -07:00
Paul Menzel
f133bc8c9c zstd.1: Remove superfluous *not* in description of --long[=#]
Resolves: https://github.com/facebook/zstd/issues/3101
2022-04-03 07:29:51 +02:00
Elliot Gorokhovsky
3e6bbdd847
Disable visual-2015 tests (#3106) 2022-03-31 12:26:20 -04:00
Yann Collet
455c2c21e6
Merge pull request #3103 from facebook/fix45586
fix minor bug in sequence_compression_api tester
2022-03-30 10:08:57 -07:00
Yann Collet
678bfff4fe fix minor bug in sequence_compression_api tester
margin was merely slightly too short for extra splitting.
2022-03-29 16:45:09 -07:00
Dirk Müller
7fbe60d577
Split help in long and short version, cleanup formatting
Adopt the more standard Usage: formatting style
List short and long options alongside where available
Print lists as a table
Use command style description
2022-03-29 12:57:47 +02:00
Nick Terrell
f229daaf42
Merge pull request #3052 from dirkmueller/gzip_keep
Keep original file if -c or --stdout is given
2022-03-28 10:35:21 -07:00
Elliot Gorokhovsky
64efba4c5e
Software pipeline for ZSTD_compressBlock_fast_dictMatchState (#3086)
* prefetch dict content inside loop

* ip0/ip1 pipeline

* add L2_4 prefetch to dms pipeline

* Remove L1 prefetch

* Remove L2 prefetching

* Reduce # of gotos

* Cosmetic fixes

* Check final position sometimes

* Track step size as in bc768bc

* Fix nits
2022-03-17 12:35:11 -04:00
Nick Terrell
eadb6c874f
Merge pull request #3095 from dpelle/typo-and-grammar-fixes
Typo and grammar fixes
2022-03-14 09:17:21 -07:00
Dominique Pelle
3a64aa29a6 On more mistake (Node -> Note) 2022-03-13 00:08:55 +01:00
Dominique Pelle
b772f53952 Typo and grammar fixes 2022-03-12 08:58:04 +01:00
Nick Terrell
05fc7c78c8
Merge pull request #3088 from cyberknight777/dev
[contrib][linux] Fix a warning in zstd_reset_cstream()
2022-03-11 10:01:11 -08:00
Nick Terrell
c3a89ef60c
Merge pull request #3093 from dirkmueller/cli_tests_fixup
Handle newer less versions in zstdless testing
2022-03-10 10:41:52 -08:00
Dirk Müller
7a3997c21a
Handle newer less versions in zstdless testing
Newer less versions appear to have changed how stderr
and stdout are showing error messages. hardcode the
expected behavior to make the tests pass with any less version.

Also set locale to C so that the strings are matching.
2022-03-10 09:47:33 +01:00
Cyber Knight
498ac8238d
[contrib][linux] Make zstd_reset_cstream() functionally identical to ZSTD_resetCStream()
- As referenced by Nick Terrelln ~ the ZSTD maintainer in the linux kernel, making zstd_reset_cstream() functionally identical to ZSTD_resetCStream() would be the perfect way to fix the warning without touching any core functions or breaking other parts of the code.

Suggested-by: Nick Terrell <terrelln@fb.com>
Signed-off-by: Cyber Knight <cyberknight755@gmail.com>
2022-03-10 15:32:13 +08:00
Nick Terrell
6a8fba9e5f
Merge pull request #3092 from terrelln/2022-03-09-decoder-errata-doc
[doc] Add decompressor errata document
2022-03-09 15:10:29 -08:00
Nick Terrell
696fa2524a [doc] Add decompressor errata document
Add a document that lists the known bugs in the decoder where valid
frames are rejected, along with the version that they were fixed.
2022-03-09 14:40:41 -08:00
Dirk Müller
3f4f8b04ed
Keep original file if -c or --stdout is given
Set removeSrcFile back to false when -c or --stdout is used to improve
compatibility with gzip(1) behavior.

gzip(1) is removing the original file on compression unless --stdout or
/-c is used. zstd is defaulting to keep the file unless --rm is used or
when it is called via a gzip symlink, in which it is removing by
default. Specifying -c/--stdout turns this behavior off.
2022-03-09 23:38:59 +01:00
Felix Handte
8814aa5bfa
Merge pull request #3059 from dirkmueller/gzip_fast_best
Implement more gzip compatibility (#3037)
2022-03-08 15:32:06 -05:00
Yann Collet
db104f6e83
Merge pull request #3079 from niamster/cmake-compat-5
build:cmake: enable ZSTD legacy support by default
2022-03-08 00:21:23 -08:00
Cyber Knight
8ff20c25f3
[contrib][linux] Use ZSTD_CCtx_setPledgedSrcSize() instead of ZSTD_CCtx_reset()
- The previous patch throws the following warning:

 ../linux/lib/zstd/zstd_compress_module.c: In function ‘zstd_reset_cstream’:
../linux/lib/zstd/zstd_compress_module.c:136:34: error: enum conversion when passing argument 2 of ‘ZSTD_CCtx_reset’ is invalid in C++ [-Werror=c++-compat]
  136 |  return ZSTD_CCtx_reset(cstream, pledged_src_size);
      |                                  ^~~~~~~~~~~~~~~~
In file included from ../linux/include/linux/zstd.h:26,
                 from ../linux/lib/zstd/zstd_compress_module.c:15:
../linux/include/linux/zstd_lib.h:501:20: note: expected ‘ZSTD_ResetDirective’ {aka ‘enum <anonymous>’} but argument is of type ‘long long unsigned int’
  501 | ZSTDLIB_API size_t ZSTD_CCtx_reset(ZSTD_CCtx* cctx, ZSTD_ResetDirective reset);
      |                    ^~~~~~~~~~~~~~~
cc1: all warnings being treated as errors

Since we have a choice to either use ZSTD_CCtx_reset or ZSTD_CCtx_setPledgedSrcSize instead of ZSTD_resetCStream, let's switch to ZSTD_CCtx_setPledgedSrcSize to not have any unnecessary warns alongside the kernel build and CI test build.

Signed-off-by: Cyber Knight <cyberknight755@gmail.com>
2022-03-08 13:38:34 +08:00
Cyber Knight
e470c940f6 [contrib][linux] Fix a warning in zstd_reset_cstream()
- This fixes the below warning:

../lib/zstd/zstd_compress_module.c: In function 'zstd_reset_cstream':
../lib/zstd/zstd_compress_module.c:136:9: warning: 'ZSTD_resetCStream' is deprecated [-Wdeprecated-declarations]
  136 |         return ZSTD_resetCStream(cstream, pledged_src_size);
      |         ^~~~~~
In file included from ../include/linux/zstd.h:26,
                 from ../lib/zstd/zstd_compress_module.c:15:
../include/linux/zstd_lib.h:2277:8: note: declared here
 2277 | size_t ZSTD_resetCStream(ZSTD_CStream* zcs, unsigned long long pledgedSrcSize);
      |        ^~~~~~~~~~~~~~~~~

ZSTD_resetCstream is deprecated and zstd_CCtx_reset is suggested to use hence let's switch to it.

Signed-off-by: Cyber Knight <cyberknight755@gmail.com>
2022-03-07 11:55:33 +08:00
Yann Collet
930b5cecaa
Merge pull request #3060 from rex4539/typos
Fix typos
2022-03-05 14:21:58 -08:00
Dimitris Apostolou
cf1894b324
Fix typos 2022-03-05 23:47:25 +02:00
Xi Ruoyao
d109cef201
fix the assertion in readLinesFromFile (#3084)
* fix the assertion in readLinesFromFile

When the file is not terminated by endline, readLineFromFile will append
a '\0' for the last line.  In this case pos + lineLength == dstCapacity.

* test: don't print very long text garbage
2022-03-04 14:56:44 -05:00
Yann Collet
e8448a3d61
Merge pull request #3080 from TocarIP/tokarip/static_bmi2
Enable STATIC_BMI2 for gcc/clang
2022-03-03 12:49:40 -08:00
Ilya Tokar
7c3d1cb3ab Enable STATIC_BMI2 for gcc/clang
Some usage (e.g. BIT_getLowerBit) uses it without checking for MSVC,
so enabling for clang gives a small performance boost.
2022-03-03 15:03:54 -05:00
Elliot Gorokhovsky
0c386afbfd
Fix CI failures by adding apt-get update to Github Actions (#3082)
* Fix CI failures by adding apt-get update to Makefile

* Fix travis failure caused by apt-get update

* Move apt-get update from Makefile to Github Actions .yml

* Revert .travis.yml change

* Fix typo
2022-03-03 14:52:06 -05:00
Nick Terrell
41665679bf
Merge pull request #3081 from terrelln/dict-load-zero
[programs] Fix infinite loop when empty input is passed to trainer
2022-03-02 12:58:51 -08:00
Nick Terrell
da737c7ab8 [programs] Fix infinite loop when empty input is passed to trainer
When an empty input file was passed to the dictionary trainer, it would infinite loop.
The added test case exposes the bug, and is fixed with this PR.
2022-03-02 11:05:26 -08:00
Dmytro Milinevskyi
03bba1b0bf build:cmake: enable ZSTD legacy support by default 2022-03-01 18:29:47 +01:00
Elliot Gorokhovsky
87406b5f3b
Merge pull request #3075 from TocarIP/tokarip/bzhi
Use helper function for bit manipulations.
2022-02-24 16:52:48 -05:00
Ilya Tokar
0178c12dd9 Use helper function for bit manipulations.
We already have BIT_getLowerBits, so use it. Benefits are 2fold:
1) Somewhat cleaner code
2) We are now using bzhi instructions, when available. Performance
delta is too small for microbenchmarks, but avoiding load still helps
larger applications, by reducing data cache pressure.
2022-02-23 17:59:56 -05:00
Elliot Gorokhovsky
621d798988
Merge pull request #3045 from embg/asm_refactor
Refactor bitwise intrinsics
2022-02-17 13:08:08 -05:00
Elliot Gorokhovsky
71d9dab76f Replace XOR with subtraction for readability 2022-02-16 16:49:42 -05:00
Elliot Gorokhovsky
856c7dc51d Fix fuzzer.c nits and replace CLZ fallback 2022-02-16 11:40:05 -05:00
Elliot Gorokhovsky
00f2acba36 Add back check to prevent Win32 static analysis issues 2022-02-15 11:41:09 -05:00
Elliot Gorokhovsky
7c674a0919 Add tests for bitwise intrinsics 2022-02-14 18:36:33 -05:00
Elliot Gorokhovsky
6994a9f99c bits.h refactor and bugfix 2022-02-14 16:59:55 -05:00
Elliot Gorokhovsky
529cd7b821 Fix nits 2022-02-14 14:24:50 -05:00
Elliot Gorokhovsky
796182652d Pull out software fallbacks 2022-02-14 11:16:03 -05:00
Elliot Gorokhovsky
db2f4a6532 Move bitwise builtins into bits.h 2022-02-14 11:16:03 -05:00
Elliot Gorokhovsky
970460f67d
Merge pull request #3063 from embg/largeNbDicts
Bugfix and new features for largeNbDicts benchmark
2022-02-11 13:55:35 -05:00