9714 Commits

Author SHA1 Message Date
ctkhanhly
358787764f
Fix for zstd CLI accepts bogus values for numeric parameters (#3268)
* add checks to mal-formed numeric values for memory and memlimit parameters

Signed-off-by: Ly Cao <lycao@fb.com>

* changed errorMsg to a literal string instead of static string in main

* moved bogus numeric error to NEXT_UINT32 + add macro NEXT_TSIZE

Signed-off-by: Ly Cao <lycao@fb.com>

Signed-off-by: Ly Cao <lycao@fb.com>
Co-authored-by: Ly Cao <lycao@fb.com>
2022-09-21 13:20:01 -07:00
Yann Collet
1c04514b31
Merge pull request #3252 from Tachi107/build-cmake-pkg-config-always
build(cmake): improve pkg-config generation
2022-09-20 11:31:15 -07:00
Yann Collet
369d63b7cb
Merge pull request #3265 from DimitriPapadopoulos/actions
Update GitHub Actions
2022-09-20 11:30:26 -07:00
Dimitri Papadopoulos
b1b1e3aa53
Update GitHub Actions 2022-09-20 09:17:29 +02:00
Yann Collet
97c23cf615
Merge pull request #3199 from JunHe77/comp
compress:check more bytes to reduce ZSTD_count call
2022-09-19 10:49:10 -07:00
Yann Collet
e9e88753d5
Merge pull request #3245 from haampie/fix/SED_ERE_OPT
drop -E flag in sed
2022-09-19 10:48:11 -07:00
Yann Collet
f7251f88b9
Merge pull request #3247 from haampie/fix/grep
Fix make variable
2022-09-19 10:47:38 -07:00
Yann Collet
ad13c4aec9
Merge pull request #3264 from sashashura/patch-1
GitHub Workflows security hardening
2022-09-19 10:47:00 -07:00
Alex
091917a4a1 build: harden GitHub Workflow permissions
Signed-off-by: Alex Low <aleksandrosansan@gmail.com>
2022-09-19 17:51:59 +02:00
Jun He
ce52acd7dc compress:check more bytes to reduce ZSTD_count call
Comparing 4B instead of comparing 1B in ZSTD_noDict
mode, thus it can avoid cases like match in match[ml]
but mismatch in match[ml-3]..match[ml-1]. So the call
count of ZSTD_count can be reduced.

Signed-off-by: Jun He <jun.he@arm.com>
Change-Id: I3449ea423d5c8e8344f75341f19a2d1643c703f6
2022-09-18 14:45:41 +08:00
daniellerozenblit
cb6dc32635
Merge pull request #3263 from daniellerozenblit/null-buffer-decompress
ZSTD_decompressStream() fuzz fix
2022-09-13 10:07:35 -04:00
Danielle Rozenblit
8bb833bb5a Merge branch 'null-buffer-decompress' of github.com:daniellerozenblit/zstd into null-buffer-decompress 2022-09-12 18:57:53 -07:00
Danielle Rozenblit
e46b12e1b4 fix indentation 2022-09-12 18:56:59 -07:00
Yann Collet
484b9d67c6
Merge pull request #3256 from facebook/clean
streamline `make clean` list maintenance
2022-09-12 13:23:34 -07:00
Yann Collet
5129b4ab10 simplify clean target maintenance within programs/ 2022-09-12 12:19:35 -07:00
daniellerozenblit
f59f797aa8
Merge branch 'facebook:dev' into null-buffer-decompress 2022-09-12 14:54:36 -04:00
Danielle Rozenblit
a1d89424c2 fuzzer error fix 2022-09-12 11:53:37 -07:00
Yann Collet
361b7bd1d0 Merge branch 'dev' into clean 2022-09-12 10:56:00 -07:00
daniellerozenblit
74ece5037c
Merge pull request #3262 from daniellerozenblit/sequence-bound
Add sequenceBound(srcSize) method
2022-09-09 17:40:42 -04:00
Danielle Rozenblit
1613caf8bd use ZSTD_sequenceBound in seqBench 2022-09-09 13:04:41 -07:00
daniellerozenblit
88b49e923b
Merge branch 'facebook:dev' into sequence-bound 2022-09-09 15:57:05 -04:00
Elliot Gorokhovsky
39ab02a71f
Merge pull request #3257 from embg/seqBench2
Benchmark program for sequence compression API
2022-09-09 15:53:28 -04:00
Danielle Rozenblit
aa82998821 add sequence bound function 2022-09-09 12:34:25 -07:00
Elliot Gorokhovsky
6600a05949
Merge pull request #3259 from DimitriPapadopoulos/codespell
Fix typos found by codespell
2022-09-09 11:15:05 -04:00
daniellerozenblit
c26aa5d0fa
Merge pull request #3258 from daniellerozenblit/null-buffer-decompress
Fix undefined behavior in ZSTD_decompressStream()
2022-09-09 09:39:59 -04:00
Danielle Rozenblit
a06e953db9 some additional comments, remove apt-get from clang jobs, better test titles 2022-09-08 18:30:07 -07:00
Dimitri Papadopoulos
0015308c0f
Fix typos found by codespell 2022-09-08 23:17:00 +02:00
Danielle Rozenblit
3d7f9a90df skip flush operation in case where op is NULL 2022-09-08 13:53:13 -07:00
Danielle Rozenblit
f3ddaaddd6 ternary operator instead of if statement 2022-09-08 12:59:49 -07:00
Danielle Rozenblit
cf255cc5e0 remove 32-bit ubsan clang test (bug in clang that produces an error) 2022-09-08 12:04:57 -07:00
Elliot Gorokhovsky
61c79bf0d6 Benchmark program for sequence compression API 2022-09-08 09:20:50 -07:00
Danielle Rozenblit
66ed3df096 add clang jobs for ubsan in github workflow 2022-09-08 09:00:39 -07:00
Danielle Rozenblit
028842788b fix zero offset to nullpointer errors 2022-09-07 17:52:26 -07:00
Yann Collet
c0b46738b4 streamline make clean list maintenance
When creating a new `Makefile` target to build,
it's also necessary to update the `clean` target,
which purpose is to remove built targets when they are present.

This process is simple, but it's also easy to forget :
since there is a large distance between the position in the `Makefile` where the new built target is added,
and the place where the list of files to `clean` is defined.
Moreover, the list of files becomes pretty long over time,
hence it's difficult to visually ensure that all built targets are present there,
or that no old target (no longer produced) is no longer in the list

This PR tries to improve this process by adding a CLEAN variable.
Now, when a new built target is added to the `Makefile`,
it should preceded by :
```
CLEAN += newTarget
newTarget:
<TAB> ...recipe...
```

This new requirement is somewhat similar to `.PHONY: newTarget` for non-built targets.

This new method offers the advantage of locality :
there is no separate place in the file to maintain a list of files to clean.
This makes maintenance of `make clean` easier.
2022-09-07 16:36:03 -07:00
Danielle Rozenblit
fe22e8c538 revert change 2022-09-07 12:37:25 -07:00
Danielle Rozenblit
ca78d101f7 do not recover pointer overflow for testing 2022-09-07 12:21:54 -07:00
Danielle Rozenblit
bc7492cefa added zstreamtest_ubsan to make file + added ubsan zstreamtest job for CI tests 2022-09-07 11:54:17 -07:00
Danielle Rozenblit
282a955d33 added test that exposes zero offset to null pointer error when built with clang 2022-09-07 08:58:08 -07:00
Danielle Rozenblit
4a6783bbbb remove zstream_ubsan from git and add to gitignore 2022-09-07 07:40:43 -07:00
Danielle Rozenblit
69022ad886 null decompress buffer test and ubsan flag added 2022-09-06 14:34:55 -07:00
Andrea Pappacoda
a0b09d0ff7
build(cmake): improve pkg-config generation
With this patch the pkg-config generation when using the CMake build
system is improved in the following ways:

- Libs.private is now filled when needed
- The JoinPaths module is now used to join paths, leading to simpler
  code
- The .pc file is always generated, regardless of the platform, as it
  can also be consumed on Windows

Here's how the .pc file is affected by these changes, in comparison to
the one generated with the official Makefiles:

    $ diff -s lib/libzstd.pc build/cmake/build-old/lib/libzstd.pc
    15c15
    < Libs.private: -pthread
    ---
    > Libs.private:

    $ diff -s lib/libzstd.pc build/cmake/build-new/lib/libzstd.pc
    Files lib/libzstd.pc and build/cmake/build-new/lib/libzstd.pc are
    identical
2022-08-28 13:24:30 +02:00
Harmen Stoppels
efef80b75e Fix make variable 2022-08-19 12:06:43 +02:00
Harmen Stoppels
ae5f273a92 drop -E flag in sed 2022-08-19 12:00:32 +02:00
Yann Collet
155d6a58a2
Merge pull request #3241 from wahern/wahern-combine-sh-faster
restore combine.sh bash performance while still sticking to POSIX
2022-08-16 16:09:43 -07:00
Elliot Gorokhovsky
ef60302af9
Merge pull request #3230 from grossws/fix3229-docs
Add description for ZSTD_decompressStream and ZSTD_initDStream
2022-08-16 12:48:23 -04:00
Chris Burgess
3b4e47092e
Document pass-through behavior (#3242)
Adds documentation to help and man pages for legacy pass-through behavior
when force is set and destination is stdout. Documents --pass-through in
man pages
2022-08-15 10:29:54 -07:00
William Ahern
60fcc36af6 escape glob pattern special characters in subject string before generating search patterns in combine.sh list_has_item 2022-08-10 20:58:55 -07:00
William Ahern
cca3544708 restore combine.sh bash performance while still sticking to POSIX 2022-08-10 16:51:17 -07:00
Yann Collet
1b249cf075
Merge pull request #3235 from facebook/docTraining
[easy] added a few documentation words about dictionary training
2022-08-08 12:06:49 -07:00
Konstantin Gribov
1c847e2e32 Add description for ZSTD_decompressStream and ZSTD_initDStream
With that these functions become visible in generated docs.

Fixes #3229
2022-08-08 18:02:50 +03:00