10540 Commits

Author SHA1 Message Date
Nick Terrell
ff0afbad58 [asm][aarch64] Mark that BTI and PAC are supported
Mark that `huf_decompress_amd64.S` supports BTI and PAC, which it trivially does because it is empty for aarch64.

The issue only requested BTI markings, but it also makes sense to mark PAC, which is the only other feature.

Also run add a test for this mode to the ARM64 QEMU test. Before this PR it warns on `huf_decompress_amd64.S`, after it doesn't.

Fixes Issue #3841.
2024-03-13 16:15:51 -04:00
Yann Collet
7c2aa73a8e
Merge pull request #3958 from facebook/doc3698
updated documentation
2024-03-13 07:15:45 -07:00
Yann Collet
084b362775
Merge pull request #3956 from facebook/fix3702
Do not truncate file name in verbose mode
2024-03-13 07:15:06 -07:00
Nick Terrell
c1e995321e [CI] Run tests with CMake on Windows
Build and run tests on Windows with CMake
2024-03-13 09:45:40 -04:00
Nick Terrell
94c102038b [cpu] Backport fix for rbx clobbering on Windows with Clang
Backport folly fix for rbx clobbering: f22f88b8b9

This supercedes PR #3646.
2024-03-13 09:45:40 -04:00
Yann Collet
4bd911ae41
Merge pull request #3954 from facebook/zeroSeq_sample
Add the zeroSeq sample
2024-03-12 15:31:01 -07:00
Yonatan Komornik
b20703f273
Updates ZSTD_RowFindBestMatch comment (#3947)
Updates the comment on the head of `ZSTD_RowFindBestMatch` to make sure it's aligned with recent changes to the hash table.
2024-03-12 15:10:07 -07:00
Elliot Gorokhovsky
f65b9e27ce
Exercise ZSTD_findDecompressedSize() in the simple decompression fuzzer (#3959)
* Improve decompression fuzzer

* Fix legacy frame header fuzzer crash, add unit test
2024-03-12 17:07:06 -04:00
Yann Collet
ff6713fd72 fix duplicated paragraph
reported by @zougloub
2024-03-12 13:09:10 -07:00
Yann Collet
5473b72a05 updated documentation
following recommendations by @zougloub at #3698
2024-03-12 12:27:33 -07:00
Elliot Gorokhovsky
92fbd42894
Export ZSTD_LEGACY_SUPPORT in tests/Makefile (#3955)
This doesn't affect most of the targets, but will help me sleep better at night knowing that future refactors won't break the legacy support.

Should have been included in https://github.com/facebook/zstd/pull/3943 but I noticed after that merged, so putting up a separate PR.
2024-03-12 14:36:54 -04:00
Yann Collet
83ec3d0164 no longer truncate file name in verbose mode
fix #3702
2024-03-12 11:27:42 -07:00
daniellerozenblit
5a66afa051
Add common file extensions to --exclude-compressed (#3951) 2024-03-12 13:49:06 -04:00
Yann Collet
37ff4f91eb removed golden-decompression/.gitignore
replaced by an exclusion rule in tests/.gitignore
2024-03-12 10:47:27 -07:00
Yann Collet
e27e156d3c
Merge pull request #3901 from robandpdx-org/convert-facebook-zstd-to-actions
Convert the CircleCI workflow to a GitHub Actions workflow
2024-03-12 09:54:40 -07:00
Felix Handte
a6ff1194d8
Merge pull request #3952 from felixhandte/stop-hardcoding-posix-version-on-bsd
Stop Hardcoding the POSIX Version on BSDs
2024-03-12 12:49:19 -04:00
Yann Collet
0ae98ba215 add same .gitignore rule in golden-decompression/
as requested by @embg
2024-03-12 09:47:54 -07:00
Yann Collet
db996d253e
Merge pull request #3933 from facebook/fix3819
prevent XXH64 from being autovectorized by XXH512 by default
2024-03-12 09:46:48 -07:00
Yann Collet
7a43609ffd
Merge pull request #3934 from facebook/risc5ci
add RISC-V emulation tests to Github CI
2024-03-12 09:46:20 -07:00
Yann Collet
3b573ba1f0
Merge pull request #3936 from facebook/debug5
minor: fix incorrect debug level
2024-03-12 09:45:20 -07:00
Yann Collet
8688c6ed92
Merge pull request #3942 from facebook/fix3719
Fix #3719 : mixing -c, -o and --rm
2024-03-12 09:44:52 -07:00
Yann Collet
4eea5af595
Merge pull request #3946 from facebook/cmake_hpux
cmake refactor: move HP-UX specific logic into its own function
2024-03-12 09:44:20 -07:00
Yann Collet
ca0fc763e3
Merge pull request #3948 from facebook/tests_2927
add tests inspired from #2927
2024-03-12 09:43:59 -07:00
Yann Collet
b39c76765b Add the zeroSeq sample
that should have been part of #3674
2024-03-12 09:28:25 -07:00
Yann Collet
fe7c645a7e
Merge pull request #3937 from facebook/offset0
new method to deal with offset==0 erroneous edge case
2024-03-12 09:25:06 -07:00
acceptacross
8ba5bc4729
chore: fix some typos (#3949)
Signed-off-by: acceptacross <csqcqs@gmail.com>
2024-03-12 11:44:42 -04:00
Elliot Gorokhovsky
ee6acaf26b
Pin tsan and msan CI jobs to ubuntu-20.04 (#3945) 2024-03-12 11:25:00 -04:00
Elliot Gorokhovsky
e0872806df
Use ZSTD_LEGACY_SUPPORT=5 in make test (#3943) 2024-03-12 10:08:26 -04:00
Yann Collet
74e856a195 add tests inspired from #2927
centered around -T# and --fast=# arguments
2024-03-11 17:57:57 -07:00
Yonatan Komornik
edab9eed66
Fix AsyncIO reading seed queueing (#3940)
Fixes a bug in AsyncIO where we queue reads after opening a file so our queue will always be saturated (or as saturated as possible).
Previous code was looping up to `availableJobsCount` not realizing `availableJobsCount` was also decreasing in each iteration, so instead of queueing 10 jobs we'd queue 5 (and instead of 2 we'd queue 1).
This PR fixes the loop to queue as long as `availableJobsCount` is not 0.
2024-03-11 16:28:32 -07:00
Yann Collet
f6039f3d5f cmake refactor: move HP-UX specific logic into its own function
reduce visual clutter of corner case configuration.
2024-03-11 16:22:13 -07:00
Yann Collet
a4db145900
Merge pull request #3862 from likema/fix-hpux-parisc
Fix building on HP-UX 11.11 PA-RISC
2024-03-11 15:55:58 -07:00
W. Felix Handte
f99a450ca4 Stop Hardcoding the POSIX Version on BSDs
BSDs should all have a `unistd.h` header.
2024-03-11 17:25:51 -04:00
Yann Collet
1362699e87 minor man page clarification 2024-03-11 12:23:37 -07:00
Yann Collet
fbd9e628ae added tests 2024-03-11 12:17:34 -07:00
Yann Collet
c610a01d7d fix #3719
only disable `--rm` at end of command line parsing,
so that `-c` only disables `--rm` if it's effectively selected,
and not if it's overriden by a later `-o FILE` command.
2024-03-11 11:38:55 -07:00
Yann Collet
372fddf4e6
Merge pull request #3860 from likema/fix-xxhash-aix-51
Fix building xxhash on AIX 5.1
2024-03-09 15:34:02 -08:00
Yann Collet
cb596b024f
Merge pull request #3929 from facebook/llu_vscode
fix LLU->ULL
2024-03-09 14:24:57 -08:00
Yann Collet
c1875a7386
Merge pull request #3928 from facebook/doc3727
update -V documentation
2024-03-09 14:24:38 -08:00
Yann Collet
eb5f7a7fa2 produced golden sample for the offset==0 decoder test
is correctly detected as corrupted by new version,
and is accepted (changed into offset==1) by older version.

updated documentation accordingly, with an hexadecimal representation.
2024-03-09 00:33:44 -08:00
Yann Collet
d2f56ba442 update documentation 2024-03-08 15:55:30 -08:00
Yann Collet
a9fb8d4c41 new method to deal with offset==0
in this new method, when an `offset==0` is detected,
it's converted into (size_t)(-1), instead of 1.

The logic is that (size_t)(-1) is effectively an extremely large positive number,
which will not pass the offset distance test at next stage (`execSequence()`).
Checked the source code, and offset is always checked (as it should),
using a formula which is not vulnerable to arithmetic overflow:
```
RETURN_ERROR_IF(sequence.offset > (size_t)(oLitEnd - virtualStart),
```

The benefit is that such a case (offset==0) is always detected as corrupted data
as opposed to relying on the checksum to detect the error.
2024-03-08 15:26:06 -08:00
Yann Collet
e127139ceb
Merge pull request #3824 from elasota/specify-zero-offset
Specify offset 0 as invalid and specify required fixup behavior
2024-03-08 15:25:48 -08:00
Yann Collet
478e5fedf9
Merge pull request #3816 from elasota/fix-state-table
Fix state table formatting
2024-03-08 15:02:00 -08:00
Yann Collet
aed172a8fe minor: fix incorrect debug level 2024-03-08 14:29:44 -08:00
Yann Collet
ad590275b4 added RISC-V emulation tests on Github CI 2024-03-07 16:54:44 -08:00
Yann Collet
007cda88ca prevent XXH64 from being autovectorized by XXH512 by default
backport fix https://github.com/Cyan4973/xxHash/pull/924 from libxxhash
2024-03-07 16:43:13 -08:00
Yann Collet
8689633fdf
Merge pull request #3840 from aimuz/fix-reserved
lib/decompress: check for reserved bit corruption in zstd
2024-03-05 13:40:12 -08:00
Yann Collet
2abe8d63e0 fix LLU->ULL
LLU is a correct prefix according to C99 & C11 standards (but not C90).
However, older versions of Visual Studio do not work with it.
Replace by ULL, which doesn't have this issue.

Fixes https://github.com/facebook/zstd/issues/3647
2024-03-04 00:16:01 -08:00
Yann Collet
4fb0a77314 update -V documentation
to answer #3727 comment
2024-03-03 23:24:40 -08:00