sharpetronics/zstd
1
0
Fork 0
mirror of https://github.com/facebook/zstd.git synced 2025-10-15 00:02:02 -04:00
Code Issues Projects Releases Wiki Activity
7,356 Commits 38 Branches 77 Tags
Commit Graph

94 Commits

Author SHA1 Message Date
Nick Terrell
d1cc9d2797
[fuzz] Allow zero sized buffers for streaming fuzzers (#1945) 
* Allow zero sized buffers in `stream_decompress`. Ensure that we never have two
  zero sized buffers in a row so we guarantee forwards progress.
* Make case 4 in `stream_round_trip` do a zero sized buffers call followed by
  a full call to guarantee forwards progress.
* Fix `limitCopy()` in legacy decoders.
* Fix memcpy in `zstdmt_compress.c`.

Catches the bug fixed in PR #1939
 2020-01-09 11:38:50 -08:00
Nick Terrell
b77ad810c9
[fuzz] Fix regression_driver.c with directory input (#1944) 
The `numFiles` variable wasn't updated, so the fuzzer didn't do anything.
I did two things to fix this:

1. Remove the `numFiles` variable entirely.
2. Error if we can't open a file and print the number of files tested.
 2020-01-08 13:20:56 -08:00
Yann Collet
c71bd45a3b Merge branch 'dev' into ahmed_file 2019-11-26 11:20:26 -08:00
Nick Terrell
e68db76b4b Update .gitignore 2019-11-20 16:36:40 -08:00
Yann Collet
aea2ff5d8d fixed wrong assert() in regression driver 2019-11-06 14:56:21 -08:00
Yann Collet
a7e33e3e10 updated fuzz tests to use FileNamesTable* abstraction 2019-11-06 14:42:13 -08:00
Sen Huang
e21a8bbecd Fix FUZZ_rand32() bug 2019-11-05 16:43:24 -05:00
Sen Huang
f2932fb5eb Fix more merge conflicts 2019-11-05 15:54:05 -05:00
Nick Terrell
60205fec02 Fix 2 bugs in dictionary loading 
* Silently skip dictionaries less than 8 bytes, unless using `ZSTD_dct_fullDict`.
  This changes the compressor, which silently skips dictionaries <= 8 bytes.
* Allow repcodes that are equal to the dictionary content size, since it is in bounds.
 2019-11-01 16:52:07 -07:00
Nick Terrell
75e7c0d107 [fuzz] Add dictionary_loader fuzzer 
* Adds the fuzzer
* Adds an additional `InputType` for the fuzzer

I ran the fuzzer for about 10 minutes and it found 2 bugs:

* Catches the original bug without any help
* Catches an additional bug with 8-byte dictionaries
 2019-11-01 15:54:24 -07:00
Nick Terrell
8c11f089a1 [fuzz] Increase output buffer size of stream_round_trip 
Fixes OSS-Fuzz crash.
Credit to OSS-Fuzz
 2019-10-18 13:39:08 -07:00
Nick Terrell
d721fcf3ee [fuzz] Fix leak in block_round_trip 2019-09-13 10:32:38 -07:00
Nick Terrell
7c4578160e [fuzz] Generate seed data up to 256KB 2019-09-12 15:02:01 -07:00
Dario Pavlovic
51e9d29a51 Merge branch 'improvDataGen' of github.com:darxsys/zstd into improvDataGen 2019-09-12 13:11:02 -07:00
Dario Pavlovic
cd8588077e It's time for all of rng seed code to go. Goodbye 2019-09-12 13:10:34 -07:00
Dario Pavlovic
47bb4c6a23
Update tests/fuzz/fuzz_data_producer.h 2019-09-12 12:45:28 -07:00
Dario Pavlovic
92c58c4d5d Use range instead of the generic uint32 method to use less bytes when generating necessary numbers. 2019-09-12 12:40:12 -07:00
Dario Pavlovic
b5b24c2a0d Combining fuzz_data_producer restrict calls into a single function 2019-09-11 10:09:29 -07:00
Dario Pavlovic
23cc2d8510 All tests should give some portion of data to the producer and use the rest. 2019-09-10 16:52:38 -07:00
Dario Pavlovic
0630d084cb [Fuzz] Improve data generation #1723 
Converting the rest of the tests to use the new data producer.
 2019-09-10 16:14:43 -07:00
Dario Pavlovic
ea1ad123da Addressing nits 2019-09-09 16:13:24 -07:00
Dario Pavlovic
3932fcfebc Fixing issues with double usage of data. 2019-09-09 15:39:04 -07:00
Dario Pavlovic
a71bbba7be [Fuzz] Improve data generation #1723 2019-09-09 08:43:22 -07:00
Nick Terrell
d0750a1c9c
Merge pull request #1733 from nmagerko/size-hint 
Add --size-hint=# option
 2019-08-23 10:16:10 -07:00
Nick Terrell
e2030a2c40 [fuzz] Add a DEBUGLOG(3) statement to print file 
Enable it by building with this command:

```
./fuzz.py build all --debug 3
```
 2019-08-22 17:27:15 -07:00
Nick Magerko
493f95c7df Fix merge conflicts 2019-08-22 11:51:41 -07:00
Nick Terrell
3982935aef [fuzz] Improve fuzzer build script and docs 
* Remove the `make libFuzzer` target since it is broken and obsoleted
  by `CC=clang CXX=clang++ ./fuzz.py build all --enable-fuzzer`. The
  new `-fsanitize=fuzzer` is much better because it works with MSAN
  by default.
* Improve the `./fuzz.py gen` command by making the input type explicit
  when creating a new target.
* Update the `README` for `--enable-fuzzer`.

Fixes #1727.
 2019-08-20 16:44:50 -07:00
Nick Magerko
c7a24d7a14 Define ZSTD_SRCSIZEHINT_MIN as 0 2019-08-20 13:06:15 -07:00
Nick Magerko
ea9d35922c Add size-hint to fuzz tests 2019-08-19 15:12:29 -07:00
Nick Terrell
e962f07d19
[fuzz] Add a compression fuzzer with randomly sized output buffer (#1670) 2019-07-02 22:05:07 -07:00
Nick Terrell
6810dd6191 [fuzz] Remove max_len from the options 2019-06-10 11:05:45 -07:00
Nick Terrell
610a81ecf9 [fuzzer] Compile with legacy support 2019-04-18 12:44:55 -07:00
Nick Terrell
cc669006dc [fuzzer] Size the decompression output buffer randomly 2019-04-18 12:44:21 -07:00
Nick Terrell
58bcc328a4 [fuzz] Add a seedcorpora target for oss-fuzz 2019-04-17 12:13:06 -07:00
Nick Terrell
09caa4d800 [fuzzer] Add a fuzzer for frame info functions 
Add a fuzzer that fuzzes all helper functions that take compressed
input. This fuzzer caught one out of bounds read in
`ZSTD_decompressBound()`.
 2019-04-17 11:29:42 -07:00
Josh Soref
a880ca239b Spelling (#1582) 
* spelling: accidentally

* spelling: across

* spelling: additionally

* spelling: addresses

* spelling: appropriate

* spelling: assumed

* spelling: available

* spelling: builder

* spelling: capacity

* spelling: compiler

* spelling: compressibility

* spelling: compressor

* spelling: compression

* spelling: contract

* spelling: convenience

* spelling: decompress

* spelling: description

* spelling: deflate

* spelling: deterministically

* spelling: dictionary

* spelling: display

* spelling: eliminate

* spelling: preemptively

* spelling: exclude

* spelling: failure

* spelling: independence

* spelling: independent

* spelling: intentionally

* spelling: matching

* spelling: maximum

* spelling: meaning

* spelling: mishandled

* spelling: memory

* spelling: occasionally

* spelling: occurrence

* spelling: official

* spelling: offsets

* spelling: original

* spelling: output

* spelling: overflow

* spelling: overridden

* spelling: parameter

* spelling: performance

* spelling: probability

* spelling: receives

* spelling: redundant

* spelling: recompression

* spelling: resources

* spelling: sanity

* spelling: segment

* spelling: series

* spelling: specified

* spelling: specify

* spelling: subtracted

* spelling: successful

* spelling: return

* spelling: translation

* spelling: update

* spelling: unrelated

* spelling: useless

* spelling: variables

* spelling: variety

* spelling: verbatim

* spelling: verification

* spelling: visited

* spelling: warming

* spelling: workers

* spelling: with
 2019-04-12 11:18:11 -07:00
Nick Terrell
c45dec12c5 [fuzzer] Use ZSTD_DCtx_loadDictionary_advanced() half the time 2019-04-09 18:02:22 -07:00
Nick Terrell
10a3d4dca9 [fuzzer] Make the regression_driver work while fuzzers are active 2019-04-09 18:01:49 -07:00
Nick Terrell
c5d70b7dbb [fuzzer] Sometimes fuzz with one less output byte 
Zstd compression sometimes does different stuff when it has at least
`ZSTD_compressBound()` output bytes, or not. Half of the time fuzz with
`ZSTD_compressBound() - 1` output bytes. Ensure that we have at least
one byte of overhead by disabling either the dictionary ID or checksum.
 2019-04-09 16:47:59 -07:00
Nick Terrell
7a1fde2957 [fuzzer] Add dictionary fuzzers 2019-04-08 21:07:28 -07:00
Nick Terrell
462918560c [fuzzer] Fix stream_round_trip for the new options 2019-04-08 21:06:19 -07:00
Nick Terrell
f871b5144e [fuzz] Use the new advanced API 2019-04-08 20:01:38 -07:00
Nick Terrell
4b0024a97d [fuzz] Add --enable-fuzzer for clang fuzzing 2019-02-27 17:15:52 -08:00
Peter (Stig) Edwards
cdb3e7af2f
-Wformat-security not needed with -Wformat=2 2019-02-01 09:38:49 +00:00
Yann Collet
39e28982cf introduced constants ZSTD_STRATEGY_MIN and ZSTD_STRATEGY_MAX 2018-12-06 16:16:16 -08:00
Yann Collet
be9e561da4 changed ZSTD_c_compressionStrategy into ZSTD_c_strategy 
also : fixed paramgrill, and limit conditions
 2018-12-06 15:00:52 -08:00
Yann Collet
3583d19c4e changed parameter names from ZSTD_p_* to ZSTD_c_* 
for naming consistency
 2018-12-05 17:26:02 -08:00
Yann Collet
d8e215cbee created ZSTD_compress2() and ZSTD_compressStream2() 
ZSTD_compress_generic() is renamed ZSTD_compressStream2().

Note that, for the time being,
the "stable" API and advanced one use different parameter planes :
setting parameters using the advanced API does not influence ZSTD_compressStream()
and using ZSTD_initCStream() does not influence parameters for ZSTD_compressStream2().
 2018-11-30 11:25:56 -08:00
Yann Collet
41c7d0b1e1 changed hashEveryLog into hashRateLog 2018-11-21 14:36:57 -08:00
Yann Collet
2e7fd6a2cb fixed remaining searchLength invocations 2018-11-20 15:13:27 -08:00