sharpetronics/zstd
1
0
Fork 0
mirror of https://github.com/facebook/zstd.git synced 2025-11-22 00:10:22 -05:00
Code Issues Projects Releases Wiki Activity
9,040 Commits 37 Branches 77 Tags
Commit Graph

9040 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sen Huang
c5869677d9 Moved ZSTD_defaultCLevel() into stable API 2021-04-16 10:15:40 -07:00
Sen Huang
9c1ca3c00b Moved ZSTD_getDictID_fromCDict() into stable API 2021-04-16 10:14:29 -07:00
sen
12c045f74d
Merge pull request #2574 from senhuang42/repcode_mismatch_detector_fix 
Correct the block splitter mismatched repcodes detection.
 2021-04-12 23:27:43 -04:00
sen
ebd41ebe56
Merge pull request #2572 from senhuang42/row_hash_hashcache_one_off_error_fix 
Adjust nb elements to prefetch in ZSTD_row_fillHashCache()
 2021-04-12 15:38:53 -04:00
Sen Huang
8844f93957 Adjust nb elements to prefetch in ZSTD_row_fillHashCache() 2021-04-12 14:24:58 -04:00
Sen Huang
550f76f131 Correct the detection of mismatched repcodes 2021-04-09 09:08:51 -07:00
sen
56421f34e4
Merge pull request #2494 from senhuang42/row_hash2 
SIMD Row Based Matchfinder 🚀
 2021-04-08 10:55:14 -04:00
Sen Huang
4d63d6e8aa Update results.csv, add Row hash to regression test 2021-04-07 10:31:41 -07:00
Nick Terrell
4694423c4f Add and integrate lazy row hash strategy 2021-04-07 09:53:34 -07:00
sen
f71aabb5b5
Move clevel override to after initLocalDict() (#2571) 2021-04-06 21:05:37 -04:00
sen
f1e8b565c2
Maintain two repcode histories for block splitting, replace invalid repcodes (#2569) 2021-04-06 17:25:55 -04:00
sen
e38124555e
Fix dictionary force reloading clevel selection (#2570) 
* Move cdict clevel override to before localdict init

* Update results.csv after dict load changes
 2021-04-06 15:35:09 -04:00
Nick Terrell
8383fc828d
Merge pull request #2541 from ihsinme/patch-1 
simple fix for using bit operator.
 2021-04-02 13:01:09 -07:00
Niclas Rosenvik
e7647180cd Stop complaining about hash tool not found 
If build_dir is set the zstd build complains about md5sum not being found.
Fix this by checking if build_dir is set before checking and using the hash tool
just like in lib/Makefile .
 2021-04-02 13:00:19 -07:00
sen
980f3bbf83
[cwksp] Align all allocated "tables" and "aligneds" to 64 bytes (#2546) 
* Perform 64-byte alignment of wksp tables and aligneds internally

* Clean up cwskp_finalize() function to only do two allocs

* Refactor aligned/buffer reservation code, remove ASAN req for alignment reservations

* Change from allocating 128 bytes always to allocating only buffer space as needed for tables/aligned

* Back out aligned/table reservation order restriction

* Add stricter bounds for new/resized wksps, fix comment in zstd_cwksp.h
 2021-04-01 20:07:19 -04:00
sen
255925c231
Fix repcode-related OSS-fuzz issues in block splitter (#2560) 
* Do not emit last partitions of blocks as RLE/uncompressed

* Fix repcode updates within block splitter

* Add a entropytables confirm function, redo ZSTD_confirmRepcodesAndEntropyTables() for better function signature

* Add a repcode updater to block splitter, no longer need to force emit compressed blocks
 2021-03-31 15:14:59 -04:00
Nick Terrell
d334ad2ff4 [contrib][linux-kernel] Add zstd_min_clevel() and zstd_max_clevel() 2021-03-30 10:30:59 -07:00
Nick Terrell
a494308ae9 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files 
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
 2021-03-30 10:30:43 -07:00
sen
84ccb81e7c
Merge pull request #2561 from senhuang42/longlength_enum 
Add enum for representing long length ID
 2021-03-26 15:55:12 -04:00
Sen Huang
b1a43455f8 Add enum for representing long length ID 2021-03-26 10:41:09 -07:00
sen
ab216bc2c5
Merge pull request #2559 from senhuang42/add_dict_regression_tests_backup 
Add different dict modes to compression ratio regression test, update results.csv
 2021-03-25 19:26:06 -04:00
Sen Huang
bbbd578f45 Update results.csv 2021-03-25 11:16:37 -07:00
sen
4fe2e7ae14
Merge pull request #2558 from senhuang42/msan_block_splitter_fix 
Fix block splitter minor MSAN warning.
 2021-03-25 13:51:43 -04:00
Sen Huang
f27e326456 Restrict dictmode regression tests only to advanced API, fix some compiler warnings 2021-03-25 10:39:08 -07:00
Sen Huang
1cadf86b39 Add tests to regression tests for dict 2021-03-25 10:39:08 -07:00
sen
b0407b9f0e
Merge pull request #2555 from senhuang42/default_clevel_func 
Add ZSTD_defaultCLevel() function to public API
 2021-03-25 13:07:28 -04:00
Sen Huang
ef4e26bda5 Add clang msan fuzz test to github actions 2021-03-25 09:11:43 -07:00
Sen Huang
2a907bf4aa Move lastCountSize into a returned struct, fix MSAN error 2021-03-25 09:11:15 -07:00
Sen Huang
e398744a35 Add ZSTD_defaultCLevel() function to public API 2021-03-25 08:04:00 -07:00
Nick Terrell
f8ac0ea7ef
Merge pull request #2539 from terrelln/linux-kernel-fixes 
Fixes for the next linux kernel patch version
 2021-03-24 10:34:29 -07:00
sen
bf542c8a8d
Merge pull request #2447 from senhuang42/block_splitter_v2 
Recursive block splitting
 2021-03-24 12:27:22 -04:00
Yann Collet
8889ca36fc
Merge pull request #2554 from facebook/fix2553 
fix man page typo
 2021-03-24 08:55:33 -07:00
Sen Huang
5b566ebe08 Rename *compressSequences*() functions for clarity 2021-03-24 08:21:29 -07:00
Sen Huang
0ef1f935b7 Add a fallback in case the total blocksize of split blocks exceeds raw block size 2021-03-24 08:21:29 -07:00
Sen Huang
c90e81a692 Enable block splitter by default when applicable 2021-03-24 08:21:29 -07:00
Sen Huang
e34332834a Clean up various functions, add debuglogging for estimate vs. actual sizes 2021-03-24 08:21:29 -07:00
Sen Huang
41c3eae6d9 Fix various fuzzer failures: repcode history, superblocks 2021-03-24 08:21:29 -07:00
senhuang42
0633bf17c3 Change 1.3.4 bugfix to be cross-compatible with superblocks and normal compression 2021-03-24 08:21:29 -07:00
senhuang42
eb1ee8686d Refactor buildSequencesStatistics() to avoid pointer increment for superblocks 2021-03-24 08:21:29 -07:00
senhuang42
e2bb215117 Add unit tests and fuzzer param 2021-03-24 08:21:09 -07:00
senhuang42
de52de1347 Add recursive block split algorithm 2021-03-24 08:21:09 -07:00
senhuang42
f06f6626ed Update function names for consistency 2021-03-24 08:20:54 -07:00
senhuang42
c56d6e49e8 Add block splitter to experimental params 2021-03-24 08:20:54 -07:00
senhuang42
2949a95224 Refactor block compression logic into single function 2021-03-24 08:20:54 -07:00
senhuang42
c05c090cc2 Centralize entropy statistics calculations to zstd_compress.c 2021-03-24 08:20:29 -07:00
Yann Collet
0f99a0d987 fix man page typo 
fix #2553 detected by @jwilk
 2021-03-24 05:55:04 -07:00
sen
c48889f097
Merge pull request #2538 from senhuang42/monotonicity_test 
Add memory monotonicity test over srcSize
 2021-03-22 16:54:34 -04:00
Nick Terrell
ebc2dfa821
Merge pull request #2524 from terrelln/huf-stack-reduction 
[huf] Reduce stack usage of HUF_readDTableX2 by ~972 bytes
 2021-03-22 12:37:54 -07:00
Nick Terrell
634bfd339f [FSE] Clean up workspace using dynamically sized struct 2021-03-22 11:07:07 -07:00
Yann Collet
8b97931ae1
Merge pull request #2550 from facebook/vvv_srcSize 
fix #2549
 2021-03-22 10:44:44 -07:00