1965 Commits

Author SHA1 Message Date
Scott Baker
1eb852854b Some fixes to address things @felixhandte found 2021-06-10 12:53:07 -04:00
Scott Baker
376a2730a8 Try enabling the BIG strings now the unsigned long long is in effect 2021-06-10 12:53:07 -04:00
Scott Baker
20b9b00b41 Try unsigned long long 2021-06-10 12:53:07 -04:00
Scott Baker
64385ef7cb Update humanSize() to skip the big numbers (it requires 64 bit) 2021-06-10 12:53:07 -04:00
Scott Baker
1ef6f3d079 Use unsigned long instead to help with some tests 2021-06-10 12:53:07 -04:00
Scott Baker
e5fc830795 human_size() should use size_t 2021-06-10 12:53:07 -04:00
Scott Baker
35576e63ce Convert tabs to spaces 2021-06-10 12:53:07 -04:00
Scott Baker
77001f00fb Use human_size() on the "multiple files compressed" output also 2021-06-10 12:53:07 -04:00
Scott Baker
894698d3b6 Use human_size() in the benchmark output also 2021-06-10 12:53:07 -04:00
Scott Baker
4e0d9f1cc8 Move the variable declarations to the top 2021-06-10 12:53:07 -04:00
Scott Baker
eefdbcd93a Make the variable types match 2021-06-10 12:53:07 -04:00
Scott Baker
b6b23dfe64 Convert names to CamelCase 2021-06-10 12:53:07 -04:00
Scott Baker
b70175e5ec Put the human_size() function in util.c 2021-06-10 12:53:07 -04:00
Scott Baker
26fab1d963 Make the CLI output the file sizes in human readable format 2021-06-10 12:53:07 -04:00
binhdvo
325952f878
Revert "Add support for --long-param flag, fix #2104" 2021-06-09 15:35:43 -04:00
Binh Vo
6583fa3f0a Add support for --long-param flag 2021-06-09 14:07:52 -04:00
Binh Vo
d2f31b6627 Fix --progress flag to properly control progress display and default progress display on when using -v 2021-06-08 17:24:38 -04:00
Binh Vo
1e17184ad0 Add documentation for --patch-from 2021-06-03 11:12:27 -04:00
Yann Collet
8fae35591e Merge branch 'dev' of github.com:facebook/zstd into dev 2021-05-12 13:12:30 -07:00
Olivier Perret
d4548c96cb
fileio: clamp value of windowLog in patch-mode (#2637)
With small enough input files, the inferred value of fileWindowLog could
be smaller than ZSTD_WINDOWLOG_MIN.

This can be reproduced like so:
$ echo abc > small
$ echo abcdef > small2
$ zstd --patch-from small small2 -o patch
previously, this would fail with the error "zstd: error 11 : Parameter is out of bound"
2021-05-12 16:11:15 -04:00
Yann Collet
cb0cad9b79 reduce Max nb Workers to 64 in 32-bit mode
and restored limit to 256 when in 64-bit mode
(it was reduced to 200 to give more room for 32-bit).

This should fix test instability issues
using lot of threads in 32-bit environments.
2021-05-12 13:10:25 -07:00
Yann Collet
8a53a882f2
updated generated man pages for v1.5.0 (#2635) 2021-05-11 18:17:31 -04:00
sen
13449d7ce1
Add PHONY targets to makefiles (#2629) 2021-05-07 14:03:19 -04:00
sen
6030cdfede
Add --progress flag (#2595) 2021-05-06 14:50:28 -04:00
Yann Collet
2f7bbd6539
Merge pull request #2620 from facebook/winFilelist
fix --filelist compatibility with Windows cr+lf line ending
2021-05-06 11:35:16 -07:00
Yann Collet
df05b2ba7c fix --filelist compatibility with Windows cr+lf line ending 2021-05-05 18:01:55 -07:00
Yann Collet
9750f3c87b improved benchmark experience on Windows
benchmark results are not progressively displayed on Windows terminal.
For long benchmark sessions, nothing is displayed,
until the end, where everything is flushed.

Force display to be flushed after each update.
Updates happen roughtly every second, or even less,
so it's not a substantial workload.
2021-05-05 16:52:21 -07:00
W. Felix Handte
4f9c6fdb7f Attempt to Fix Windows Build Error 2021-05-05 13:13:56 -04:00
W. Felix Handte
da61918c75 Also Pass Mode Bits in on Windows
I think in some unix emulation environments on Windows, (cygwin?) mode bits
are somehow respected. So we might as well pass them in. Can't hurt.
2021-05-05 13:10:34 -04:00
W. Felix Handte
45c4918ccf Fix Build for Windows 2021-05-05 13:10:34 -04:00
W. Felix Handte
1fb10ba831 Don't Block Removing File on Being Able to Read It
`open()`'s mode bits are only applied to files that are created by the call.
If the output file already exists, but is not readable, the `fopen()` would
fail, preventing us from removing it, which would mean that the file would
not end up with the correct permission bits.

It's not clear to me why the `fopen()` is there at all. `UTIL_isRegularFile()`
should be sufficient, AFAICT.
2021-05-05 13:10:34 -04:00
W. Felix Handte
b87f97b3ea Create Files with Desired Permissions; Avoid chmod(); Remove UTIL_chmod() 2021-05-05 13:10:34 -04:00
Felix Handte
2d10544b84
Merge pull request #2613 from felixhandte/allow-block-device
Allow Reading from Block Devices with `--force`
2021-05-05 13:06:32 -04:00
Yann Collet
455fd1a067 updated documentation regarding minimum job size 2021-05-05 09:03:11 -07:00
W. Felix Handte
33f3e293e8 Allow Reading from Block Devices with --force 2021-05-04 16:25:26 -04:00
Nick Terrell
a8ecf4ff88
Merge pull request #2597 from terrelln/public-headers
[1.5.0] Move `zstd_errors.h` and `zdict.h` to `lib/` root
2021-05-04 11:28:41 -07:00
W. Felix Handte
ee122baacf Detect Presence of md5 on Darwin
This fixes #2568.
2021-05-04 12:33:19 -04:00
Nick Terrell
09149beaf8 [1.5.0] Move zstd_errors.h and zdict.h to lib/ root
`zstd_errors.h` and `zdict.h` are public headers, so they deserve to be
in the root `lib/` directory with `zstd.h`, not mixed in with our private
headers.
2021-04-30 15:13:54 -07:00
Nick Terrell
4694423c4f Add and integrate lazy row hash strategy 2021-04-07 09:53:34 -07:00
Niclas Rosenvik
e7647180cd Stop complaining about hash tool not found
If build_dir is set the zstd build complains about md5sum not being found.
Fix this by checking if build_dir is set before checking and using the hash tool
just like in lib/Makefile .
2021-04-02 13:00:19 -07:00
Nick Terrell
a494308ae9 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
2021-03-30 10:30:43 -07:00
Sen Huang
f27e326456 Restrict dictmode regression tests only to advanced API, fix some compiler warnings 2021-03-25 10:39:08 -07:00
Yann Collet
0f99a0d987 fix man page typo
fix #2553 detected by @jwilk
2021-03-24 05:55:04 -07:00
Yann Collet
9fb4a42c7b fix #2549 2021-03-20 17:29:41 -07:00
Paul Bone
4d6c78fb89 Only set numPhysicalCores if ratio is valid 2021-03-03 10:59:00 +11:00
Paul Bone
eb1a09df61 If cpuinfo parsing fails fallback to sysconf 2021-03-03 10:58:51 +11:00
W. Felix Handte
61db590ad8 Detect .. in Paths Correctly
This commit addresses #2509.
2021-02-26 12:29:42 -05:00
Yann Collet
61b63e9060
Merge pull request #2492 from niacat/dev
Use standard md5 tool on NetBSD.
2021-02-24 16:38:10 -08:00
Stephen Kitt
adb54293ab
Stop using deprecated reset?Stream functions
These are replaced by the corresponding context resets. When
converting resetCStream, CCtx_setPledgedSrcSize isn't called if the
source size is "unknown".

This helps reduce the reliance on "static only" symbols, as well as
reducing the use of deprecated functions.

Signed-off-by: Stephen Kitt <steve@sk2.org>
2021-02-23 21:56:01 +01:00
Felix Handte
a2adc6df9f
Merge pull request #2495 from felixhandte/umask
Use umask() to Constrain Created File Permissions
2021-02-17 17:03:23 -05:00