Commit graph

74 commits

Author SHA1 Message Date
Josh Holmer fe585ac24c
Use std::available_parallelism instead of num_cpus (#633)
* Explicitly set minimum rust version

* Use std::available_parallelism instead of num_cpus
2022-05-21 08:55:43 +00:00
Josh Holmer 3003f646c9
Zones fixes (#624)
* Fix formatting of zones help text

* Fix issue where wrong number of frames are read in scenechange with zones active

* Fix scenechange progress bar when zones active
2022-05-01 03:35:59 +00:00
Josh Holmer ec9331283c
Improvements to grain synth gamma handling (#622)
- Fixes BT.1886 to use the more correct formula and gamma
- Simplify the `mid_tone` function
- Add quickcheck tests around `to_linear`/`from_linear`
2022-04-26 01:23:37 +00:00
Josh Holmer 02ffc86601
Use published ffmpeg-next (#609) 2022-04-10 16:32:03 +00:00
Josh Holmer 84d46bb40c
Add zones support (#598)
* Add zones support

Currently implemented:
- Overriding the selected encoder & number of passes
- Overriding or adding video params
- Overriding photon noise setting
- Overriding min/max scene length

Closes #267

* Error if zoned encoder does not support output pixel format

* Fix crash if zones change number of passes

* Set passes to 1 for zones with rt mode
2022-03-28 15:23:08 +00:00
redzic d9d15a90d0
Use git version of console-rs to avoid regex (#600) 2022-03-25 21:34:07 +00:00
Josh Holmer d480428fc6
Fix a memory leak in standard scene check method (#593) 2022-03-15 12:38:53 +00:00
redzic b0bb4687ee
Update dependencies (#589) 2022-03-12 13:37:29 +00:00
redzic 6634bc77f8
Update to ffmpeg 5.0 (#575)
* Update to ffmpeg 5.0
2022-02-22 22:17:37 +02:00
redzic 6c70e2c0cc
Update dependencies (#571) 2022-02-11 23:03:37 +00:00
Josh Holmer c78d01528d
Update several dependencies (#568)
Notably:
- parking_lot
- sysinfo
and many patch version bumps
2022-02-07 05:39:01 +00:00
redzic 9dc898a67e
Update rav1e and vapoursynth (#565) 2022-02-01 06:52:08 +00:00
Josh Holmer 351d19accb
Rework log levels to be more useful (#546)
- Update log levels so that the log file contains more verbose
  information by default, while leaving the stderr progress indicator
  relatively uncluttered
- Explicitly set the log level to `error` for all dependencies
- rav1e scenechange info has been moved to `--log-level trace`
- Avoid outputting colors if stderr is not a tty
- Fix a crash when using `--quiet`
2022-01-18 23:20:01 +00:00
redzic d893cd5aad
Update vapoursynth and other dependencies (#540) 2022-01-17 08:24:52 +00:00
redzic 36fbff1dcd
Update version to 0.3.1 (#541) 2022-01-17 07:16:42 +00:00
Josh Holmer 6ab07aa7f0
Fix av1an versions in root cargo toml (#515) 2022-01-06 02:06:47 +00:00
Zen 7bb1b47eaa 0..3.0 2022-01-06 02:49:36 +02:00
Josh Holmer d7c3db84ca
Pre release changes (#511)
* Use latest indicatif release

* Fix some clippy lints

* Also validate formatting and warnings in CI

* Minor dependency updates

* Do not show '/0' frames if we haven't computed frame count yet
2022-01-05 03:48:35 +00:00
redzic 1b27ef7e28
Remove ctrlc dependency (#509)
* Remove ctrlc dependency

* Update other dependencies
2022-01-04 10:23:00 +00:00
Josh Holmer dc2d3ebdab
Add a CLI option for changing chunk order (#493)
People have been requesting this because the default chunk order tends
to skew toward underestimating the final filesize. Longest first is kept
as the default because it provides the best performance.
2022-01-02 05:49:47 +00:00
redzic 214adba05b
Migration to clap-v3 (#487) 2022-01-02 01:55:11 +00:00
redzic bdb983e884
Update indicatif and other dependencies (#483) 2021-12-31 22:42:14 +00:00
Josh Holmer a9f3fb266e
Regular dependency upgrades (#471) 2021-12-25 03:24:18 +00:00
Josh Holmer e85a4bc38f
Bump av-ivf to 0.2.3 (#464)
av-ivf was updated to nom 7. This removes several duplicate versions of
dependencies, reducing the total number of dependencies for av1an by 7.
2021-12-22 05:51:35 +00:00
Josh Holmer 0aa8fc5281
Automatically add cq-level arg if it is not present (#460)
* Automatically add cq-level arg if it is not present

Currently, target quality requires a cq-level (or equivalent for the
encoder) arg to be present in order to work, so that it can replace the
existing cq-level with the new one. This commit makes it so that if the
arg is not present, av1an will insert it with the proper quality level.

* Use format! for insert_q

Co-authored-by: Redzic <yusufredzic@gmail.com>
2021-12-22 01:14:34 +00:00
redzic 4014164c81
Update indicatif (#455) 2021-12-16 14:06:16 +00:00
Redzic a5b6f18aee fix indicatif... again 2021-12-15 10:58:52 -06:00
redzic 9e8c2da1d9
cargo update + revert indicatif (#447) 2021-12-15 10:57:51 -06:00
Josh Holmer 2be2c79ca7
Retry chunk without TPL if a TPL-related crash occurs (#444)
aomenc has had a history of crashes related to TPL on certain
chunks, particularly in videos with less motion, such as animated
content. This commit enables a workaround to retry a chunk with TPL
disabled if such a crash is detected. Although there is some amount of
psychovisual quality loss with TPL disabled, this is preferable to being
unable to complete the encode.
2021-12-14 17:17:07 -06:00
redzic 6f46ef8531
Replace last regex with hand written parser (#441)
* Replace panic with error return

* Replace last regex with hand written parser

* Update dependencies

* Fix clippy warnings

* Adjust idx check

* More idiomatic rust
2021-12-10 01:23:24 -06:00
Redzic cdf277eeaf Reduce codesize of construct_target_quality_command
- `x.to_string()` generates much more code than `format!("{}", x)`,
  so we should prefer `format!`. There are other ways to further
  reduce the codesize of construct_target_quality_command, but
  they are much more complicated, so they are reserved for a
  future PR.

- Update dependencies.
2021-12-01 21:02:48 -06:00
Redzic d843bd190f Subtract encoded frames from progress bar on chunk restarts 2021-11-25 13:07:21 -06:00
redzic de5cfda00f
Remove/replace remaining regex parsing code (#424) 2021-11-20 21:49:25 -06:00
redzic 5a23cc3565
Implement SSE4.1/SSSE3 SIMD-accelerated frame parsing for aomenc/vpxenc (#419)
The SSE4.1/SSSE3 implementation of the parsing routines for
aomenc/vpxenc outperform the old regex by a factor of about 5000x
(3.5 μs vs. 0.7 ns to parse a single line on a Zen 1 1950x).

The idea is to eventually remove regex as a dependency entirely,
especially since there is an open PR to indicatif that speeds up the
template expansion (which replaces regexes with hand-written parsing).
2021-11-20 10:35:31 -06:00
redzic 34d991d06d
Delegate chunk starts/finishes to debug log level, other logging changes (#412)
* Rename `--logging` to `--log-file` (`-l` shorthand still works)
* Introduce new `--log-level` option to set the log level filter

This change makes `--verbose` not log chunk starts and finishes, and
that information is not logged in the log file either unless
`--log-level=debug` is set. By default, `--log-level` is set to `info`.

The `debug` log level also logs the rav1e scenechange decision info
for each frame.
2021-11-18 14:06:50 -06:00
redzic db5e569f9d
Adjust CLI output to be more colorful (#407)
* Adjust verbose progress style for workers

* Also add `-y` flag to overwrite output file without confirmation

* Update dependencies

* Fix compilation on older versions of rustc
2021-11-17 23:01:46 -06:00
redzic f94c65f992
Improve logging (especially with --verbose) (#404)
* Improve logging (especially with `--verbose`)

Much more information is now logged when using `--verbose` (all INFO
messages), and the output is also much cleaner.

The issue of logs being hidden while the progress bar is active
because of conflicting ANSI escape codes has been completely fixed, and
no more undesirable hacks are required to correctly display the
set_thread_affinity warning message correctly, for example. This has
been achieved by using indicatif's `println` function instead of just
printing to stderr directly.

flexi_logger has been moved to av1an-cli as it should be, since a
logging framework that actually prints things and handles log level
filtering is not the responsibility of the core library. However, the
actual progress bar code with indicatif still lives in av1an-core for
now.

* Make `--quiet` conflict with `--verbose`

* Fix resuming and improve error handling slightly
2021-11-17 10:57:08 +02:00
redzic 637314e7f1
Fix compilation on macos (#403) 2021-11-16 03:42:41 +02:00
redzic 98bb4cc4b6
Allow setting thread affinity for each worker (#402)
Introduces a new command line option called `--set-thread-affinity=n`,
where `n` specifies the number of threads to assign to each worker.

We use the `affinity` crate instead of the `nix` crate directly to
get abstractions for Windows and Linux.
2021-11-16 00:15:26 +02:00
redzic 7f2734c7e1
Initial round of improvements for error handling (#401)
The main change that this commit introduces is an `EncoderCrash` type
for errors, which also reports the stderr of the process piping to
the encoder so that it is easier to tell what the actual error is.

Target quality probe crashes are now properly logged with the
`EncoderCrash` type, replacing the old panicking code and making the
error significantly easier to read.

An "optimization" has been implemented which does not wait for the child
process piping to the encoder to exit, and instead collects its stderr
output asynchronously and immediately exits upon an encoder crash. This
does not yet apply to target quality probes, as there is not yet a
single interface for running an encoder (it is done manually for now).
This will change in the future, but this patch is getting too big for
everything so for now we just capture stderr synchronously for target
quality probes.

- Also update `get_percentile` to use an O(N) algorithm from the
  standard library instead of O(N*log N).
2021-11-11 04:52:32 +02:00
redzic 754c1192fe
Always copy subtitles and all audio tracks (#400)
* Always copy subtitles and all audio tracks

Require mkvmerge for x265

* cargo update

* Fix concat if audio file does not exist

* Use stable rustc in Docker, do not compile VVC

* Use old Docker image for now which works with vapoursynth

* Fix tests.yml

* Use mkvmerge for tests

* Add mkvtoolnix as a dependency in tests.yml
2021-11-07 14:20:12 +02:00
Zen a874dec452 av1an 0.2.0 release 2021-10-31 22:56:47 +02:00
redzic eb1047aa30
Fix vergen for release (#393) 2021-10-31 22:20:38 +02:00
Zen 782f74982b Bump version 2021-09-11 21:24:58 +03:00
Josh Holmer 911b75013e
Use cargo version of rav1e 0.5 (#391) 2021-10-31 20:20:58 +02:00
redzic e2451684b8
Refactor: Better ffmpeg pipe handling + more info in --version (#389)
* Refactor: Better ffmpeg pipe handling + more info in --version

* Update dependencies
* Better documentation in `--help`
* Include git commit hash and more info in `--version`
* Add `-k` as shorthand for `--keep`
* Do not pass `ffmpeg_pipe` around everywhere
* Use macro to generate encoder bit depth functions
* Fix documentation in README

* Fix some clippy warnings
2021-10-30 14:49:26 +03:00
redzic 2f50335b6e
Fix hanging of VMAF calculation (#386)
* Fix run_vmaf by removing Stdio::piped() when not needed

* Update dependencies
2021-10-24 23:10:29 +03:00
Zen 3e9f729d3c fix building 2021-10-14 02:13:14 +03:00
Josh Holmer c8547c7c9b
Changes for rav1e's hybrid scene detection update (#381) 2021-10-14 02:04:11 +03:00
redzic 3147a942a7
Reduce piping overhead to encoder in certain situations (#379)
Previously, we always piped the chunk method into FFmpeg
to apply filters and convert the pixel format, but this
just adds unnecessary overhead if there is no FFmpeg filter
specified and if the pixel format of the source and encode
are the same. So, in that situation, we now pipe the chunk
method directly into the encoder instead. This should
nicely speed things up a bit (and possibly reduce memory
consumption).

Additionally, `--pix-format` is now verified using the FFmpeg
API when the CLI args are parsed, making it impossible to
accidentally specify an incorrect format. Previously, there
were no checks at all, so if you specified the wrong format it
would just fail during the encoding process with no obvious
indication of the root cause.

This does not apply to vapoursynth input for now, as that
would be slightly more complicated since it has a different
API for getting the pixel format.
2021-10-13 05:21:55 +03:00