* Log the command used for concat (activated with log level `debug`)
(the command along with process output is logged again at log level
`error` if the command failed, regardless of `--log-level`)
* Replace some unwraps with `Result`s
* Add more `with_context()`s
* Rename `--logging` to `--log-file` (`-l` shorthand still works)
* Introduce new `--log-level` option to set the log level filter
This change makes `--verbose` not log chunk starts and finishes, and
that information is not logged in the log file either unless
`--log-level=debug` is set. By default, `--log-level` is set to `info`.
The `debug` log level also logs the rav1e scenechange decision info
for each frame.
* Adjust verbose progress style for workers
* Also add `-y` flag to overwrite output file without confirmation
* Update dependencies
* Fix compilation on older versions of rustc
Previously, these warn statements came before we create
the temp directory. I'm assuming when the first log
statement is run, flexi_logger then attempts to create
the log file. If it fails, it doesn't try again.
Ensuring we create the temp directory before attempting
to log should resolve this issue.
* Improve logging (especially with `--verbose`)
Much more information is now logged when using `--verbose` (all INFO
messages), and the output is also much cleaner.
The issue of logs being hidden while the progress bar is active
because of conflicting ANSI escape codes has been completely fixed, and
no more undesirable hacks are required to correctly display the
set_thread_affinity warning message correctly, for example. This has
been achieved by using indicatif's `println` function instead of just
printing to stderr directly.
flexi_logger has been moved to av1an-cli as it should be, since a
logging framework that actually prints things and handles log level
filtering is not the responsibility of the core library. However, the
actual progress bar code with indicatif still lives in av1an-core for
now.
* Make `--quiet` conflict with `--verbose`
* Fix resuming and improve error handling slightly
Introduces a new command line option called `--set-thread-affinity=n`,
where `n` specifies the number of threads to assign to each worker.
We use the `affinity` crate instead of the `nix` crate directly to
get abstractions for Windows and Linux.
The main change that this commit introduces is an `EncoderCrash` type
for errors, which also reports the stderr of the process piping to
the encoder so that it is easier to tell what the actual error is.
Target quality probe crashes are now properly logged with the
`EncoderCrash` type, replacing the old panicking code and making the
error significantly easier to read.
An "optimization" has been implemented which does not wait for the child
process piping to the encoder to exit, and instead collects its stderr
output asynchronously and immediately exits upon an encoder crash. This
does not yet apply to target quality probes, as there is not yet a
single interface for running an encoder (it is done manually for now).
This will change in the future, but this patch is getting too big for
everything so for now we just capture stderr synchronously for target
quality probes.
- Also update `get_percentile` to use an O(N) algorithm from the
standard library instead of O(N*log N).
* Always copy subtitles and all audio tracks
Require mkvmerge for x265
* cargo update
* Fix concat if audio file does not exist
* Use stable rustc in Docker, do not compile VVC
* Use old Docker image for now which works with vapoursynth
* Fix tests.yml
* Use mkvmerge for tests
* Add mkvtoolnix as a dependency in tests.yml
* Refactor: Better ffmpeg pipe handling + more info in --version
* Update dependencies
* Better documentation in `--help`
* Include git commit hash and more info in `--version`
* Add `-k` as shorthand for `--keep`
* Do not pass `ffmpeg_pipe` around everywhere
* Use macro to generate encoder bit depth functions
* Fix documentation in README
* Fix some clippy warnings
* Remove sc-downscale-height CLI option
The standard scenecut detection method at 1080p
is currently as fast as downsampling to 720p
and running scenecut detection on that, and the
fast scenecut method is significantly faster
without downsampling, because the algorithm
is faster than ffmpeg's resizing algorithm.
As such, it doesn't make sense to have this option anymore.
* Fix conditional use of newer VS APIs
It makes sense to enable these by default,
since the vast majority of users
should have a moderately recent version
of Vapoursynth.
We also needed to disable the static ffmpeg
as the default for now. libaom returns None
for the pixel format, so it is not usable
for the changes needed later in this PR.
I attempted to get the git version of
ffmpeg-next compiling with libdav1d,
but was running into issues, I believe
because Arch's package manager only includes
a dynamic library for libdav1d and not
a static version. It probably makes sense
to disable it anyway given the issues
Windows users have been having with it.
* No longer convert input to 8-bit for scene detect
It appears that downconverting does have an impact
on scene detection accuracy. The speed penalty
for running in 10-bit compared to 8-bit is only
10%. Given this, it seems preferable to run
the scene detect in the more accurate format.
* Skip ffmpeg intermediate step for vapoursynth input
* Fix skipping of ffmpeg pipe step
This optimization added in #379 was never triggered.
Because the initialize() method always added a set
of default params to self.ffmpeg_params, the clause
to skip the ffmpeg pipe, self.ffmpeg_params.is_empty(),
was unreachable.
* Revert "Remove sc-downscale-height CLI option"
This reverts commit a51763607faddae07a595f567d4f4cf815de3299.
* Validate pixel formats based on encoder support
* Fix CI build
The dynamic linking of ffmpeg doesn't seem to play nicely with
the old version of ffmpeg which is packaged in Ubuntu's last LTS.
This enables static linking when we are running in CI to work around
that issue, given that that workaround is easier than overhauling
which docker container we use.
Previously, we always piped the chunk method into FFmpeg
to apply filters and convert the pixel format, but this
just adds unnecessary overhead if there is no FFmpeg filter
specified and if the pixel format of the source and encode
are the same. So, in that situation, we now pipe the chunk
method directly into the encoder instead. This should
nicely speed things up a bit (and possibly reduce memory
consumption).
Additionally, `--pix-format` is now verified using the FFmpeg
API when the CLI args are parsed, making it impossible to
accidentally specify an incorrect format. Previously, there
were no checks at all, so if you specified the wrong format it
would just fail during the encoding process with no obvious
indication of the root cause.
This does not apply to vapoursynth input for now, as that
would be slightly more complicated since it has a different
API for getting the pixel format.
* Refactor into `Input` type for handling vapoursynth input
Fixes several issues, namely `--vmaf` not working on vapoursynth input
as well as unifying the VMAF interface, which also results in
`--vmaf-threads` being passed correctly to `--vmaf`, which is massively
faster now as a result. If `--vmaf-threads` is not specified, then
the number of detected CPU threads is used for `--vmaf`.
Additionally, VMAF plots are rendered as SVGs now, which also removes
some dependencies since we don't use the bitmap backend at all anymore.
* Small cleanup: more consistent usage of `as_path`
* Use a different method for ffmpeg frame count
This method uses ffprobe to count the number of packets
(which is identical to the number of frames, but faster)
in a video stream. This works with more video formats,
including with --enable-keyframe-filtering=2 in aomenc.
Performance should be similar or better than ffmpeg -copy.
* Add sanity check when using keyframe filtering 2
* Use ffmpeg-next crate for getting frame count
* Add LLVM/Clang to Github Actions
* Enable ffmpeg static and build features by default
This leaves the default at medium, which is an improved version
of the current scenecut detection method.
The fast method has also been improved from its previous version.
The slow method is a completely new implementation, which has
been shown to be more accurate, but is also about 15% slower.
This also removes the av-scenechange-fast split method,
in favor of making a separate CLI option for --sc-method,
which can be set to "fast", "medium", or "slow".
This also adds a CLI option for scenecut downscaling,
which can be specified by providing a maximum height e.g.
--sc-downscale-height 720 to downscale any input which is
above 720p to 720p for scenecut detection.
This will avoid upscaling anything below 720p, because that
would just slow down scenecut detection for no benefit.
Because of the scenes variable updating,
this would print the number of scenes after
extra splitting instead of the number of scenes
before splitting and then the number after splitting.
- This removes the `parking_lot::Mutex` wrapping the multi progress bar, as it was not actually necessary. This also removes `parking_lot` as a dependency, as it is unused now.
- Rework the implementation of `mkvmerge` concatenation to not perform any unnecessary UTF-8 validation or allocations. This requires an algorithm to specify chunks with a '+' in between, but without a trailing '+', without later removing arguments (as the API of `std::process:Command` does not support this). This does not change the behavior of `-c mkvmerge` in any way, however.
- Also fix some miscellaneous clippy warnings.
In some cases concatenation silently fails. To prevent loss of the
encoded data we now check if the output file exists. If it does not
exist we do not delete the temp folder.
