In a batch of 100's of files, maybe 1 in 50 hangs the sequence.
The CPU still appears to be consumed with an apparent Handbrake conversion,
but [NO] progress on the conversion is occurring.
Stopping the Handbrake conversion and then retrying as a single non-batched conversion
in a 2nd Handbrake instance always works.
This problem may be slightly worse in Handbrake 0.99 compared to 0.98 version.

Is there any progress on fixing this serious Bug ?
If the developers are having difficulty finding this Bug, then implement a workaround
in which a timer is set on progress through the file as measured by the percentage in
the extreme left bottom of the window --
if no change after 30 seconds, then
1) terminate the conversion,
2) enter the pathname of the hanging file in a file-based Log,
3) either a) retry the same file and continue the batch sequence, or
b) continue the batch sequence with the next file in sequence.

The hanging pathname always will complete conversion when pulled out of the batch
and converted independently,
thus the Bug has something to do with the batch handling software.
The hang usually becomes stuck after around 4% of the file converted --
thus the hang is not occurring on startup but during the conversion.

Some batch conversion series will last for several Days, usually scheduled during
a time of non-use or limited use of the PC, thus it is critical
that everything be done to ensure that it runs to completion without hanging.

Hopefully, no future release of handbrake will occur without dealing this serious Bug,
one way or the other.

Not yet.
I read the description of the Beta, and nothing in that description
dealt explicitly with this serious Bug.
And since this Bug has a global effect on a sequence of Conversions
(although the hanging file is usually an ATSC file),
I would have expected the Beta release documentation to address
this Bug explicitly.

There's been hundreds of changes since 0.9.9, and some of them are actually updating third party libraries have have had hundreds of changes themselves. I wouldn't expect a bug like this to appear in the release highlights, notwithstanding your repeated overdramatization as a "serious Bug." You won't get any developer attention unless you reproduce the problem with the nightlies or at least 0.10b1.

mduell wrote:There's been hundreds of changes since 0.9.9, and some of them are actually updating third party libraries have have had hundreds of changes themselves. I wouldn't expect a bug like this to appear in the release highlights, notwithstanding your repeated overdramatization as a "serious Bug." You won't get any developer attention unless you reproduce the problem with the nightlies or at least 0.10b1.

Over-dramatization ?
This Bug affects the overall general sequences of usages of the program.
Regardless of what someone is trying to perform with the program, if it is hung,
it will not be performed.
That is a showstopper Bug.

This program performs 2 features which are important and are hard to find elsewhere:
1) Closed Captions pass-through of many forms of Closed Captions,
(notably absent from this is Windows Media Center (WMC) .wtv file Closed Captions
and I am not aware if the previously reported SCTE-20 Closed Captions failure was fixed), and

2) batch series of conversions, which allow the consumption of the PC CPU throughput
which conversions demand be scheduled at times when the user may not be using the PC
(of course, those users who have not figured out the arcane method to set up a
batch of conversions might have a hard time exploiting this feature).

I am downloading the Beta today, but I no longer have a long sequence to convert.
On average this hangs on maybe 1 in 20 conversions (around 60% of which are ATSC)
in the version I documented.
It can take hours to reach the hang.
As mentioned, at least a hang recovery timer-terminate-continue with a Log
of the pathnames of hanging files, should be implemented,
whether you think you have fixed this Bug or not.

Batch Hang Bug definitely Not fixed in the Beta.
This time hung at 98.72% of the 20th conversion file after around 2 hours of conversions
(pathnames edited).
It is easy for a project administrator of a program like this to become distracted
by all the paid advertising shills who hang out on forums like this all day.
This project should focus on its core mission (and its special sauce, i.e. Closed Captions and Batch)
first, before diverging onto whatever nonsense is being pitched.
Strangely, Frames Per Seconds may have been slower, with this new GPU feature turned on
(Nvidia CUDA) -- conversion speed is important and is definitely part of the mission.

As recommended, implement a timer-terminate-retry-continue-to-next-conversion-log
workaround, then the Bug's effects will not be as catastrophic.
(Of course, this would not be catastrophic to a paid advertising shill
who does not really use this program).

This isn't necessarily a bug.
Recorded ts files really need to be pre-processed first before running them though HandBrake. HandBrake is not a ts repair tool, so if your ts file has any errors/corruption in it, it'll cause problems. From the log above, it looks like your file has problems.

TsDoctor is one example of a tool that is designed for this purpose.

If you can provide a sample that re-creates it, we can take a look see if there are any workarounds, otherwise there is nothing more we can do about it.

There should be more logs after that message with additional statistics about the encode. But since the log you posted is broken, I really cant determine if it was handbrake that hung at exactly this point, or if whatever corrupted your log caused this missing output.

This log you provided for the nightly indicates that the encoding process got much further (essentially finishes) than your log from the releases. Your release log indicates it hangs very early in the encode. So I'm not convenced that we are even looking at the same issue in the nightlies as you see in the release.

Aeneas wrote:
This project should focus on its core mission (and its special sauce, i.e. Closed Captions and Batch)
first, before diverging onto whatever nonsense is being pitched.

This project has *no* "core mission". It is a hobby that has attracted a small number of developers who work on whatever interests them.

We would of coarse like HandBrake to work as expected and we do make an effort to fix bugs as we find them. But your bug is not reproducable by anyone. So it leaves us with no way to make progress on it.

s55 wrote:This isn't necessarily a bug.
Recorded ts files really need to be pre-processed first before running them though HandBrake. HandBrake is not a ts repair tool, so if your ts file has any errors/corruption in it, it'll cause problems. From the log above, it looks like your file has problems.

TsDoctor is one example of a tool that is designed for this purpose.

If you can provide a sample that re-creates it, we can take a look see if there are any workarounds, otherwise there is nothing more we can do about it.

As I have mentioned, the same hanging file, when pulled out of the sequence,
will convert to completion.
Thus, the inherent file structure and possible discontinuities in the data
are not causing this Hang.
I have never had a hang when using this program in non-batch mode.

I have already recommended a simple Timer-Terminate workaround which should trigger when
no progress is being made through the file.

As I have mentioned, it is likely that few other users are using this Batch feature because
it is hard to figure out how to use and poorly documented.
Who knows, maybe some of the developers and QA people on this project have
not figured out the batch configure sequence.

Continuing the Batch to the next manually, I had later on another Hang, at 99.49%.
The fact that these Hangs are happening late seems to be a difference in this Beta.
Attaching the debugger to this Hang-ed HandbrakeCLI.exe, this is the data visible:

Aeneas wrote:
This project should focus on its core mission (and its special sauce, i.e. Closed Captions and Batch)
first, before diverging onto whatever nonsense is being pitched.

This project has *no* "core mission". It is a hobby that has attracted a small number of developers who work on whatever interests them.

We would of course like HandBrake to work as expected and we do make an effort to fix bugs as we find them. But your bug is not reproducible by anyone. So it leaves us with no way to make progress on it.

This is the first I have heard that this Bug is not reproducible.
Is your QA process set up to perform long sequences of Batch conversions
overnight over several hours ?

HandBrakeCLI.exe does not support batches. Each encode is a new process so "batching" is not the source of the problem.
Also note that HandBrakeCLI.exe is not compiled with symbols, so useful debugging is not possible.

Without a sample that reproduces the problem, there is nothing we can do to investigate it.

My "QA process" is to stack up a directory full of collected samples that are known to have caused problems in the past, load them into the queue, let it run to completion and scan the logs produced for any unusual and obvious changes in behavior.

I can't speak for how the other developers test things. I only run such tests on the linux gui since that is the platform I develope for.

s55 wrote:HandBrakeCLI.exe does not support batches. Each encode is a new process so "batching" is not the source of the problem.
Also note that HandBrakeCLI.exe is not compiled with symbols, so useful debugging is not possible.

Without a sample that reproduces the problem, there is nothing we can do to investigate it.

The Debugger output that I produced does directly represent the program state when this Bug occurs.

What do you mean Handbrake does not support batches ?
Batches are clearly in the user interface of the program, though not well documented.
If you are saying that you provide the functionality "as is" that is a different statement.
Again, the workaround I have recommended is trivial for you to implement.
How hard is a Timer to implement in your code and perform a simple test on the
percentage in the left bottom corner of the Handbrake window ?
This is not a serious coding undertaking.
Every 30 seconds, check to see if the percentage has changed -- if not, terminate. Trivial.

Last edited by Aeneas on Thu Aug 21, 2014 6:50 pm, edited 1 time in total.

Read what I said. "HandBrakeCLI.exe" (as in the command line interface) does not support batches. Since the Windows GUI calls a new instances of this for every new encode on the queue, it means it has nothing to do with the order of the queue that triggers this issue. This means another factor is contributing to the problem.

s55 wrote:Read what I said. "HandBrakeCLI.exe" (as in the command line interface) does not support batches. Since the Windows GUI calls a new instances of this for every new encode on the queue, it means it has nothing to do with the order of the queue that triggers this issue. This means another factor is contributing to the problem.
A timer is a hack, not a solution. Not going to happen.

If you cannot fix this catastrophic Bug, this Timer solution is all you have.
This is a showstopper Bug, in one of your 2 main features --
it has to be fixed one way or the other.

The appearance is that there are 2 programs operating in the Batch operation:
Handbrake and HandbrakeCLI. Handbrake.exe seems to be the supervisor which
spawns HandbrakeCLI.exe to actually perform each conversion.
If your point is to blame the Windows Kernel for this Hang, that is not the case.

Fact is, you are the *only* person to report this problem, and have not provided sufficient data for us to investigate further or any kind of proof that the issue is not a factor external to HandBrake.

Unless you can provide a sample that reproduces this issue, I consider this topic closed.

Fact is, you are the *only* person to report this problem, and have not provided sufficient data for us to investigate further or any kind of proof that the issue is not a factor external to HandBrake.

Unless you can provide a sample that reproduces this issue, I consider this topic closed.

Again, I have stated that the same file which causes the Hang, when pulled out of the Batch
and converted separately, will convert fine.
This either indicates that the Batch process is flawed, or maybe this is simply
a Hang which occurs on every roughly 20 independent conversions.
i.e. a random Bug which occurs unpredictably, which occurs in Batches
because that increases the number of opportunities for the Bug to occur.

Try documenting the Batching process better, and maybe more people
will report this failure.

Also, such a Timer is not a "hack" -- every serious piece of equipment, especially
something like a pacemaker or other medical device, has a Timer of this type,
not just implemented in software, but built into the hardware.
Practically, every microcontroller sold nowadays has some sort of Timer of this type built-in.

Last edited by Aeneas on Thu Aug 21, 2014 10:17 pm, edited 1 time in total.

Aeneas wrote:
Again, I have stated that the same file which causes the Hang, when pulled out of the Batch
and converted separately, will convert fine.
This either indicates that the Batch process is flawed, or maybe this is simply
a Hang which occurs on every roughly 20 independent conversions.

...or it could be something unique to your environment (*very* likely since you are the only person to report this). Perhaps you have a failing hdd. Or if one of your drives is a network mapped drive perhaps a random network failure. There are dozens of things this could be that are not related to handbrake.

It might help to post both a log of a failure together with a log of the subsequent success when you retry encoding of the same source file. We might see a difference between the 2 logs that indicate the source of the problem.

Aeneas wrote:
Again, I have stated that the same file which causes the Hang, when pulled out of the Batch
and converted separately, will convert fine.
This either indicates that the Batch process is flawed, or maybe this is simply
a Hang which occurs on every roughly 20 independent conversions.

...or it could be something unique to your environment (*very* likely since you are the only person to report this). Perhaps you have a failing hdd. Or if one of your drives is a network mapped drive perhaps a random network failure. There are dozens of things this could be that are not related to handbrake.

It might help to post both a log of a failure together with a log of the subsequent success when you retry encoding of the same source file. We might see a difference between the 2 logs that indicate the source of the problem.

Nothing else in my system is hanging, and there are many large complex applications
which run reliably on this PC.

If you are having difficulty figuring out what is causing this problem, the normal response
from committed developers is to add debug code at various points where
operations are instigated.
Maybe pump that data into a circular buffer so that the most recent data would
be present to identify the cause of this Hang.

I notice there are 2 Log tabs -- encode and scan. What is the difference.
I notice right now that they are describing 2 different conversion source pathnames.

Last edited by Aeneas on Thu Aug 21, 2014 10:53 pm, edited 1 time in total.

There should be more logs after that message with additional statistics about the encode. But since the log you posted is broken, I really cant determine if it was handbrake that hung at exactly this point, or if whatever corrupted your log caused this missing output.

This log you provided for the nightly indicates that the encoding process got much further (essentially finishes) than your log from the releases. Your release log indicates it hangs very early in the encode. So I'm not convenced that we are even looking at the same issue in the nightlies as you see in the release.

What appears fragmented or incomplete ?
As I stated, the pathnames are edited.

As I mentioned a few messages ago, the fact that the percentage is much later,
is different from the earlier hangs reported.
Thus, I was implying that the developers may have made progress on part of this problem.
I have only used this new Beta version for a few hours though, so far.

Finally, I would add that when these hangs occur, both before and in this Beta,
Windows Task Manager reports HandbrakeCLI.exe at a consistent 25% of this quad CPU --
usually,HanbrakeCLI.exe is up at 75%-80%, during healthy conversions.

Last edited by Aeneas on Thu Aug 21, 2014 11:02 pm, edited 1 time in total.

Look, I'm pretty much done with this thread. I will not be reading it again. At every turn, you are trying to tell us how we should debug or develop or support HandBrake. And frankly, I'm not interested in your opinions about this. As I already said, HandBrake is a hobby. When I gets too frustrating, I set it aside for a while. This thread has become frustrating, so I am setting it aside.