question(s) about bjam code

question(s) about bjam code

as I'm getting sucked deeper into the low-level bjam source code,
I have a few questions, notably about the inner workings of the
execution state machine in make1.c

(Please let me know if I should ask these questions offline, or
in form of github issues (which github suggests to create if I
click on an individual line of code in its browser).

Here is my first question:

I'm looking at
https://github.com/boostorg/build/blob/develop/src/engine/make1.c#L326,
and wonder why, despite the inline comment above the referenced
line, this is not the first statement to be executed in the make1a
function. I understand that the make1a function may be called
multiple times from different locations, but should be ignored
(i.e., should be a noop) in all but one case.

However, each time the function is called, its parent's async
counter is incremented (as long as this target 's progress is not
yet RUNNING).

Re: question(s) about bjam code

AMDG

On 10/05/2017 08:12 PM, Stefan Seefeld via Boost-build wrote:

> <snip>
> I'm looking at
> https://github.com/boostorg/build/blob/develop/src/engine/make1.c#L326, and
> wonder why, despite the inline comment above the referenced line, this is not
> the first statement to be executed in the make1a function. I understand that the
> make1a function may be called multiple times from different locations, but
> should be ignored (i.e., should be a noop) in all but one case.
>
> However, each time the function is called, its parent's async counter is
> incremented (as long as this target 's progress is not yet RUNNING).
>

Example:

A depends on C
B depends on C

We enter make1a for C twice, once with each parent,
A and B. A and B both need to wait for C to complete,
hence we increment the async counter both times.

We only skip this if C has already completed, and
no waiting is required.

Re: question(s) about bjam code

On 05.10.2017 22:31, Steven Watanabe
via Boost-build wrote:

AMDG
On 10/05/2017 08:12 PM, Stefan Seefeld via Boost-build wrote:

<snip>
I'm looking at
https://github.com/boostorg/build/blob/develop/src/engine/make1.c#L326, and
wonder why, despite the inline comment above the referenced line, this is not
the first statement to be executed in the make1a function. I understand that the
make1a function may be called multiple times from different locations, but
should be ignored (i.e., should be a noop) in all but one case.
However, each time the function is called, its parent's async counter is
incremented (as long as this target 's progress is not yet RUNNING).

Example:
A depends on C
B depends on C
We enter make1a for C twice, once with each parent,
A and B. A and B both need to wait for C to complete,
hence we increment the async counter both times.
We only skip this if C has already completed, and
no waiting is required.

Fair enough. However, if the parent counter is incremented,
shouldn't there also (directly or indirectly) be a MAKE1B state that
is pushed onto the stack for that parent ?
This happens if the child target hasn't started processing, but it
doesn't if it's already on the stack.
In my modified code I'm observing a bug caused by the state machine
running out of states without completing all tasks, and I wonder
whether that isn't caused by this asymmetry.

Re: question(s) about bjam code

On 13.10.2017 10:45, Stefan Seefeld
wrote:

On 05.10.2017 22:31, Steven Watanabe
via Boost-build wrote:

AMDG
On 10/05/2017 08:12 PM, Stefan Seefeld via Boost-build wrote:

<snip>
I'm looking at
https://github.com/boostorg/build/blob/develop/src/engine/make1.c#L326, and
wonder why, despite the inline comment above the referenced line, this is not
the first statement to be executed in the make1a function. I understand that the
make1a function may be called multiple times from different locations, but
should be ignored (i.e., should be a noop) in all but one case.
However, each time the function is called, its parent's async counter is
incremented (as long as this target 's progress is not yet RUNNING).

Example:
A depends on C
B depends on C
We enter make1a for C twice, once with each parent,
A and B. A and B both need to wait for C to complete,
hence we increment the async counter both times.
We only skip this if C has already completed, and
no waiting is required.

Fair enough. However, if the parent counter is incremented,
shouldn't there also (directly or indirectly) be a MAKE1B state
that is pushed onto the stack for that parent ?
This happens if the child target hasn't started processing, but it
doesn't if it's already on the stack.

<snip>
I'm looking at
https://github.com/boostorg/build/blob/develop/src/engine/make1.c#L326, and
wonder why, despite the inline comment above the referenced line, this is not
the first statement to be executed in the make1a function. I understand that the
make1a function may be called multiple times from different locations, but
should be ignored (i.e., should be a noop) in all but one case.
However, each time the function is called, its parent's async counter is
incremented (as long as this target 's progress is not yet RUNNING).

Example:
A depends on C
B depends on C
We enter make1a for C twice, once with each parent,
A and B. A and B both need to wait for C to complete,
hence we increment the async counter both times.
We only skip this if C has already completed, and
no waiting is required.

Fair enough. However, if the parent counter is incremented,
shouldn't there also (directly or indirectly) be a MAKE1B state
that is pushed onto the stack for that parent ?
This happens if the child target hasn't started processing, but
it doesn't if it's already on the stack.

so an async counter being set to N for target T means there are N
prerequisite targets "in flight". For this state machine, "in
flight" means that there are N `make1B` states for T either already
on the stack, or eventually will be (pushed from prerequisites).
Correct ?

I'm debugging a case where that assumption (invariant ?) is broken
(leading to a build process that suddenly stops), and I wonder where
the logic error is...
(I'm trying to capture the state machine logic in a UML state
diagram in an attempt both to document and to understand, but find
the actual logic very elusive due to the async stuff. Any idea what
formal language (doesn't have to be UML) might be useful to capture
the state machine in pseudo-code ?)