It is fixed now. It looks like in testing the script or just before uploading I might have pressed a key or something in gvim. That (unintentionally) altered some part of the script. Sorry for this. I hate bugs. _________________emerge --quiet redefined | E17 vids: I, II | Now using e17 | e18, e19, and kde4 sucks :-/

~> genlop -t "=dev-libs/apr-1.3.9" --date 2 minute ago
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1039, <_GEN_0> line 23512.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1040, <_GEN_0> line 23512.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1042, <_GEN_0> line 23512.
* dev-libs/apr

I spoke too soon... I now have the issue myself, with the latest quietemerge ...

Code:

* Install app-emulation/emul-linux-x86-qtlibs-20081109. 49 seconds.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1075, <_GEN_0> line 103950.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1076, <_GEN_0> line 103950.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1078, <_GEN_0> line 103950.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1114, <_GEN_0> line 103950.
* Install net-fs/samba-libs-3.4.3-r1. 6 days, 22 hours, 46 minutes and 3 seconds.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1075, <_GEN_0> line 103959.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1076, <_GEN_0> line 103959.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1078, <_GEN_0> line 103959.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1114, <_GEN_0> line 103959.
* Install net-fs/samba-server-3.4.3-r1. 6 days, 22 hours, 57 minutes and 57 seconds.

I spoke too soon... I now have the issue myself, with the latest quietemerge ...

Code:

* Install app-emulation/emul-linux-x86-qtlibs-20081109. 49 seconds.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1075, <_GEN_0> line 103950.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1076, <_GEN_0> line 103950.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1078, <_GEN_0> line 103950.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1114, <_GEN_0> line 103950.
* Install net-fs/samba-libs-3.4.3-r1. 6 days, 22 hours, 46 minutes and 3 seconds.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1075, <_GEN_0> line 103959.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1076, <_GEN_0> line 103959.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1078, <_GEN_0> line 103959.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1114, <_GEN_0> line 103959.
* Install net-fs/samba-server-3.4.3-r1. 6 days, 22 hours, 57 minutes and 57 seconds.

And obviously, the 6 days isn't real!

I'll try to check genlop to see if I could find the bug

Hi,
The problem is that genlop gives the correct estimate if I increase the time in

Code:

genlop -t "=cat/pkg-ver" --date 3 minute ago

This was set to 1 minute in an earlier version of the script, and I noticed that increasing it to 3 minutes is good enough to stop that error from occuring in genlop; except it surely isn't good enough for all cases.

I don't want to increase the time too much since then it would create a problem with my parsing of genlop's output, if you emerged the same package twice in a row within the same time limit. The other option is for me to implement a counter myself. But that is a really clumsy option, especially when genlop already does it._________________emerge --quiet redefined | E17 vids: I, II | Now using e17 | e18, e19, and kde4 sucks :-/

I don't want to increase the time too much since then it would create a problem with my parsing of genlop's output, if you emerged the same package twice in a row within the same time limit. The other option is for me to implement a counter myself. But that is a really clumsy option, especially when genlop already does it.

I don't want to increase the time too much since then it would create a problem with my parsing of genlop's output, if you emerged the same package twice in a row within the same time limit. The other option is for me to implement a counter myself. But that is a really clumsy option, especially when genlop already does it.

Yes, something like that would be an option. Not that I am happy about using a bunch of pipes.
I will update the script with something similar to this. Let's hope genlop doesn't fail me again.

I don't want to skip the --date part. A simple thing like showing the merged time would be a highly inefficient operation if I don't give the --date part. I expect that genlop will take less time to parse emerge.log if I give it a date argument. If genlop still parses the whole emerge.log, then it is being really inefficient. Some people here have installations which are several years old and hence might have huge emerge.logs._________________emerge --quiet redefined | E17 vids: I, II | Now using e17 | e18, e19, and kde4 sucks :-/

It is fixed now. It looks like in testing the script or just before uploading I might have pressed a key or something in gvim. That (unintentionally) altered some part of the script. Sorry for this. I hate bugs.

Yeah, dunno what would make it give that error. Possibly some not properly encoded character somewhere. Anyway, all better now!

I don't want to skip the --date part. A simple thing like showing the merged time would be a highly inefficient operation if I don't give the --date part. I expect that genlop will take less time to parse emerge.log if I give it a date argument. If genlop still parses the whole emerge.log, then it is being really inefficient. Some people here have installations which are several years old and hence might have huge emerge.logs.

shocking truth: genlop is 'really' inefficient.

I've ran these commands multiple times and I'm giving you the representative numbers:
With the date argument:

It's faster because it doesn't have to do date comparison all the time...

I know by experience, genlop is highly inefficient. It's a really good tool that I like alot, but it is not coded in a way that is scalable. I've submitted a patch to improve it but I didn't had an answer yet. It's the 2nd time that I try to contribute to this tool, but to no avail... Maybe I should just 'fork' it.

I know by experience, genlop is highly inefficient. It's a really good tool that I like alot, but it is not coded in a way that is scalable. I've submitted a patch to improve it but I didn't had an answer yet. It's the 2nd time that I try to contribute to this tool, but to no avail... Maybe I should just 'fork' it.

Maybe you can contact the author(s) directly by email? And point them to the bug reports perhaps?_________________emerge --quiet redefined | E17 vids: I, II | Now using e17 | e18, e19, and kde4 sucks :-/

I know by experience, genlop is highly inefficient. It's a really good tool that I like alot, but it is not coded in a way that is scalable. I've submitted a patch to improve it but I didn't had an answer yet. It's the 2nd time that I try to contribute to this tool, but to no avail... Maybe I should just 'fork' it.

Maybe you can contact the author(s) directly by email? And point them to the bug reports perhaps?

Maybe as a rather hacky workaround you could create a second file with just the last however many lines you need to check from /var/log/emerge.log and get genlop to parse that?

I've just discovered the EMERGE_DEFAULT_OPTS make.conf variable, which comes in handy (before I was creating an alias to emerge, so that --ask was the default), however I've noticed that your script doesn't handle --ask when it's passed via EMERGE_DEFAULT_OPTS. If you can work out a way to detect --ask being passed in this form, that would be awesome, otherwise it might be best to add "--ignore-default-opts" to the emerge calls in the script.

I've just discovered the EMERGE_DEFAULT_OPTS make.conf variable, which comes in handy (before I was creating an alias to emerge, so that --ask was the default), however I've noticed that your script doesn't handle --ask when it's passed via EMERGE_DEFAULT_OPTS. If you can work out a way to detect --ask being passed in this form, that would be awesome, otherwise it might be best to add "--ignore-default-opts" to the emerge calls in the script.

Does --pretend even work when you have --ask in your EMERGE_DEFAULT_OPTS?

I think I can not even ignore the EMERGE_DEFAULT_OPTS since some people may have options other than --ask (for example --with-bdeps=y) which do not affect this script, and which will actually enhance/modify their emerge behaviour. If I ignore the default opts, then this script will not work as I intended it to,- that is, to be a simple way of showing what emerge is doing without interfering with emerge itself.

EDIT: -p seems to work even with --ask in EMERGE_DEFAULT_OPTS. But, I can see where the problem will be. I will be backgrounding emerge by emerge -q pkg >& log_file and then emerge will wait for an answer. In fact, when I tried it out just now, the emerge process had stopped:

So, unless you foreground emerge you will not be able to pass any options to emerge. I think there may be other complications too. Best option would be to launch this script as EMERGE_DEFAULT_OPTS="" quietemerge <other options>. You can create an alias for this._________________emerge --quiet redefined | E17 vids: I, II | Now using e17 | e18, e19, and kde4 sucks :-/

I've just discovered the EMERGE_DEFAULT_OPTS make.conf variable, which comes in handy (before I was creating an alias to emerge, so that --ask was the default), however I've noticed that your script doesn't handle --ask when it's passed via EMERGE_DEFAULT_OPTS. If you can work out a way to detect --ask being passed in this form, that would be awesome, otherwise it might be best to add "--ignore-default-opts" to the emerge calls in the script.

Does --pretend even work when you have --ask in your EMERGE_DEFAULT_OPTS?

Yeah, if you pass both --pretend and --ask to emerge, it ignores --ask.

Quote:

I think I can not even ignore the EMERGE_DEFAULT_OPTS since some people may have options other than --ask (for example --with-bdeps=y) which do not affect this script, and which will actually enhance/modify their emerge behaviour. If I ignore the default opts, then this script will not work as I intended it to,- that is, to be a simple way of showing what emerge is doing without interfering with emerge itself.

Yeah, that's a very good point. I wonder if there's a way to specifically ignore --ask, using sed, or some such? I'll try and hack something together to parse EMERGE_DEFAULT_OPTS and post back.

EDIT: -p seems to work even with --ask in EMERGE_DEFAULT_OPTS. But, I can see where the problem will be. I will be backgrounding emerge by emerge -q pkg >& log_file and then emerge will wait for an answer. In fact, when I tried it out just now, the emerge process had stopped:

So, unless you foreground emerge you will not be able to pass any options to emerge. I think there may be other complications too. Best option would be to launch this script as EMERGE_DEFAULT_OPTS="" quietemerge <other options>. You can create an alias for this.

Yeah, that's essentially what I've done.

Quote:

BTW, you don't need to hack anything. My script already does all the necessary hacking when it parses the command line arguments. I won't need to rewrite much, but my above comments still hold.

I was thinking perhaps something to read EMERGE_DEFAULT_OPTS, catch everything that isn't '--ask' and add it to the options passed on the command line, then run emerge with --ignore-default-opts and the options parsed from EMERGE_DEFAULT_OPTS as well as those read from stdin? Obviously if --ask is detected in EMERGE_DEFAULT_OPTS you'd then set e_opt_ask=1.

I guess that's probably similar to what you've done for parsing the command line arguments.

I was thinking perhaps something to read EMERGE_DEFAULT_OPTS, catch everything that isn't '--ask' and add it to the options passed on the command line, then run emerge with --ignore-default-opts and the options parsed from EMERGE_DEFAULT_OPTS as well as those read from stdin? Obviously if --ask is detected in EMERGE_DEFAULT_OPTS you'd then set e_opt_ask=1.

I guess that's probably similar to what you've done for parsing the command line arguments.

Yes. I got another idea with not providing EMERGE_DEFAULT_OPTS in the emerge command line itself. I will try it out and update the script if it works. Otherwise I will use this method that you mention._________________emerge --quiet redefined | E17 vids: I, II | Now using e17 | e18, e19, and kde4 sucks :-/

Update to version 20091120: Changelog:
* Respect --ask in EMERGE_DEFAULT_OPTS

@jw5801 My idea didn't work out I wanted to do emerge <<<'yes' > log_file &. But it didn't work for some reason. I can't be bothered to figure out why. Your method works just fine._________________emerge --quiet redefined | E17 vids: I, II | Now using e17 | e18, e19, and kde4 sucks :-/

of course nothing has been "done" because the package couldn't be downloaded. the other reinstall of "claws-mail" didn't process either of course. if i set "SHOW_MERGE_TIME=1" then it just exits without any information (no "done") of what just happened (because ibm-jdk-bin is new and has never been installed).

the file /tmp/output.log shows the whole message of the ibm-jdk-bin package and where to download it though.
is there a way to make quietemerge more friendly with such packages?

another thing is security related, called "symlink attack" (e.g. see http://www.infosecwriters.com/texts.php?op=display&id=159 ). could you please change the hardcoded TMPFILE=/tmp/output.log in line 314 to TMPFILE=`mktemp`. for users being able to find the tmp file location for debug output you'd have to output the path somewhere while running quietemerge.

of course nothing has been "done" because the package couldn't be downloaded. the other reinstall of "claws-mail" didn't process either of course. if i set "SHOW_MERGE_TIME=1" then it just exits without any information (no "done") of what just happened (because ibm-jdk-bin is new and has never been installed).

the file /tmp/output.log shows the whole message of the ibm-jdk-bin package and where to download it though.
is there a way to make quietemerge more friendly with such packages?

Very good! I think I don't have any fetch restricted files in my system which is why I never ran into this bug (I recently did a quietemerge -e world). I will update the script soon to catch such events.

Quote:

another thing is security related, called "symlink attack" (e.g. see http://www.infosecwriters.com/texts.php?op=display&id=159 ). could you please change the hardcoded TMPFILE=/tmp/output.log in line 314 to TMPFILE=`mktemp`. for users being able to find the tmp file location for debug output you'd have to output the path somewhere while running quietemerge.

That is something I was unaware of. The TMPFILE I create is created with the id of root and uses default permissions which gives only read attribute to other users. If you have another malicious user with root privileges then it is really unfortunate. I am against changing this to mktemp since it will create a bunch of files (over time) in /tmp which I want to avoid. Secondly there is also the case that the users will have to remember to read my output carefully everytime.

What I will do though is delete the file just before redirecting to it. That should mitigate any security concerns. There is a final security concern and that is what happens when your malicious root user changes the file during compilation. That is some vulnerability which can happen even with mktemp, so I don't think it is something which can be avoided.

I guess I can use mktemp for some other tmpfiles in the script. That I will do._________________emerge --quiet redefined | E17 vids: I, II | Now using e17 | e18, e19, and kde4 sucks :-/