>> > Ethan A Merritt writes:
...
>> > Furthermore, this would present an excuse^H^H^Hopportunity to
>> > re-work how data values are stored internally. Right now setting
>> > log-scaling on an axis causes the corresponding data coordinate to
>> > be transformed and stored as the log. This creates great headaches
>> > if you want to toggle the log-scaling and redraw the plot from the
>> > data previously read in. I think it would be much cleaner and more
>> > flexible to store the original coordinate value on input, and only
>> > apply the log- or other scaling during plot generation.
>>
>> Yes! One reason I didn't take my patch any further was that the "last
>> minute scaling" revision ought to be done first.
>
>I had come to the opposite conclusion.
>I figured to leave the existing log-scale code in place, as you
>suggested above, while adding a new more general mecanism that worked
>on the original stored coordinates. Once the general mechanism was
>in place and working, the old log-scale code could be removed without
>ever having to modify it to work with "last-minute" scaling.
I think I see how that would work:
Currently:
Upon "set log x", all stored x values are transformed and on "unset
log x" they are inverse transformed.
Phase 1:
Add a scaling "newlog".
Add to AXIS a pointer to a function that transforms the data. The
function takes two parameters: the datum to be transformed, and a
double. For scaling "linear" or "log" the function is a no-op. For
"newlog", the extra parameter is the base of the logs. Otherwise the
extra parameter is ignored.
Always do "last-minute" scaling: When plotting a value, call the
function via the supplied pointer to transform it.
On "set log x" or "unset prob x" etc., update the function pointer.
On "(un)set log x", continue to (un)transform the stored data.
Phase 2 (after initial checkout):
Rename scaling "log" to "oldlog", and rename "newlog" to "log".
Phase 3 (after a major release):
Delete the "oldlog" commands and the code to (un)transform the stored
data.
The above only implements standard scalings (linear, log, probability,
and maybe weibull). To accommodate a user-defined scaling function,
the scaling step needs to be able to point to an action table. We
could use that mechanism to implement the standard scalings too -
building the action tables from static strings like "_linear(x,b)=x"
or "_log(x,b)=log(x)/log(b)". However, I'm worried that would be too
slow. The faster method would be to give the scaling function three
parameters (datum, extra, and action table), and let the standard
scaling functions ignore the third parameter.
- Jim Van Zandt

On Saturday 31 May 2008 07:26, Allin Cottrell wrote:
> I just noticed that if a gnuplot file contains a detailed
> specification of an area to be filled with the filledcurve style,
> the metapost terminal writes out a very long "fill" line, which
> can exceed the length that mpost will process.
>
> The attached patchette arranges for such a fill line to be broken,
> in the same way that a long "draw" line is broken.
Applied to CVS
--
Ethan A Merritt

I just noticed that if a gnuplot file contains a detailed
specification of an area to be filled with the filledcurve style,
the metapost terminal writes out a very long "fill" line, which
can exceed the length that mpost will process.
The attached patchette arranges for such a fill line to be broken,
in the same way that a long "draw" line is broken.
--
Allin Cottrell
Department of Economics
Wake Forest University, NC

Since I never use the logscale toggle keys ('l' and 'L') I hadn't realized
they are broken. The problem is that coloring options have proliferated, and
many kinds of plots now use the palette for things other than Z-coordinate.
For example, consider the last plot in heatmaps.dem.
One might plausibly want to toggle log scaling on the colorbar but you can't,
because the hotkey tries to toggle Z also (or instead?)
Many other examples of confusion are possible, also in 2D now that
palette coloring can be used in 2D also.
I propose to restrict the effect of 'l' and 'L' to the X,Y axes in 2D
and the Z axis in 3D. We can add another hotkey, probably 'c', to
explicitly toggle log scaling on the colorbar in both 2D and 3D plots.
Yes, this is a change.
Is that OK with everyone?
--
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle 98195-7742

> A log file might be a good things to have. But I wonder if it would be worth aiming
> for a portable solution - although we do not have the problem on other platforms.
> (Some people would prefer to drop the windows console completly, I guess. IMHO
> we should just make it just a bit more user friendly: readline support, more drag'n drop,
> descent menu layout and a new button bar etc.)
It definitly could need it a bit polishing. :)
I don't think that one needs a portable solution. All other systems can
redirect the error stream which is more natural there (would also be my
favorite solution in windows, but this does not work). So I would leave
it in the windows part, except someone really wants it for all systems.
> The implementation for Windows via "MyPrintf" etc. would just be fine. Maybe putting the
> code in TextPutStr() (wtext.c) would be even simpler - one has to check.
In the My* functions there is still the file handle available. So I can
check if the text is addressed to stderr and log only errors. Put
apparently everything is coming via stderr, even the information at
startup. I checked via (f==stderr) like it's done in the isterm() macro.
Does someone know if this is right? Maybe it's a windows problem because
the streams do not really exist.
However since I call with a plotfile as parameter all the terminal parts
like startup information and prompt aren't written anyway.
> Btw. I would suggest -l <filename> instead of -log <filename>
that's fine by me.
Tim
--
Tim Hoffmann Raum 052
HISKP Tel: +49-228-73-2942
Universitaet Bonn Fax: +49-228-73-2505
Nussallee 14-16 hoffmann@...
53115 Bonn http://www.hiskp.uni-bonn.de/gruppen/hikari

Allin Cottrell wrote:
> On Wed, 28 May 2008, Tim Hoffmann wrote:
> wgnuplot 2>error.log
>
> That's supposed to work for win32 console programs at least.
Right, but since wgnuplot is not a console program it does not. I tried
but it creates just empty log files. As far as I could figure out, a
WIN32 GUI application silently throws away everything written to stderr!
However one can create a new console and redirect output to it. But it
appears to be impossible to redirect it to the calling console.
On the other hand, you could make it a console application and start the
window yourself. But then you'll have a console poping up when you start
the program from the menu.
I think it's the same reason why there has to be pgnuplot.exe for piped
input.
So I suppose, a log file is the best choice in this case.
Tim

On Wed, 28 May 2008, Tim Hoffmann wrote:
> I am calling wgnuplot from an external program. Since there is no error
> stream attached one cannot receive error messages. The exit code just
> indicates that there is an error but not which. Therefore it would help,
> if there was an option to write errors to a file, e.g.:
>
> wgnuplot -log error.log
What happens if you do
wgnuplot 2>error.log
That's supposed to work for win32 console programs at least.
Allin Cottrell

On Wednesday 28 May 2008, Tim Hoffmann wrote:
> Hello,
>
> I am calling wgnuplot from an external program. Since there is no error
> stream attached one cannot receive error messages.
Errors are sent to stderr, the usual error stream. In a normal operating
environment (which may exclude MSWindows) you can receive the
messages just fine.
> The exit code just
> indicates that there is an error but not which.
The current CVS version also stores the most recent error message in the
variable GPVAL_ERRMSG. You can print it to a file if you want to.
> Therefore it would help,
> if there was an option to write errors to a file, e.g.:
>
> wgnuplot -log error.log
>
> I think it should be quite simple. One could intercept the error
> messages in MyFPrintF and similar functions.
>
> Before starting to implement, I would like to have some comments on the
> idea form more experienced gnuplot developers.
I am not sure there are any such people, with regard to Windows development.
I gather that the Windows user interface (by which I mean Windows itself,
not the gnuplot GUI) is fundamentally different from the stdin/stout/stderr
model used by gnuplot and pretty much all other unix or C programs.
--
Ethan Merritt
(on the road)

Hello,
I am calling wgnuplot from an external program. Since there is no error
stream attached one cannot receive error messages. The exit code just
indicates that there is an error but not which. Therefore it would help,
if there was an option to write errors to a file, e.g.:
wgnuplot -log error.log
I think it should be quite simple. One could intercept the error
messages in MyFPrintF and similar functions.
Before starting to implement, I would like to have some comments on the
idea form more experienced gnuplot developers.
Thanks,
Tim

On Mon, 26 May 2008 01:41:31 +0200, Philipp K. Janert <janert@...>
wrote:
> The code does what all current gnuplot
> smoothing algos do: they stop at the min
> and max data point in the sample. I think
> this is reasonable.
> Best,
> Ph.
What I would suggest is that it only produces a plot line over the range
where the data is complete.
If the sample is large enough for this to be negligable , it won't notice
anyway.
If it is significant in relation to the data sample, the plot line will
stop at the point where it is no longer mathematically valid. That would
seem to be the correct thing to do.
I see little justification for extending it beyond that range.
It must be a trivial change to make if you accept the principal.
best regards, Peter.

On Mon, 26 May 2008 01:41:31 +0200, Philipp K. Janert <janert@...>
wrote:
> On Saturday 24 May 2008 00:11, you wrote:
>> On Sat, 24 May 2008 01:15:37 +0200, Philipp K. Janert <janert@...>
>>
>> wrote:
>> > I just submitted a patch (1970923) which
>> > draws a smooth histogram-like curve
>> > for a random collection of points, using
>> > a Gaussian kernel density estimation
>> > algorithm.
>> > Demos are found here:
>> > http://www.philipp-janert.com/kdensity
>>
>> very interesting.
>>
>> My initial impression on looking at your top left example is that there
>> is
>> a phase shift of +half a box in x most visible in the 0.01 and 0.05
>> plots.
>>
> The edge effect is actually in the histogram,
> not in the kernel density (yet another advantage
> of k-densities over histograms: the annoying
> bin-placement problem goes away).
hmm, never been much of a fan of bins and histograms, that's probably why.
More for sociologists and economists.
>
> The code does what all current gnuplot
> smoothing algos do: they stop at the min
> and max data point in the sample. I think
> this is reasonable.
>
Well I'm not sure that is comparable. IRRC all the "smoothing" algos
(appart from unique) are splines , these are calculated over 4 data. In
fact they would require just one point outside the data range at each end.
I have not looked how they are dealt with but it is unlikely to be
important for one point.
However, techniques using a kernel require half the kernel width outside
each end of the data range.
I would guess by looking at your examples that the missing data are
initialised as zero. Is that correct?
Sorry to be a stickler for detail, it must be my rigourous physics
training coming out. As people become less and less aware of what all
these software tools are actually doing for them, it becomes more and more
important that they do not introduce distortions.
Don't think I knocking your efforts, I'm pretty impressed overall.
best regards, Peter.
> Best,
>
> Ph.
>

There seems to be a problem with the "line sample"
that is placed in the key for all styles that involve
boxes (w boxes, w boxerrorbars, w candlesticks, ...).
For such styles, no line sample (actually: box sample)
is visible in the key. This problem is new with 4.3 and
seems to exist for all terminals (well, I tried wxt and eps).
Best,
Ph.

On Saturday 24 May 2008 00:11, you wrote:
> On Sat, 24 May 2008 01:15:37 +0200, Philipp K. Janert <janert@...>
>
> wrote:
> > I just submitted a patch (1970923) which
> > draws a smooth histogram-like curve
> > for a random collection of points, using
> > a Gaussian kernel density estimation
> > algorithm.
> > Demos are found here:
> > http://www.philipp-janert.com/kdensity
>
> very interesting.
>
> My initial impression on looking at your top left example is that there is
> a phase shift of +half a box in x most visible in the 0.01 and 0.05 plots.
>
The edge effect is actually in the histogram,
not in the kernel density (yet another advantage
of k-densities over histograms: the annoying
bin-placement problem goes away).
The code does what all current gnuplot
smoothing algos do: they stop at the min
and max data point in the sample. I think
this is reasonable.
Best,
Ph.

[Cc: to list reinstated...]
Philipp K. Janert wrote:
> How does everybody else feel about this?
I think it's quite bad. With the '=' operator in 'using' expressions,
gnuplot shouldn't even have to be changed to compute that value.
And yes, the proposed syntax is abominable. Yes, that's me speaking,
who accepted the MAXITER=0 special case eleven years ago:-)

On Sat, 24 May 2008 01:03:49 +0200, Philipp K. Janert <janert@...>
wrote:
>
> Ethan has asked me to take a look at this
> patch.
>
> I guess the purpose of this patch is to allow
> the user to enter a negative value for
> MAX_ITER, whereupon the fit command will
> calculate the residual using the starting
> values of all fit parameters and exits. (That is
> really zero iteration steps, but MAX_ITER==0
> has already been used to indicate unlimited
> iteration).
>
> I am not very happy with this patch:
> - I don't understand the purpose. Why do
> you care about the unfitted residual?
> - It leads to partially garbage output (the
> errors in the fit parameters are not
> defined for no iteration, etc)
> - We are (ab-)using the MAX_ITER parameter
> by introducing a magic value with
> non-trivial semantics (which is never
> a good idea in my mind).
>
> How does everybody else feel about this?
> Who would find this patch useful, and why?
>
> (If it is useful, can we find a better way of
> achieving the same goal, rather than shoe-horning
> semantics into a negative iteration count?)
>
> Best,
>
> Ph.
>
Hi,
I agree with your general comment , I find there is too much of this
"magic number" and syntax tweeking approach in gnuplot. It makes it very
difficult to remember and even harder to discover some very useful
functions.
It can get very aesoteric at times.
best regards, Peter.

On Sat, 24 May 2008 01:15:37 +0200, Philipp K. Janert <janert@...>
wrote:
> I just submitted a patch (1970923) which
> draws a smooth histogram-like curve
> for a random collection of points, using
> a Gaussian kernel density estimation
> algorithm.
> Demos are found here:
> http://www.philipp-janert.com/kdensity
very interesting.
My initial impression on looking at your top left example is that there is
a phase shift of +half a box in x most visible in the 0.01 and 0.05 plots.
Considering that x=0 is in the middle of the first box it appears that the
fits are responding in a way that aligns with the right of each box.
It's a bit subjective due to the nature of the data but this is my
impression for the peaks at 0.2 0.4 and 0.5
Maybe you could test this effect with a rapid change in the data.
I also think there is an egde effect at the begining and end of the data.
This is a common problem when applying this sort of technique to image
data. How to deal with edges when the kernel goes outside the data. There
are several "solutions" which involve falsely extening the data but
applying a kernel to an incompete sample range is effectively filling it
with zeros and is equally false.
It's like a running mean cannot be meaningful upto the edges of the sample
range since there are not enough samples to take the mean over. This also
gives an artificial drop off at the edges. This is also rather marked near
the origin in your lognormal example.
In image processing it's just a case of prettying up the edges but in a
scientific context this is clearly not appropriate.
I think the only rigourous way to deal with this is not to plot the part
where the data is incomplete.
I hope the comments are useful.
best regards, Peter.

On Sat, 24 May 2008 01:03:49 +0200, Philipp K. Janert <janert@...>
wrote:
>
> Ethan has asked me to take a look at this
> patch.
>
> I guess the purpose of this patch is to allow
> the user to enter a negative value for
> MAX_ITER, whereupon the fit command will
> calculate the residual using the starting
> values of all fit parameters and exits. (That is
> really zero iteration steps, but MAX_ITER==0
> has already been used to indicate unlimited
> iteration).
>
> I am not very happy with this patch:
> - I don't understand the purpose. Why do
> you care about the unfitted residual?
> - It leads to partially garbage output (the
> errors in the fit parameters are not
> defined for no iteration, etc)
> - We are (ab-)using the MAX_ITER parameter
> by introducing a magic value with
> non-trivial semantics (which is never
> a good idea in my mind).
>
> How does everybody else feel about this?
> Who would find this patch useful, and why?
>
> (If it is useful, can we find a better way of
> achieving the same goal, rather than shoe-horning
> semantics into a negative iteration count?)
>
> Best,
>
> Ph.
>
Hi,
I agree with your general comment , I find there is too much of this
"magic number" and syntax tweeking approach in gnuplot. It makes it very
difficult to remember and even harder to discover some very useful
functions.
It can get very aesoteric at times.
best regards, Peter.

I just submitted a patch (1970923) which
draws a smooth histogram-like curve
for a random collection of points, using
a Gaussian kernel density estimation
algorithm.
Demos are found here:
http://www.philipp-janert.com/kdensity
The new method has the following advantages
over the classic way of generating histograms
using "smooth frequency":
- the resulting histogram is a smooth curve,
making the effect of binning less severe
- it handles intermediate "bins" with no points
in them gracefully. (smooth freq does so
only if used "with boxes", but if you use
"with lines" for example, the line will not
drop to zero if an intermediate bin is
empty)
The method is invoked like a weighted smoothing
algorithm:
plot "data" u 1:(1):(1) smooth kdensity
where the 2nd parameter is the weight of each
point and the 3rd parameter is the bandwidth
to be used.
This patch complements the "smooth cumulative"
algorithm as another way to visualize the
distribution of a collection of random points.
Comments and suggestions are welcome.
Best,
Ph.

Ethan has asked me to take a look at this
patch.
I guess the purpose of this patch is to allow
the user to enter a negative value for
MAX_ITER, whereupon the fit command will
calculate the residual using the starting
values of all fit parameters and exits. (That is
really zero iteration steps, but MAX_ITER==0
has already been used to indicate unlimited
iteration).
I am not very happy with this patch:
- I don't understand the purpose. Why do
you care about the unfitted residual?
- It leads to partially garbage output (the
errors in the fit parameters are not
defined for no iteration, etc)
- We are (ab-)using the MAX_ITER parameter
by introducing a magic value with
non-trivial semantics (which is never
a good idea in my mind).
How does everybody else feel about this?
Who would find this patch useful, and why?
(If it is useful, can we find a better way of
achieving the same goal, rather than shoe-horning
semantics into a negative iteration count?)
Best,
Ph.

On Friday 23 May 2008 14:13, Philipp K. Janert wrote:
>
> I have applied and closed the patches:
> 1962130 (smooth cumulative)
> 1827826 (additional dgrid3d kernels)
>
> I suggest we also close patch
> 632289 (dgrid3d: scaling and range)
> because it is by now over 5 years out
> of date and from what I can see has
> been superseded by the just applied
> patch 1827826. Any comments or
> objections?
OK by me.
I'm sure it's not the only patch that is out of date.
Ethan
--
Ethan A Merritt

I have applied and closed the patches:
1962130 (smooth cumulative)
1827826 (additional dgrid3d kernels)
I suggest we also close patch
632289 (dgrid3d: scaling and range)
because it is by now over 5 years out
of date and from what I can see has
been superseded by the just applied
patch 1827826. Any comments or
objections?
Best,
Ph.

Hi,
On Thu, 22 May 2008, Ethan Merritt wrote:
[[about the PBM driver]]
> > -- I still use it fairly often
> > to generate individual frames for encoding into a movie with ppmtompeg
> > (which only groks a limited set of native input formats: PPM, PNM,
> > YUV, JPEG, and JMOVIE, but *not* PNG).
>
> That's not much of a reason.
> You could use the far more featureful png driver and interpose
> a png->pnm filter in the output:
> set term png truecolor enhanced font "verdana,11"
> set output '| convert png:- mypic.pnm'
Good idea, I'll play around the next time I make a movie. The
only problem is that crunching 3000 frames is already a cpu- and
disk-intensive process, and firing up ImageMagick for each frame
would make it considerably worse. What might be better would be
to do a pnmtopnm (part of netpbm) conversion on each frame, since
I already have a pnmchange (also part of netpbm) conversion there.
The netpbm programs are a lot lighter-weight than ImageMagick.
ciao,
--
-- Jonathan Thornburg (remove -animal to reply) <J.Thornburg@...>
School of Mathematics, U of Southampton, England
"Washing one's hands of the conflict between the powerful and the
powerless means to side with the powerful, not to be neutral."
-- quote by Freire / poster by Oxfam

On Thursday 22 May 2008 13:30, Jonathan Thornburg wrote:
> On Thu, 22 May 2008, someone whose nested quoting has overflowed
> my mental stack :) wrote:
> > The bitmap terminals are pretty ancient code at this point.
> > Other than special devices like your DPU gadget, there
> > isn't any incentive to use or modernize them. The PBM
> > driver was at one point a reasonable output path, but for
> > years now PNG has been better for any purpose I can think
> > of.
>
> I hope we keep the PBM driver around
I wasn't proposed to dump it. I was just trying to prioritize where
development effort is spent. There are plenty of things to work on
that will benefit many terminal types (new plot modes, continued work
on internationalization, updated numerical routines) and other
terminal types with more potential users (cairopdf).
> -- I still use it fairly often
> to generate individual frames for encoding into a movie with ppmtompeg
> (which only groks a limited set of native input formats: PPM, PNM,
> YUV, JPEG, and JMOVIE, but *not* PNG).
That's not much of a reason.
You could use the far more featureful png driver and interpose
a png->pnm filter in the output:
set term png truecolor enhanced font "verdana,11"
set output '| convert png:- mypic.pnm'
--
Ethan A Merritt