GnuCash CLI

GnuCash CLI

OK, I've got a framework for the business objects inside the command-line
interface from Pilot-QOF / QOF-generator. Accounts, Trans, Lots, etc. will
follow.

I'm using it as a test bed for the remaining problems in QOF/QSF; namely
QOF_TYPE_COLLECT, QOF_TYPE_CHOICE and maps.

Notes:
1. This will be G2 only - it does NOT read the current GnuCash XML. Data will
need to be exported (which is why I wrote it in the first place).

2. This is for data-mining using SQL and it outputs more QSF XML which can be
used to make your own reports, invoices, print-outs, calculations,
abstractions, databases .... whatever you like. You can use XSL, Perl, PHP,
whatever you fancy.

3. The data itself can always be updated from more recent G2 exports - just
save the SQL statements in a file and run them again.

4. The QSF XML itself will be importable back into GnuCash (G2) too.

5. So far, I've called it "cashutil". (I was short on inspiration).

6. Remaining problem is gncCommodity which doesn't work with QOF. (yet)

7. There's limited financial logic in this program - you cannot create data
that is incompatible with the *object* but you might be able to create data
that is financially inconsistent - non-existent accounts are the most
obvious. GnuCash (via qof_book_merge) will deal with problems like that when
merging the data back into it's own files.

The question is:

Is this a *new* project (SF), a new program and a new package?
(personally, I think it may be better standalone).

Or should it be folded into the GnuCash G2 tree where it will always have the
same object definitions as the rest of GnuCash? (not a big problem, really).

It'll be useful for some, maybe even many, but does it merit being installed
for everyone? If it's left on it's own, will people shy away from using it?

It's dependencies are:
libxml2 >= 2.5.10
glib2
qof1 (which will be the package name for the QOF 0.6.0 source tree).

Umm, actually, that's it. i.e. the same as QOF. It does not depend on GnuCash
- strange as that may sound. I don't see what you'd do with it without
GnuCash but you never can tell what users will get up to. :-))

It builds it's manpage using xsltproc, it is gettext ready and it compiles on
GNU/Linux and MacOSX (almost).

There's work left to do - the objects are in but not fully functional - but it
can be done. (Everything *except* the objects is already working.)

/opt/working/cashutil/src$ ./cashutil -l

cashutil currently supports these object names:
You can use the names with cashutil -d
and in SQL queries (as the table name) with cashutil -s|f
Descriptions are shown only for readability.

Use '-d <database> --explain' to see the list of fields within
any supported database.

Thank you for using cashutil

:-)

Once G2 is out, I'm going to look at allowing QOF to accept objects at
run-time using shared libraries (as it does with backends), instead of
writing a new program for each collection of objects. One front-end, many
objects. There will still be a need for bigger programs to keep their own but
these small utility programs could get out of control. I'm also going to need
some form of object version control with more fine tuning than
QOF_OBJECT_VERSION.

Note: This is not a formal release and the tarball is not signed. Feel free to
inspect the code. (manpage included).

It won't run against existing GnuCash CVS yet but this should be sorted out
with my next commit - the changes are relatively minor.

> The question is:
>
> Is this a *new* project (SF), a new program and a new package?
>
> Or should it be folded into the GnuCash G2 tree where it will always have
> the same object definitions as the rest of GnuCash?

> It does not depend on
> GnuCash - strange as that may sound. I don't see what you'd do with it
> without GnuCash but you never can tell what users will get up to. :-))

> There's work left to do - the objects are in but not fully functional - but
> it can be done. (Everything *except* the objects is already working.)

The objects should be working now too - the only changes from current GnuCash
source for the objects are engine events and gncCommodity (both commented out
for now simply because I haven't included those files in the tarball).

From a previous thread, Mon Jul 4 13:39:32 EDT 2005
https://lists.gnucash.org/pipermail/gnucash-user/2005-July/014153.html> > Yes, a quick CLI to just add transactions would be nice (eg one gets
> > home from 'shopping' with a pile of receipts - it would be nice to
> > quickly enter these transactions without having to fire up the whole
> > GUI. CLI access for various 'common' queries (current balances,
> > projected minimum, etc.) would also be nice.
>
> You're not the first to request this. Indeed there's a relatively
> long-standing RFE on this topic. I'm hoping that the QSF import
> will help on this ...

This CLI is currently SQL biased (supporting SELECT and INSERT) - what kind of
interface would be best for a CLI that would close the RFE?

I can extend this to run under ncurses and provide simple menus or Q&A data
entry, it could use defaults of some kind.

We can have something that looks like xf86config, or something like debconf
(although that could take longer) or ....

It could load a previous file, add transactions to the partial book and save
it out. Merging a partial book from one file is quicker than lots of small
ones. Users can keep the QSF as a backup and start a new file or overwrite
the old one.

Once some defaults are configured, it could be launched as an alias or bash
script to remove the need for lots of options on the command line.

If it's extended in such ways, would it be better within GnuCash than
separate?

Re: GnuCash CLI v0.0.1

I must admit that I haven't had the time to read through all
of this. I'm not sure if it's something that should be part
of gnucash or not. part of me thinks "yes" and part of me thinks
"no"..

If it's really targetting at gnucash users, to provide a CLI
to adding new items, then it might well deserve to be part of
gnucash. However the part of me that thinks "no" is the fact
that it's not modifying the gnucash backend directly, but rather
it's creating a QSF file that you later have to import.

> First tarball now available - needs v.latest QOF CVS.
>
> http://code.neil.williamsleesmill.me.uk/cashutil.tar.gz>
> http://www.linux.codehelp.co.uk/#cashutil>
> Doxygen output: http://code.neil.williamsleesmill.me.uk/cashutil/>
> Note: This is not a formal release and the tarball is not signed. Feel free to
> inspect the code. (manpage included).
>
> It won't run against existing GnuCash CVS yet but this should be sorted out
> with my next commit - the changes are relatively minor.
>
>> The question is:
>>
>> Is this a *new* project (SF), a new program and a new package?
>>
>> Or should it be folded into the GnuCash G2 tree where it will always have
>> the same object definitions as the rest of GnuCash?
>
>> It does not depend on
>> GnuCash - strange as that may sound. I don't see what you'd do with it
>> without GnuCash but you never can tell what users will get up to. :-))
>
>> There's work left to do - the objects are in but not fully functional - but
>> it can be done. (Everything *except* the objects is already working.)
>
> The objects should be working now too - the only changes from current GnuCash
> source for the objects are engine events and gncCommodity (both commented out
> for now simply because I haven't included those files in the tarball).
>
> From a previous thread, Mon Jul 4 13:39:32 EDT 2005
> https://lists.gnucash.org/pipermail/gnucash-user/2005-July/014153.html>> > Yes, a quick CLI to just add transactions would be nice (eg one gets
>> > home from 'shopping' with a pile of receipts - it would be nice to
>> > quickly enter these transactions without having to fire up the whole
>> > GUI. CLI access for various 'common' queries (current balances,
>> > projected minimum, etc.) would also be nice.
>>
>> You're not the first to request this. Indeed there's a relatively
>> long-standing RFE on this topic. I'm hoping that the QSF import
>> will help on this ...
>
> This CLI is currently SQL biased (supporting SELECT and INSERT) - what kind of
> interface would be best for a CLI that would close the RFE?
>
> I can extend this to run under ncurses and provide simple menus or Q&A data
> entry, it could use defaults of some kind.
>
> We can have something that looks like xf86config, or something like debconf
> (although that could take longer) or ....
>
> It could load a previous file, add transactions to the partial book and save
> it out. Merging a partial book from one file is quicker than lots of small
> ones. Users can keep the QSF as a backup and start a new file or overwrite
> the old one.
>
> Once some defaults are configured, it could be launched as an alias or bash
> script to remove the need for lots of options on the command line.
>
> If it's extended in such ways, would it be better within GnuCash than
> separate?
>
> Once this is decided, I can get the rest sorted - like a decent name!
>
> --
>
> Neil Williams
> =============
> http://www.data-freedom.org/> http://www.nosoftwarepatents.com/> http://www.linux.codehelp.co.uk/>
> _______________________________________________
> gnucash-devel mailing list
> [hidden email]> https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: GnuCash CLI v0.0.1

On Tuesday 12 July 2005 6:17 pm, Derek Atkins wrote:
> If it's really targetting at gnucash users, to provide a CLI
> to adding new items, then it might well deserve to be part of
> gnucash.

This will be 100% targetted at GnuCash users - yet how many users will use it
regularly? Also, it can be more than *just* a data entry point for GnuCash -
it's built around queries and data manipulation - a basis for data mining and
customised data handling / reporting.

There are plenty of packages that have "recommended" or "optional" add-on
utilities. There aren't many that I can think of that have BOTH GUI and CLI
programs within one package - beyond config and help switches.

> However the part of me that thinks "no" is the fact
> that it's not modifying the gnucash backend directly, but rather
> it's creating a QSF file that you later have to import.

1. Direct manipulation of existing data files significantly increases the
testing burden for this CLI. It is far easier to spot simple errors in a GUI
display than to create a CLI display that can do the same.

2. Some users' data files get v.large - loading in a CLI is not going to be a
whole lot faster than a GUI (once the GUI is started), that may defeat the
purpose of the CLI.

Having the output as a separate file under user control is a less risky
proposition, IMHO, as it can use the checks and balances of importing to
catch errors that CLI users may not spot. Provided CashUtil simply adds
incrementally to a single file, it shouldn't be much hassle to import the
file when there is time.

Packages can readily suggest related packages, it's just a case of asking each
package maintainer to mention that cashutil is a CLI for GnuCash.

The more I think about it, the more I think it will work best as a separate
unit. I'll setup an SF project for cashutil at the weekend and bug reports /
queries can be handled on the QOF-devel mailing list and SF pages, as with
Pilot-QOF. It should be available as a package in time for G2.

GnuCash isn't a database, it doesn't have a database but it can and will
continue to use databases for storage as we see fit. The explanation took up
most of that month, we really don't want to go back there!

It's not as easy as you think to separate the data from the logic without
losing all the meaning of the data.

Re: GnuCash CLI v0.0.1

On Tue, 2005-07-12 at 22:47 +0100, Neil Williams wrote:
> However, don't get the impression that there are Perl bindings or Java methods
> available - the most we'll have even with G2 is generic XML:
> http://code.neil.williamsleesmill.me.uk/qsf.html>
Are Java bindings wanted ? I am willing to implement them but as I
remember from the devel-list discussions another language implementing
some gnucash logic is not such an agreed idea :) ... so, this is why I
ask this.

Re: GnuCash CLI v0.0.1

For the record, I responded to Neil offline on purpose. I do _not_ desire a
direct API to business logic, I was simply responding informally to Neil's
comment below.

Dan W.

On Tue, Jul 12, 2005 at 09:37:01PM +0100, Neil Williams wrote:
> On Tuesday 12 July 2005 6:17 pm, Derek Atkins wrote:
> > If it's really targetting at gnucash users, to provide a CLI
> > to adding new items, then it might well deserve to be part of
> > gnucash.
>
> This will be 100% targetted at GnuCash users - yet how many users will use it
> regularly?
_______________________________________________
gnucash-devel mailing list
[hidden email]https://lists.gnucash.org/mailman/listinfo/gnucash-devel

langauge bindings [WAS: Re: GnuCash CLI v0.0.1]

On Wed, 2005-07-13 at 08:27 +0300, Daniel Tudosie wrote:
> On Tue, 2005-07-12 at 22:47 +0100, Neil Williams wrote:
> > However, don't get the impression that there are Perl bindings or Java methods
> > available - the most we'll have even with G2 is generic XML:
> > http://code.neil.williamsleesmill.me.uk/qsf.html> >
> Are Java bindings wanted ? I am willing to implement them but as I

You're welcome to contribute whatever you like, especially if you're
willing to support it. :) Frankly, though, I'd like to see a specific
use for the bindings before/alongside them; random bindings that aren't
being used isn't a win.

> remember from the devel-list discussions another language implementing
> some gnucash logic is not such an agreed idea :) ... so, this is why I
> ask this.

Right now we use g-wrap to bridge C and scheme; it seems like it would
be better to do two things:

1/ reduce the prevelance of scheme in the gnucash core

2/ use swig rather than g-wrap to provide external bindings in many
languages, if needed

It would be further better to move a lot of the logic, which is tightly
coupled to the UI, down into a layer just above the current "engine",
and expose it via an API.

Re: langauge bindings [WAS: Re: GnuCash CLI v0.0.1]

On Wed, 2005-07-13 at 13:55 -0400, Josh Sled wrote:
> > Are Java bindings wanted ? I am willing to implement them but as I
>
> You're welcome to contribute whatever you like, especially if you're
> willing to support it. :) Frankly, though, I'd like to see a specific
> use for the bindings before/alongside them; random bindings that aren't
> being used isn't a win.
>
I totally agree on this one; I just asked because Java was mentioned and
I assumed that Neil has thought of some use for them; but I am also
working in C/C++ and Scheme is not a just strange word ;-) - so I will
contribute to the existing code/implementation (when I will have
something to contribute with)

However, I realize that I was not so clear... I do not want to
reimplement existing logic in another language but providing an
interface would be nice... of course: if there is something meaningfull
to use it for (e.g. a Java binding could be used for web interface either applets
or servlets, JSP)

> It would be further better to move a lot of the logic, which is tightly
> coupled to the UI, down into a layer just above the current "engine",
> and expose it via an API.
>
This would be great... I think it was discussed before and the
conclusion was that is not that easy, it needs some serios thought and
engineering to make a more better/robust/etc architecture...
However, a time will come when this should be done
>
> ...jsled
>
Regards,
Daniel

Re: GnuCash CLI v0.0.1

On Wed, Jul 13, 2005 at 06:39:31PM +0100, Neil Williams wrote:
> On Tuesday 12 July 2005 10:47 pm, Neil Williams wrote:
>
> Sorry about the repeated emails - something went wrong at my ISP's SMTP
> server. The headers show I sent it once but it got re-sent by the first SMTP
> connection.

Oh, that's what it was? I just thought you were trying to make your
points really, really, really, really, really clear. :)

Re: langauge bindings [WAS: Re: GnuCash CLI v0.0.1]

> On Wed, 2005-07-13 at 13:55 -0400, Josh Sled wrote:
> > > Are Java bindings wanted ? I am willing to implement them but as I
> >
> > You're welcome to contribute whatever you like, especially if you're
> > willing to support it. :) Frankly, though, I'd like to see a specific
> > use for the bindings before/alongside them; random bindings that aren't
> > being used isn't a win.
>
> I totally agree on this one; I just asked because Java was mentioned and
> I assumed that Neil has thought of some use for them;

Sorry, a little background might help. I started on the GnuCash code to
automate the creation of my daily invoices, using data from my Palm PDA. That
quickly showed that I'd need to also get into the pilot-link code and this is
where I have come across language bindings - Java, Perl and Python. I didn't
mention Python because I've never learnt it and haven't understood it even
when I've read it! (A bit like my reaction to Scheme.)

So I mentioned language bindings - using Java only as an example - not because
I had any GnuCash role in mind but because I had seen such bindings being
used by others in pilot-link.

> but I am also
> working in C/C++ and Scheme is not a just strange word ;-) - so I will
> contribute to the existing code/implementation (when I will have
> something to contribute with)

You'll be very welcome here.

> However, I realize that I was not so clear... I do not want to
> reimplement existing logic in another language but providing an
> interface would be nice... of course: if there is something meaningfull
> to use it for (e.g. a Java binding could be used for web interface either
> applets or servlets, JSP)

That would compliment the current work very well, IMHO.

> > It would be further better to move a lot of the logic, which is tightly
> > coupled to the UI, down into a layer just above the current "engine",
> > and expose it via an API.

A clearer dividing line between GnuCash and QOF is appearing (at least to me)
as part of the CLI and I am cleaning up a few areas so that GnuCash can use
QOF more like other QOF programs - with the aim of GnuCash using QOF as an
external library sometime after G2 - removing all qof*.c|h from src/engine
and src/backend/ and ancillary files.

The CLI itself may illlustrate the kind of logic required and how it could be
implemented.

On the CLI, I'm implementing a 'shell' mode based on a pilot-link example to
ease data entry with add [entity], edit [entity], load [book], write [book],
sql (possibly interactive) and merge [book] functionality. This will be done
using nested menus, ala GnuPG - gpg --edit-key or gdb although actually based
on code from pilot-link (dlpsh.c). I'll also investigate a possible
'recording' idea that shell commands could be exported as SQL to be run again
at a later date. I'm quite confident that the CLI can have at least Perl
language bindings - Java and Python would need someone else's skill but
there's no reason to think they wouldn't work. Shell mode should be ready for
the first release, the rest will keep me occupied after G2.

> This would be great... I think it was discussed before and the
> conclusion was that is not that easy, it needs some serios thought and
> engineering to make a more better/robust/etc architecture...
> However, a time will come when this should be done

I agree and I don't think it's as hard as it may appear. It's non-trivial and
must be carefully planned, but I don't think it's something we can afford to
let lie.

G2 is clearly everyone's priority - (I'm only writing the CLI now because
it'll help me finalise my new code in QOF and therefore G2).