Hi,
I am trying to rewrite userland programs (especially daemons) to
support userland DTrace probes to make it possible to trace the
behavior by using dtrace(1). The purpose is to provide a consistent
interface to enable/collect debug log and show internal states. A
lot of daemons define their own debug output function like dprintf()
and there is no consistency of where to be recorded.

I would like your comment about this change because I want to know
whether this change (or direction) is acceptable or not. I put an
example by using syslogd at the following URL:
(diff)
https://people.freebsd.org/~hrs/syslogd_usdt.20161220-1.diff
(tarball)
https://people.freebsd.org/~hrs/syslogd_usdt.20161220-1.tar.gz
You can try to compile a new syslogd, run it, and then attach
dtrace(1) to the syslogd process by "dtrace -q -CI./
-s ./syslogd_trace.d -p `pgrep syslogd`" in the same directory.
Basically this change is invisible for normal users. This includes
some rough edges but I think it is sufficient to understand the
concept. I do not intend to commit this soon.
Questions from me are the following:
1. Where should foo_probes.d and foo_trace.d be installed? And if
they depend on foo.h, where should foo.h be?
2. Is documenting probes into foo.8 reasonable?
The details are as follows.
* Rewrite example
This rewrite is twofold. First, simply replace a call of the
logging function into DTrace USDT like this:
(old) | dprintf("an error occurred, errno=%d\n", errno);
(new) | FOO_LOG(LOG_ERR, "an error occurred, errno=%d\n", errno);
And then if it involves a result of a specific operation, replace
trace probes for the simple logging with dedicated ones:
(old) | error = bind(s, res->ai_addr, res->ar_addrlen);
| if (error) {
| dprintf("bind failed, errno=%d\n", errno);
| close(s);
| return (-1);
| }
(new)
| error = bind(s, res->ai_addr, res->ar_addrlen);
| if (error) {
| FOO_SOCK_BIND_FAILED(errno, s, res->ai_addr, res->ar_addrlen);
| close(s);
| return (-1);
| }
| FOO_SOCK_BIND_SUCCESS(s, res->ai_addr, res->ar_addrlen);
and implement the functionality of the original log message in D
script:
| foo$target:::sock-bind-failed
| {
| printf("bind failed, errno=%d", arg0);
| /* The other argN can be used to report in more detail. */
| }
* Additional files in the existing directory layout
After this rewrite, the directory layout for a daemon "foo" will look
like the following:
| Makefile
| foo.8
| foo.c
| foo.h
| foo_probes.d
| foo_trace.d
foo_probes.d and foo_trace.d are added. The former is the
definition of USDT probes, and the latter is a D script example to
reproduce the original debug log by dprintf() or something like
that. A section to describe what probes are available is added into
foo.8. One can trace the foo daemon using "dtrace -Cs foo_trace.d -p
`pgrep foo`" on runtime, and also can create own script.
foo.h may be added because foo_probes.d and foo_trace.d often require
information of data structure used in foo.c.
* Possible incompatible change
A debug flag to activate additional logging is no longer necessary
after this rewrite, so we can remove it (-d flag in the case of
syslogd). And dump of the internal state can be implemented as a
SIGINFO handler. In the syslogd example, SIGINFO dumps syslogd
configuration and access control list.
-- Hiroki