Search Results: "tfheen"

25 March 2008

This post is both an attempt at replying to a bug against
telepathy-glib, but also an attempt at explaining what
Requires.private do (and don't).
I am using Evolution as my example here, not to pick on Evolution or its
authors in any way, but because it's a convenient example. Currently,
on Ubuntu Hardy, evolution links against 75 different libraries.
Amongst those, we find libz.so.1, libXinerama.so.1 and many more.
I'll go out on a limb here and claim that Evolution does not call any of
the functions in libXinerama directly. Let that be the assumption from
here on.
An obvious question then is, why does evolution link against
libXinerama.so.1 if it doesn't use it? To answer that question, we
need to go back in time to before we had dynamic linking. If you wanted
to build a binary like evolution you had to have 75 -l statements
when you linked and you ended up with the whole code for Xinerama
embedded in your email and calendar client. For various reasons, we
stopped doing that and switched to dynamic linking where the evolution
binary just contains a reference to libXinerama. At some point we
also grew the ability for libraries to contain those references to other
libraries, so you don't have to hunt down all the dependencies of
libfoo when you are linking with it. We also got tools such as
libtool which try to abstract away a lot of the problems of building
on older platforms which don't support inter-library dependencies.
Now, since evolution still doesn't use anything directly in
libXinerama.so.1 but just uses a library which in turn links against
libXinerama.so.1, it shouldn't be linking against it. Then why is it
linked with it? Again, we need to look back at history, and for this
part I am at least partially responsible.
pkg-config was originally written as a replacement for gnome-config
and various other -config utilities. Lots of libraries and
applications now ship .pc files and we have a standardised interface
for querying those files. One of the problems the original authors of
pkg-config faced was the problem of dependencies. They added
dependencies so the authors of gst-python-0.10 could say "We need
pygtk-2.0 too" and so the compilation flags needed for gst-python-0.10
would also include those for pygtk-2.0. Note that I'm using
"compilation flags" loosely here, I am not just talking about CFLAGS.
This did not fix the problem of inflated dependencies. Not at all. I
talked with some of the Debian Release Managers back in 2004/2005 and
we worked out a solution which should help us have correct, uninflated
dependencies since the then-current way of handling dependencies caused
big problems for migrations of packages between unstable and
testing.
The plan was to introduce a new field, Requires.private which would
not show up unless you passed --static to the pkg-config invocation
(since you need all libraries if you are linking statically). This
definition of Requires.private was mostly useless since GNOME and GTK+
have a habit of including each other's headers. To make a long story
short, I changed the semantics so the Cflags field from private
dependencies were included even when not linking statically.
A problem which pkg-config does nothing to guard against in this case
is if you have libfoo.so.2 linking against libbar.so.1 and
libfoo.so.2 exports some of libbar's types in its ABI (and not just as
pointers, but actual structs and such). If libbar's soname is then
bumped to libbar.so.2 and libfoo is rebuilt, libfoo's ABI has changed
without a soname bump. This is bad and will cause problems. If your
application is linked against both libfoo.so.2 and libbar.so.1,
you'll still get problems since libfoo.so.2 then suddenly pulls in
symbols from libbar.so.2. If you used symbol versioning, you would at
least not get symbol conflicts and your application would continue to
work, but you would have a spurious dependency and the package
containing libbar.so.1 would be kept around until your application was
recompiled.
With this background, you might ask the question why we still have
Requires since it is seemingly useless. For C, it is useless in all
but the most special cases, just use Requires.private instead (and its
sibling Libs.private). Other languages have different semantics.
Some people use .pc files for other purposes such as
gnome-screensaver having variables defining where themes and
screensavers go.
Hopefully this blog post has explained a bit about why we have
Requires.private and what the difference between this and their
regular counterpart is. If there's anything unclear, please do not
hesitate to contact me.

9 January 2008

I am currently working on implementing a cryptographic file system using
FUSE. It is different from EncFS and similar in that it just
mirrors a normal directory tree, but encrypts the contents of the files
as they are read or decrypted as they are written.
My use case is backups. I have some machines where I and only I have
access, machines which may contain proprietary information, personal
emails and so on. Of course, I want backups of those, so when the hard
drives stop working, I don't lose any data. The machine(s) I am backing
up to, however are not always machines where I trust all the people with
physical access to not make a copy of my data. In addition, I don't
want broken hard drives returned under warranty to contain unencrypted
data. This use case is the reason for why I'm encrypting on read rather
than on write.
I have chosen to use CTR (counter) mode together with AES which should
give acceptable security. One of the requirements CTR needs to work
well is a nonce, typically 64 bits (for 128 bit AES) which must not ever
be used twice. If you use it twice, you leak information about your
plaintext, which is, for obvious reasons, bad.
My current design headache is how to choose a good nonce. Ideally, I
believe it should be persistent for each version of the file and unique
per file. Using the inode number takes up 64 bits (on AMD64 at any
take, or when using -D_FILE_OFFSET_BITS=64 on 32 bit platforms). So
while this gives me the latter, it doesn't give me the former at all. I
am wondering if I should use the inode number modulo 2^32 (effectively
choosing the lower 32 bits of the inode number) and then something which
is fairly sure to never be the same, such as mtime (or at least the
lower 32 bits of it, when time_t becomes 64 bit). The reason for not
just choosing a completely random value is I don't want a command like
diff file1 file1 to claim there are differences in the file.
My hope was I'd get a great idea on how to solve the problem as part of
writing it down. Alas, that hasn't happened, so if you happen to come
across a great solution (or a reason to avoid a particular choice), feel
free to email me

10 October 2006

People tend to just ping me on IRC, which is annoying and useless. If
people say something like tfheen: I have this problem with
blah. Do you know a workaround? I can just respond later when I'm
awake or around. Instead, people are going: tfheen: ping? and I
pong five hours later and they're not around. Annoying for all parts.
To counter this, I have now written a small irssi script which
responds to contentless pings with "You sent me a contentless ping.
This is a contentless pong. Please provide a bit of information about
what you want and I'll respond when I am around."
The script is available.

For those of you wondering why there is no editor on the list; I use
emacs and it's in the 11th place (with 1903 runs). I tend to let it run
for a while and open more than one file, so it doesn't get that high on
the list. I also cd and ls a lot.

19 September 2006

I am currently working on something which hopefully end up being a good
calendar server. The goal is to make it easy to add new frontends such
as a CalDAV frontend, a Web frontend, etc.
So far, I am just working on getting the data model right. The
iCalendar RFC has a data model which I am trying to formulate
into SQL, something which is ending up being quite hard. Shannon
Clark blogs about why calendars are hard, and all of
those issues are issues I am running up against when trying to make the
data model.
The main problem seems to be related to time zones, more closely: "How
do you store timezone information in your database". The easy (but
wrong) solution is to just normalise everything to UTC. This will not
handle the case of events crossing timezone changes, such as to or from
daylight savings.
What I am currently considering is making sure all data in the database
is stored in UTC, but also store the original time zone information.
While a bit more complex, this allows applications to get back the same
timezone they put in (I am not convinced Evolution will be happy if the
timezones are renamed, for instance) and it allows my application to
generate recurring events properly. My main issue with this is having
the application responsible for making sure the data in the database
makes sense, but without putting half my app in PL/perl, I can not see a
way around that.
Oh, and if anybody has great ideas on how to represent timezones in a
relational database (postgres), please do mail me or grab me on IRC or
something similar.

10 January 2006

Everybody who has used an Ubuntu live cd over the last nine months or so
has used casper. It started out as a special udeb, called by the
debian-installer code to bootstrap a live environment. While d-i is
fairly flexible, this was stretching the limits and not really a great
solution. Amongst the problems were user interactivity halfway through
the boot and a very slow boot.
In the middle of December, mdz asked me if I could take a look at
implementing the
SimplifiedLiveCD
specification. As I had played a bit with casper already, I did.
Casper is nothing like what it used to be, it now uses initramfs, so no
user interactivity after the bootloader. It uses unionfs where
available, which speeds it up a fair bit
(compare to
devmapper + cloop), and if the cd image has
squashfs, it uses that too, which makes it even
faster. Boot time improvements from around 368 to
about 231 seconds is fairly good, but I hope to get it even lower.
What I really, really like about casper however is how hackable it is.
I added cd integrity check in less than a day (modulo some bugs in
usplash I had to fix). Today, I integrated it with the new usplash in
initramfs, so we actually have progress in the initramfs as well.
(Instead of "mounting root file system" taking about 40 seconds.)
Another neat feature is the persistence support. It will now look for
filesystems with the label casper-cow (that will be changed to
ubuntu-live-rw, I think) if persistent is seen on the kernel command
line. This makes it easy to drag your setup around with just an USB key
and any Ubuntu live cd.
Next out is getting keyboard selection better and more speedups.