Emdebian tdeb support

Translation packages (tdebs)

Emdebian uses a technique from
OpenEmbedded to handle translations which is becoming known within
Debian as "Tdeb" - translation debian packages. TDebs are a method for
splitting translation files out of packages and into a separate
repository, organised by language and program, allowing individual
users to only install translations files for the locales supported on
their own machine.

Debian TDebs and Emdebian TDebs

Emdebian TDebs are different to the Debian TDeb proposal
but development will progress in both types, improvements in Debian
support being implemented in the Emdebian processes.

Differences with Debian

There are more Emdebian TDebs than the equivalent Debian TDebs
and, predictably, each is smaller. For ~240 source packages, Emdebian
Crush currently creates ~1,800 TDebs compared to ~700 that would be
created under the draft specification for Debian. Further, Debian
retains that 700 for all architectures when Emdebian Crush will
add another 1,800 for each architecure supported.

TDebs are already implemented in Emdebian but Debian needs more
work to resolve the scalability issues described above.

Emdebian will use the same TDebs for Emdebian Grip and Emdebian Crush
until TDebs are supported in Debian (some time before the release
of Debian 6.0 "Squeeze").

Once TDebs are supported in Debian, Emdebian Grip will switch to
a derivative of Debian TDebs in order to gain from the improved tool
support that will be implemented in Debian once TDebs are supported.
The switch will add a few related locales to Grip installations and fold
TDebs back into the main Grip repository but the change will otherwise
be invisible.

Emdebian TDebs are currently organised one file per source package, per
locale, per architecture. Debian TDebs will be organised per source
package and contain per locale root sections within a single file.
Emdebian Grip will use Debian TDebs to create TDebs as one file per
source package per locale root.

Debian TDebs might or might not end up containing the untranslated
documentation as well as translated ones. Emdebian TDebs will not contain
any documentation.

Prior to Debian Squeeze, Emdebian Grip will use the same TDebs
as Emdebian Crush. The following table shows the status of TDebs
as expected to exist at the time of the Debian Squeeze release.

Emdebian Crush (from Lenny)

Emdebian Grip (after Squeeze)

Debian Squeeze (tool support only)

Organisation

one TDeb for one locale, per source, per architecture

one TDeb for one locale root, per source

one TDeb for one source package (with one extra, only in special
cases).

filenames

$source-locale-$lang_$version_$arch.tdeb
where $lang is pt_BR, distinct from pt

$source-tdeb-$root_$version_all.tdeb where root
includes pt_BR with pt, distinct from fr.

$source-tdeb_$version_all.tdeb where the locale
roots are separate data members within one archive.

Architecture

any

all

all

endianness

Architecture-dependent

use wrapper

use wrapper

Content

Only .mo files

Only .mo files

Translated manpages, images, documentation and .mo files
together with changelog, copyright and other package docs.

langupdate - installing TDebs

langupdate is a frontend package for
the Emdebian implementation of tdebs to handle the selection and
installation of TDebs to match supported locales and the current set of
installed packages.

Why create TDebs for Emdebian?

Disc space is not cheap

Current Debian packages collate all available translations into a
single package which can easily lead to
> 250Mb of unused translation files on a typical installation. i.e. the
translations alone could take up four times the amount of space intended
for the entire operating system. Emdebian generates a single package for every
translation of each Emdebian package, leading to a 70% reduction in
installation size but a tenfold increase
in the number of binary packages built from each source package. To solve this
scalability problem, langupdate supports a secondary sources list and secondary
apt cache so that the main apt cache can be kept as small as possible.

In Debian 6.0 "Squeeze", Debian TDebs will be supported
(actual TDebs will be in the subsequent release) but these will also
contain all translations in a single file. The Debian TDeb will be
organised to separate translation content for each locale root into
dedicated sections within the TDeb file format. Emdebian Grip will use
this functionality to separate those data components into separate
TDebs. In due course, Debian is likely to deploy similar support on
the client side to strip unwanted locale roots from downloaded TDebs.
Emdebian Grip transfers that burden to the repository, to allow for
smaller downloads to Grip devices.

Repository support

em_installtdeb is a new script in emdebian-tools based on debhelper, to create
translation packages (tdebs) for Emdebian. em_installtdeb is intended to
separate out the individual translation files from the current Debian
packages into packages without any translation files and a series of tdeb
locale packages, one per translation. Generated packages
use the syntax: $package-locale-$language_code_$version_all.deb

Once a package uses em_installtdeb, translation files
should be removed from all packages in the normal build.
em_installtdeb runs as a second build (very small, very fast)
that simply converts and packages the *.po files into tdebs.
Tdeb packages have no dependencies and no packages may depend upon them.

Each package now built for Emdebian includes tdeb support so that if
the package contains gettext-compliant translations, tdeb packages
are created. Emdebian also supports a locale repository with search support. Currently, certain
compromises are made to allow tdebs to work without changes to packages
like dpkg or reprepro - including the need to make
two uploads, one for the main package (without translations) and one
for the generated tdeb .changes file and all of the tdeb packages. For
more information on em_installtdeb, see the em_installtdeb
manpage.