Karel Zak's blog

Thursday, February 15, 2018

This release (rc1 now) is without dramatic changes and game-changing improvements.

We have again invested our time and love to make cal(1) more usable. The most
visible change is possibility to specify calendar system.

The current
(backwardly compatible) default is to use Gregorian calendar and Julian
calendar for dates before September 1752 (British Empire calendar reform). Unfortunately, this default is pretty frustrating if you want
to use cal(1) for old dates before 1752 and you don't want to follow UK calendar.

The new command line option --reform={Julian,Gregorian,iso,1752,...} allows to
specify exclusively calendar system or reform date. The currently supported
reform is only UK reform in 1752. In the next versions we will probably add
support for another reforms as it's very region specific (for example 1584 in
my country, 1873 in Japan and 1926 Turkey, etc.).

Linux kernel supports multi-line log messages. Unfortunately, dmesg(1) support
for this feature was insufficient. Now dmesg(1) provides better support and by
new command line option --force-prefix allows to to print facility, level or
timestamp information to each line of a multi-line message.

The command fallocate(1) --dig-holes has been significantly improved and it's
faster more effective now (thanks to Vaclav Dolezal).

The command lscpu(1) provides more details about ARM CPUs now.

The command lsmem(1) supports memory zones now.

The command lsns(8) provides netnsid and nsfs columns now. ip(1) command allows
to create network namespace, add logical name and ID for the namespace. Now all is visible by lsns(8). For example copy & past from our regression tests:

The command rtcwake(8) has been improved to wait stdin to settle down before entering a system
sleep. This is important on systems where wireless USB devices (mouse,
keyboard, ...) generate "noise" for fraction of a second after rtcwake(8)
execution.

The library libblkid has been extended to support LUKS2, Micron mpool, VDO and
Atari partition table.

Thanks to all 43 contributors!

The next release v2.33 is planned for May 2018 (yes, the goal is to have 3-4
releases per year rather than 2 releases like in last years).

The command su has been refactored and extended to create pseudo terminal for
the session (new option --pty). The reason is CVE-2016-2779, but the issue
addressed by this CVE is pretty old and all the problem is silently ignored for
for years on many places (on only su(1)). The core of the problem is that
unprivileged user (within su(1) session) shares terminal file descriptor with
original root's session. The new option --pty forces su(1) to create
independent pseudo terminal for the session and than su(1) works as proxy
between the terminals. The feature is experimental and not enabled by default
(you have to use su --pty).

rfkill -- this is a new command in util-linux. The command was originally
written by Johannes Berg and Marcel Holtmann and maintained for years as
standalone package. We believe that it's better to maintain and distribute it
with another commands on one place. The util-linux version is backwardly
compatible with the original implementations. The command has been also improved
(libsmartcols ouotput, etc.), the new default output:

I can imagine system where for example per-user or per-architecture partition
UUIDs are based on this system. For example use UUID specific for the system root as --namespace and username as --name, or so.

wipefs and libblkid have been improved to provide all possible string
permutations for a device. It means that wipefs does not return the first
detected signature, but it continues and tries another offsets for the
signature. This is important for filesystems and partitions tables where the
superblock is backuped on multiple places (e.g. GPT) or detectable by multiple
independent ways (FATs). This all is possible without a device modification
(the old version provides the same, but only in "wipe" mode).

The libfdisk has been extended to use BLKPG ioctls to inform the kernel about
changes. This means that cfdisk and fdisk will not force your kernel to reread
all of the partition table, but untouched partitions may remain mounted and used
by the system. The typical use-case is resizing the last partition on the system
disk.

You can use cfdisk to resize a partition. Yep, cool.

The hwclock command now significantly reduces system shutdown times by not
reading the RTC before setting it (except when the --update-drift option is
used). This also mitigates other potential shutdown and RTC setting problems
caused by requiring an RTC read.

Friday, June 9, 2017

The command tailf is dead thing. (RIP ... years ago I had nice time to improve it with inotify:) You have to use "tail -f" from coreutils project.

blkzone -- this new command is excellent example of the open source collaboration. The command has been developed by people from WD, Seagate and SanDisk (thanks to Shaun Tancheff, Damien Le Moal and others). The goal is to have command line interface to run zone commands on block devices that support Zoned Block Commands (ZBC) or Zoned-device ATA Commands (ZAC). For now the supported zone commands are "reset" and "report". See http://www.storagereview.com/methods_of_smr_data_management for more details about zones.

fincore (file in core)-- this is nice useful command to get information about
number of memory pages used by file content. For example my fulltext email DB:

lsmem (list memory) and chmem (change memory) -- another new commands. The commands have been originally
implemented in Perl for s390-tools, now re-implemented in C in more generic way
and to be usable on another architectures too. (thanks to Clemens von Mann and Heiko
Carstens from IBM.)

The command fallocate supports an "insert range" operation now.

We continue on hwclock cleanup, some things in the code have been simplified,
dead and useless things removed. (thanks to J William Piggott)

The code behind "column -t|--table" uses libsmartcols now. This change
dramatically increased number of available features for table formatting. Now
it's possible to define header for columns, truncate text in cells, align text
to the right, change order of columns, JSON output or create tree-like output.
Now almost all libsmartcols features are available on command line, example:
pstree-like output:

Friday, January 6, 2017

I can't believe that it's already 10 years we have active community around basic Linux utils.

Yes, we had util-linux before (and many thanks to Adrian Bunk and Andries E. Brouwer), but I believe that with git and close collaboration between Linux distributions and Linux kernel community it better now :-)

~11000 commits

~460 unique contributors

~630 regression tests

~100 utils, 5 shared libs with public API

~16 major releases (from v2.13 to v2.29)

~26 translated languages

~10000 e-mails on mailing list

ported to GNU Hurd, FreeBSD and XOS

used by all mainstream Linux distros (as well as by Systemd haters ;-)

Thursday, October 6, 2016

The release v2.29 (now rc1) is without dramatical changes, the small exception
is libsmartcols where we have many improvements.

The old good cal(1) is more user-friendly now. It's possible to specify month
by name (e.g. "cal January 2017") and use relative placeholders, for example:

cal now
cal '1 year ago'
cal '+2 months'

fdisk(8) allows to wipe newly created partitions -- the feature is possible to
control by new command line option --wipe-partitions[==auto|never|default].

The
default in the interactive mode is to ask user when a filesystem or RAID signature
is detected. The goal is to be sure that new block devices are usable without
any collisions and extra wipefs(8) step (because users are lazy and mkfs-like
programs are often no smart enough to wipe the device).

findmnt --verify is probably the most attractive new feature for admins. The
command scans /etc/fstab and tries to verify the configuration. The traditional
way is to use "mount -a" for this purpose, but it's overkill. The new --verify
does not call mount(2), but it checks parsability, LABEL/UUID/etc.
translation to paths, mountpoints order, support for specified FS types.
The option --verify together with --verbose provides many details.

When you create multiple loop block devices from
one backing file then Linux kernel does not care about possible collisions
and the same on-disk filesystem is maintained by multiple independent in-memory
filesystem instances. The result is obvious -- data lost and filesystem damage.

Now mount(8) rejects requests to create another device and mount filesystem for the same backing file.
The command losetup --nooverlap reuse loop device if already exists for the
same backing file. All the functionality calculate with offset and sizelimit
options of course, so it's fine to have multiple regions (partitions) in the same image file and mount all of them in the same time. The restriction is that the regions should not overlap. Thanks to Stanislav Brabec from Suse!

Heiko Carstens from IBM (thanks!) has improved lscpu(1) for s390. Now it supports
"drawer" topology level, static and dynamic MHz, machine type and a new option
--physical.

The most important libsmartcols change is probably better support for
multi-line cells. Now the library supports custom cell wrap functions -- this
allows to wrap your text in cells after words, line breaks, etc.
See multi-line cells (WRAPNL column) output:

The idea
is to use libsmartcols as output formatter for Fedora/RHEL dnf (package manager
for RPM-based Linux distributions, yum replacement). This is also reason why
libsmartcols has been massively extended and improved in the last releases.

Tuesday, March 22, 2016

The release v2.28 does not contain any dramatical changes and huge
improvements. It's another release to keep users happy and absorb new features
provided by kernel. We all love this kind of release, right? :-)

The fdisk programs (sfdisk, cfdisk and fdisk) have been improved to wipe old filesystem, RAID and
partition tables from the device before libfdisk writes a new partition table.

The fdisk-like programs traditionally care about begin of the device, but it's
insufficient. This new feature has been introduced to avoid collisions between
new partition table and old unwanted signatures and it's possible to control
it by --wipe[=auto|never|always]. For backward compatibility on non-terminals
(non interactive fdisk execution) the feature is disabled by default.

Linux kernel 3.14 is really not a hot news, but standard Linux userspace still does not
support DEADLINE scheduler. chrt since v2.28 supports the DEADLINE scheduling class and
the new options --sched-runtime --sched-period and --sched-deadline.

The command logger supports RFC 5424 structured data through the new options
--sd-id and --sd-param. For example:

The library libsmartcols has been massively improved (thanks to Igor Gnatenko
for testing and reviews). Now it's possible to specify title for table, table
supports multi-line cells, it's possible to print subset of table and the
library supports continuous printing.

The portability of the util-linux package is not our primary goal, but in many
cases port code to the another libc or another operation system (if possible)
is a way how to detect code disadvantages, obsolete functions etc. v2.28 is
possible to compile on OSX and improved has been also support for kFreeBSD and GNU Hurd (of
course you cannot compile Linux specific stuff, but build-system is smart
enough to automatically disable utils irrelevant for your OS).

This is in
connection with our regression tests suite, where many things have been
improved to make the tests more stable in all random environments.
(thanks to Ruediger Meier).

Tuesday, December 1, 2015

The namespaces are commonly used way how to isolate global (ipc, mount, net, ...) resource instances. Unfortunately, we have no command line tool to list namespaces. The new command lsns(8) tries to fill this gap.

The important detail is that you can see only namespaces accessible from currently mounted /proc filesystem. The lsns(8) is not able to list persistent namespaces without processes where the namespace instance is hold by bind mounts of the /proc/[pid]/ns/[type] files and the output may be affected by unshared PID namespace and unshared /proc (see unshare(8) for more details).

... it will be probably available in util-linux v2.28 (~ January 2016).