= Netfilter's list of ideas for students participanting in GSoC 2019 =
This document provides an introduction to the students that are willing to
apply to this round of Netfilter's GSoC 2019. Please, take the time to read
before you make questions.
== Introduction ==
In this edition, we propose that the students focus again on the nftables [1]
project, the successor of the popular iptables [2] firewalling tool. There is
a huge ongoing development effort to get nftables into production state.
We believe that GSoC students will help to boost this development, more
specifically on tasks that are relatively simple but time consuming and that
really need to be done.
== Prerequisites ==
General requirements for students to participate are:
* must know C, writing code fluently.
* computer networking at a good level, more specifically, layer 2/3/4 of TCP/IP
stack.
== Proposed tasks ==
We propose several tasks for GSoC students in the next sections, this year
we provide mostly tasks that range from average to hard in terms of difficulty.
All these tasks also involve helping in bug hunting and fixing.
We added a tag to the classify proposed tasks following the description
available in GSoC's "Defining a Project (Ideas List)" [6].
== Task 1: Implement missing features in nftables ==
* Description: As of Linux kernel 4.20, nftables provides around 90% of the
iptables feature-set [5]. We believe that this is fundamental to help users
to migrate to nftables. We describe a list of possible subtasks for
reference.
* Tag: Core development.
* Tasks: Help by implementing missing features available in iptables as
matches/targets.
* Level of difficulty: Average. There is already code that you can use as
reference for this task.
* Mentors: Pablo Neira Ayuso / Arturo Borrero / Florian Westphal
=== Subtask 1.1: native "-m time" support in nftables ==
The first and also most simple feature that could be implemented in native nft
is the --datestart / --datestop option of -m time, i.e. to match when the packet
arrives between two fixed, absolute points in time. For instance "match between
Feb 14th 2019 and March 30th 2019". For instance "match between Feb 14th 2019
and March 30th 2019".
To implemend this one would first have to add support in the kernel, i.e. add a
new key to the "meta" expression, e.g.: NFT_META_TSTAMP_NS.
Then, extend net/netfilter/nft_meta.c to handle NFT_META_TSTAMP_NS in the
evaluation function:
case NFT_META_TSTAMP_NS:
if (skb->tstamp == 0)
__net_timestamp((struct sk_buff *)skb);
*reg64 = ktime_to_ns(skb->tstamp);
break;
in other words, exposes the 64bit nanosecond timestamp of the packets arrival
time in a register.
You can refer to: https://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf-next.git/commit/?id=0fb4d21956f4a9af225594a46857ccf29bd747bc
to see an example of adding new meta functionality.
This is enough to allow matching a fixed time interval from a kernel
point of view. To make this useful, next step is to extend nft and libnftnl
to be able to make use of this new functionality.
nft userspace needs to be extended to handle a new 'meta timestamp' keyword.
You will need to extend the scanner (flex) and/or parser (bison) to teach the
frontend how to interpret a new keyword.
Adding a new keyword for meta is simple, for example
http://git.netfilter.org/nftables/commit/?id=512795a673f999fb04b84dbbbe41174e9c581430
shows all that is needed to add a new meta keyword on the nftables side.
Once nft parser understands 'meta timestamp'.
=== Subtask 1.2: complete osf support ==
The existing OSF support for nftables does not support for matching the
OS version type. You have to add a new NFTA_OSF_FLAGS attribute:
enum nft_osf_flags {
NFT_OSF_F_VERSION = (1 << 0),
};
By default, if there is no NFTA_OSF_FLAGS, flag is set to zero,
hence no version matching needs to be done. You have to extend nf_osf_find() to
pass a structure as parameter:
struct nf_osf_data {
const char *os_name;
const char *version;
};
Then, pass this structure to nf_osf_find() that is called from packet path:
if (nf_osf_find(skb, ..., &data)) {
switch (priv->type) {
case NFT_OSF_GENRE:
strncpy((char *)dest, os_name, ...);
break;
case NFT_OSF_VERSION:
...
break;
}
} else {
strncpy((char *)dest, "unknown", NFT_OSF_MAXNAMELEN);
}
=== Subtask 1.3: Extending stateful object infrastructure ===
Extend the stateful object [1] support to allow to runtime update, eg. upgrade
a 'quota' object without losing stateful values, such as rising or lowering the
quota limit of an existing object. This also would allow us to update timeout
policies for conntrack in runtime. Support for this needs to integrate into de
transaction infrastructure.
Moreover, allowing for internal toggles for stateful objects, such as connection
tracking helpers is also desiderable, e.g, we will not need a global 'loose'
switch in the FTP helper anymore, instead this you be explicitly set on from the
nftables ruleset policy file.
[1] https://wiki.nftables.org/wiki-nftables/index.php/Stateful_objects
=== Subtask 1.4: allow deletion of set elements from ruleset ==
iptables' SET target has a "--del-set" option that will remove the matching
address from the given set.
in nftables, its possible to add and update, but not delete a set element.
Example ruleset:
table ip test {
set testset {
type ipv4_addr
size 65535
flags timeout
}
chain testchain {
type filter hook input priority filter; policy accept;
ip saddr @testset counter
update @testset { ip saddr timeout 1m }
# add @testset { ip saddr timeout 1h }
}
}
The difference is that 'add' will not add the element with timeout of 2h
if it already exists, whereas 'update' will change the timeout even if
the entry already exists.
This is useful when elements get also added via
nft update element ip test testset { 127.1.2.3 timeout 2h} .
In this case, a packet sent to that address (matching ip saddr) would
change timeout to 1 minute.
If only the 'add @testset' rule would be used, the timeout would
remain at 2 hours, whereas a different source address would be added
with a 1 hour timeout.
Its currently not possible to early-remove elements from the set.
One can add a rule that sets a very low timeout, e.g.
... update @testset { ip saddr timeout 1ms }
but it would be better to add ''proper'' support for a delete
feature, e.g.
delete @testset { ip saddr }
=== Subtask 1.5: performance optimizations in netfilter ===
CONFIG_RETPOLINE (cpu bug workaround ...) makes indirect calls
expensive.
One way to improve performance is to remove such indirections where
possible.
The set infrastructure would be one field where indirection removal
might make sense.
This task involves demodularization of nf_tables_set (make it a boolean,
not a module), a size analysis of the nf_tables module (absolute size increase,
percentage increase) after this change.
Then, because functions can be called directly, the lookup indirections can be removed.
https://git.breakpoint.cc/cgit/fw/nf-next.git/commit/?h=nft_set_rework_01
contains a untested hack that illustrates the second part (demodularization is missing,
so this won't work with NF_TABLES_SET=m config).
DO NOT BOTHER investigating the NAT engine, work on this is almost
complete already at this point, but not integrated into the main
development tree yet).
=== Subtask 1.6: rework netfilter logging ===
netfilter supports two logging backends, one is prink-based (logs decoded packet information
like ip addresses, port numbers and so on to dmesg buffer), one is nfnetlink based (copy
of packet gets sent to userspace listener for processing).
Its not possible to use both at the same time:
sysctl -a |grep nf_log_ipv4
net.netfilter.nf_log.2 = nfnetlink_log
When nfnetlink_log is enabled, the dmesg support disappears, and users need to run
sysctl net.netfilter.nf_log.2=nf_log_ipv4
to re-enable dmesg logging, but that turns off the nfnetlink backend.
Because only two packet logger types exist (printk-based and nfnetlink one) it
could be worth to investigate if we can support both being active at the same time and
implement this.
== Task 2: Improving automated test infrastructure ==
* Description: Test infrastructure is fundamental to catch regressions.
This project already comes with a nice test infrastructure, but we always
consider good to have more coverage.
* Tag: Infrastructure/Automation.
* Tasks: Help by extending the existing infrastructure to support more tests
based on recent fixes and new features that got merged upstream.
Add a new 'make tests' target to iptables and nftables code base
to auto-run all test cases (python and shell based) that exist.
Add a new build test for iptables/nftables that makes sure
the projects can be compiled when various 'configure' switches
get enabled or disabled.
There was a recent build breakage in nftables because developer
ran a build test without json support enabled and failed to notice
that nftables would fail to compile for users that tried to do
"./configure --with-json && make".
Add more netfilter functionality tests to the kernel kselftest infrastructure,
see tools/testing/selftests/netfilter in the linux kernel directory.
For example, one could add a test that makes sure connection tracking
events (conntrack -E) work, that conntrack packet/byte accounting
works, and so on.
The existing iptables test infrastructure cover per-netns control plane
path. This is missing in nftables, we need an extension for nft-test.py
and shell/run-tests.py to extend coverage to the network namespace path.
Basically, in order to make sure resources are properly cleaned up after
netnamespace removal.
* Level of difficulty: Easy. There is already code in place that can be used
as reference.
* Mentors: Pablo Neira Ayuso / Arturo Borrero / Florian Westphal
== Task 3: nftlb: nftables load balancer ==
* Description: We have userspace daemon to configure load balancing scenario
using the nftables framework that provides a REST API.
* Tag: Fun/Peripheral.
* Tasks: Improve the existing codebase, add new features, such as:
** hardening of the REST API json parser.
** replication of sets and maps between nftlb instances.
** implement least connection schedulers.
** support of conntrack events management.
** add compatibility with kubernetes and docker rules.
** add support of tproxy.
* Level of difficulty: Easy/Average. this is userspace codebase after
all, but this is the kind of software that utilizes the nftables
framework that people find useful :)
* Mentors: Pablo Neira Ayuso / Arturo Borrero / Florian Westphal
= More information on nftables =
The next Netfilter workshop in June-July 2019 [3] will surely focus on nftables
ongoing and future development discussions. The kernel components were already
merged into mainstream Linux kernel 3.13. Nonetheless, implementation works are
still far from complete.
All existing code is available under git.netfilter.org. More specifically:
* libnftnl: low-level userspace library for nftables (for libmnl) iptables.
* which already includes the iptables compatibility layer working over
nftables.
* nft: the new user-space command line tool, with a new syntax different from
iptables.
The Linux kernel tree containing the nftables modules is currently available in
a different repository [4].
= Contact us / Make us questions =
If you are a student willing to participate in GSoC 2019 and you're interested
in any of our tasks, please subscribe to this mailing list:
https://lists.netfilter.org/mailman/listinfo/gsoc2013
Subscribing to this mailing list requires approval from the administrator, so
please be patient, we'll accept it asap. You can use this mailing list to ask
your questions regarding Netfilter's task during the GSoC 2019.
You can also drop a line to arturo@netfilter.org, please make sure you Cc
gsoc2013@lists.netfilter.org in your questions since most likely what you
ask and the reply you get will help others in the community too.
= Applying to netfilter's GSoC =
If you want to be selected, go start getting familiarized with the nftables
software asap. Patches for the userspace library libnftnl, the command line
utility nft and kernel patches will make you rank higher in the student
selection process. No patches at all mean little chances to be selected.
= References =
[1] http://en.wikipedia.org/wiki/Nftables
[2] http://www.netfilter.org/projects/iptables/index.html
[3] http://workshop.netfilter.org/
[4] http://git.kernel.org/cgit/linux/kernel/git/pablo/nftables.git
[5] http://wiki.nftables.org
[6] https://google.github.io/gsocguides/mentor/defining-a-project-ideas-list
Author: Pablo Neira Ayuso et al.
Last update: 10:31:44 +01:00 16/Feb/2019
-EOF-