Re: [Lse-tech] Proposal for Enhanced Accounting HOWTO

I think Andrew removed pagg for lack of code that uses pagg?
I added Andrew to cc on this.
Pagg has completed code review cycle. I am not concerned of this.
The users of pagg are trying to get accepted :) CSA is one user
of pagg and we have other software that needs pagg is working on
release under GPL license.
Thanks,
- jay
Guillaume Thouvenin wrote:
> Enhanced Accounting HOWTO
> =========================
>
> According to discussion on the lse-tech mailing-list, it appears that
> three steps (at least) are required to improve accounting.
>
> 1) Improve accounting structure
> ----------------------------
>
> The current BSD-accounting structure doesn't have enough informations.
> Metrics computed by CSA module can be added to BSD accounting. According
> to other discussion (like Andi Kleen's comment on the patch I wrote when
> I wanted to add CSA IO values in the BSD accounting
> ( http://lkml.org/lkml/2004/8/2/70 ) the current method to get metrics
> about blocks/char read/write is not accurate since most writes can be
> accounted by some pdflush threads. Maybe add a counter in the routine
> mpage_writepages() but I don't know if we can recover a process ID from
> the struct page and I don't know if it will be enough... I'm looking if
> this is the right way.
>
>
> 2) Group of processes management
> ------------------------------
>
> We need to be able to manage groups of processes as it's clear that
> a major accounting improvement is the per-job accounting. I don't know if
> "job" is the right noun. There are several implementation that already exist
> and some of them are already in the kernel. The property needed here is that
> if a process is in a container, its children will be in the same container.
> Different implementations can be:
>
> - PAGG + JOB (job)
> - ELSA (bank)
> - CKRM (class)
> - CPUSET (a cpuset of all CPUs can act as a container)
>
> The interface between kernel and user space application can be a new virtual
> file system (like CKRM does with /rcfs) but it can also be a device driver with
> ioctl operations (like ELSA does). Both solutions are interesting and need to
> be discussed. We can notice that CPUSET is in 2.6.9-rc1-mm2 tree and it seems
> that PAGG has been removed (don't know why) from -mm tree.
>
> 3) Data presentation
> -----------------
>
> We can have several different implementations.
>
> 4) General overview
> ----------------
>
> KERNEL SPACE | USER SPACE (or MODULE)
> |
> ---------------- |
> | BSD accounting | |
> | + | | ------------
> | CSA |<=======>| |
> ---------------- | | Enhanced |
> | | Accounting |
> ------------- | | Core |
> | group of |<=======>| |
> | processes | | ------------
> | manager | |
> ------------- |
> |
>
> Communication between EAC (Enhanced Accounting Core) and kernel space can be
> done via virtual fs or a device. The goal of EAC is to keep a trace of the
> different groups of processes during the accounting period using the group of
> processes management module. The idea is that the group of processes management
> can send a message to EAC when a process ended to indicate the job
> ID if any. The other thing that has to be done by EAC is the data presentation.
>
> Here is an example of what can be done:
>
> 1) First we can add processes into containers using the GPM (group of
> processes manager) module. For example we can add an ftp server with
> pid #123 and a daemon ssh with pid #234. Thus, inside the GPM you have:
> container 1 -> 123
> container 2 -> 234
>
> 2) Now, a user can login via ssh, so, sshd will create new children. Thus,
> inside the GPM you will have something like:
> container 1 -> 123
> container 2 -> 234 333 334 335
>
> 3) As soon as a process is terminated, the EAC must be aware of that fact
> (using a signal from GPM for example). There is just this modification
> that is needed. With the signal, the GPM will transfer information like
> pid, jib (job ID), command, ... to the EAC. This will allow to keep trace
> about what happens in the system. Thus, in the EAC we keep information
> about all processes that are finished.
>
> 4) If the sysadmin asks information about process 234, the EAC will know that
> it belongs to container 2 and there are other processes in this container.
> As all processes are exited, accounting information are written in the
> accounting file (like it is currently with BSD accounting). With those
> informations (accounting + job information) we can do per-job accounting.
>
> I am working on the EAC implementation to check how we can make it work.
>
> Any comments?
>
> Best
> Guillaume
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by BEA Weblogic Workshop
> FREE Java Enterprise J2EE developer tools!
> Get your free copy of BEA WebLogic Workshop 8.1 today.
> http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click
> _______________________________________________
> Lse-tech mailing list
> Lse-tech@...
> https://lists.sourceforge.net/lists/listinfo/lse-tech

Thread view

Enhanced Accounting HOWTO
=========================
According to discussion on the lse-tech mailing-list, it appears that
three steps (at least) are required to improve accounting.
1) Improve accounting structure
----------------------------
The current BSD-accounting structure doesn't have enough informations.
Metrics computed by CSA module can be added to BSD accounting. According
to other discussion (like Andi Kleen's comment on the patch I wrote when
I wanted to add CSA IO values in the BSD accounting
( http://lkml.org/lkml/2004/8/2/70 ) the current method to get metrics
about blocks/char read/write is not accurate since most writes can be
accounted by some pdflush threads. Maybe add a counter in the routine
mpage_writepages() but I don't know if we can recover a process ID from
the struct page and I don't know if it will be enough... I'm looking if
this is the right way.
2) Group of processes management
------------------------------
We need to be able to manage groups of processes as it's clear that
a major accounting improvement is the per-job accounting. I don't know if
"job" is the right noun. There are several implementation that already exist
and some of them are already in the kernel. The property needed here is that
if a process is in a container, its children will be in the same container.
Different implementations can be:
- PAGG + JOB (job)
- ELSA (bank)
- CKRM (class)
- CPUSET (a cpuset of all CPUs can act as a container)
The interface between kernel and user space application can be a new virtual
file system (like CKRM does with /rcfs) but it can also be a device driver with
ioctl operations (like ELSA does). Both solutions are interesting and need to
be discussed. We can notice that CPUSET is in 2.6.9-rc1-mm2 tree and it seems
that PAGG has been removed (don't know why) from -mm tree.
3) Data presentation
-----------------
We can have several different implementations.
4) General overview
----------------
KERNEL SPACE | USER SPACE (or MODULE)
|
---------------- |
| BSD accounting | |
| + | | ------------
| CSA |<=======>| |
---------------- | | Enhanced |
| | Accounting |
------------- | | Core |
| group of |<=======>| |
| processes | | ------------
| manager | |
------------- |
|
Communication between EAC (Enhanced Accounting Core) and kernel space can be
done via virtual fs or a device. The goal of EAC is to keep a trace of the
different groups of processes during the accounting period using the group of
processes management module. The idea is that the group of processes management
can send a message to EAC when a process ended to indicate the job
ID if any. The other thing that has to be done by EAC is the data presentation.
Here is an example of what can be done:
1) First we can add processes into containers using the GPM (group of
processes manager) module. For example we can add an ftp server with
pid #123 and a daemon ssh with pid #234. Thus, inside the GPM you have:
container 1 -> 123
container 2 -> 234
2) Now, a user can login via ssh, so, sshd will create new children. Thus,
inside the GPM you will have something like:
container 1 -> 123
container 2 -> 234 333 334 335
3) As soon as a process is terminated, the EAC must be aware of that fact
(using a signal from GPM for example). There is just this modification
that is needed. With the signal, the GPM will transfer information like
pid, jib (job ID), command, ... to the EAC. This will allow to keep trace
about what happens in the system. Thus, in the EAC we keep information
about all processes that are finished.
4) If the sysadmin asks information about process 234, the EAC will know that
it belongs to container 2 and there are other processes in this container.
As all processes are exited, accounting information are written in the
accounting file (like it is currently with BSD accounting). With those
informations (accounting + job information) we can do per-job accounting.
I am working on the EAC implementation to check how we can make it work.
Any comments?
Best
Guillaume

On Fri, Sep 03, 2004 at 03:30:48PM +0200, Guillaume Thouvenin wrote:
> Enhanced Accounting HOWTO
> =========================
>
> Here is an example of what can be done:
>
> 1) First we can add processes into containers using the GPM (group of
> processes manager) module. For example we can add an ftp server with
> pid #123 and a daemon ssh with pid #234. Thus, inside the GPM you have:
> container 1 -> 123
> container 2 -> 234
>
> 2) Now, a user can login via ssh, so, sshd will create new children. Thus,
> inside the GPM you will have something like:
> container 1 -> 123
> container 2 -> 234 333 334 335
Can/will this support project based accounting ?
i.e. Users being member of several projects. Each project having
multiple users as members. And the user is allowed to decide which
project-id each container is going to run under / be accounted to ?
-jf

On Fri, Sep 03, 2004 at 10:48:33AM -0700, Jay Lan wrote:
> Support of project concept is certainly one thing we need to consider.
> We have received requests to support project in CSA.
>
> Cheers,
> - jay
> >
> >Can/will this support project based accounting ?
> >
> >i.e. Users being member of several projects. Each project having
> >multiple users as members. And the user is allowed to decide which
> >project-id each container is going to run under / be accounted to ?
Here is an update of the Enhanced Accounting HOWTO.
Here is the ChangeLog:
6/09/2004: Guillaume Thouvenin
- Add a section concerning projects based accounting
proposed by Jan-Frode Myklebust.
- Add a section concerning the EAC (Enhanced Accounting
Core) part.
And here is the proposal:
Enhanced Linux accounting Howto?
================================
According to discussion on the lse-tech mailing-list, here are some
stages that are required to improve accounting.
1) Improve accounting structure
----------------------------
The current BSD-accounting structure doesn't have enough informations.
Metrics computed by CSA module can be added to BSD accounting. According to
other discussion (like Andi Kleen's comment on the patch I wrote when I wanted
to add CSA IO values in the BSD accounting ( http://lkml.org/lkml/2004/8/2/70 )
the current method to get metrics about blocks/char read/write is not accurate
since most writes can be accounted to pdflsuhd.
2) Group of processes management
------------------------------
We need to be able to manage groups of processes as it's clear that a major
accounting improvement is the per-job accounting. I don't know if "job" is the
right noun. There are several implementation that already exist and some of them
are already in the kernel. The property needed here is that if a process is in
a container, its children will be in the same container. Different
implementations can be:
- PAGG + JOB (job)
- ELSA (bank)
- CKRM (class)
- CPUSET (a cpuset of all CPUs can act as a container)
The interface between kernel and user space application can be a new virtual
file system (like CKRM does with /rcfs) but it can also be a device driver with
ioctl operations (like ELSA does). Both solutions are interesting and need to
be discussed. We can notice that CPUSET is in 2.6.9-rc1-mm2 tree and PAGG has
been removed from -mm tree because there is a lack of code that uses PAGG.
3) Project based accounting
------------------------
It could be interesting to add a support for project based accounting. It
means that a user can be a member of one or more projects and he is allowed
to decide which project-id each container is going to run. This could be done
inside the Enhanced Accounting Core module.
4) Data presentation
-----------------
We can have several different implementations.
5) General overview
----------------
KERNEL SPACE | USER SPACE (or MODULE)
|
---------------- |
| BSD accounting | |
| + | | ------------
| CSA |<=======>| |
---------------- | | Enhanced |
| | Accounting |
------------- | | Core |
| group of |<=======>| |
| processes | | ------------
| manager | |
------------- |
|
Communication between EAC (Enhanced Accounting Core) and kernel space can be
done via virtual fs or a device. The goal of EAC is to keep a trace of the
different groups of processes during the accounting period using the group of
processes management module. The idea is that the group of processes management
can send a message to EAC when a process ended to indicate the job
ID if any. The other thing that has to be done by EAC is the data presentation.
Here is an example of what can be done:
a) First we can add processes into containers using the GPM (group of
processes manager) module. For example we can add an ftp server with
pid #123 and a daemon ssh with pid #234. Thus, inside the GPM you have:
container 1 -> 123
container 2 -> 234
b) Now, a user can login via ssh, so, sshd will create new children. Thus,
inside the GPM you will have something like:
container 1 -> 123
container 2 -> 234 333 334 335
c) As soon as a process is terminated, the EAC must be aware of that fact
(using a signal from GPM for example). There is just this modification
that is needed. With the signal, the GPM will transfer information like
pid, jib (job ID), command, ... to the EAC. This will allow to keep trace
about what happens in the system. Thus, in the EAC we keep information
about all processes that are finished.
d) If the sysadmin asks information about process 234, the EAC will know that
it belongs to container 2 and there are other processes in this container.
As all processes are exited, accounting information are written in the
accounting file (like it is currently with BSD accounting). With those
informations (accounting + job information) we can do per-job accounting.
6) Enhanced Accounting Core (EAC) implementation
---------------------------------------------
The EAC can be implemented in the user space or as a kernel module.
We will see later differences between both implementation. The EAC has
several goals.
o It will manage a group of processes historic. It means that it
will keep a trace concerning processes that belong to the same
container. We don't care about how container are managed, we just
need to know when a new process is created and if he is in a
container. It means that the job manager (or group of processes
manager) must provide a mechanism to share this information. If
we decide to implement EAC in user space, information can be pass
through a signal or a device. If it's a module, we can add an hook
in the code of the job manager.
o It will provide routine to add/remove processes from container. It
will act as a wrapper for routine that already exist in the job
manager.
o It will get information from the accounting file. Probably from
BSD accounting extended with CSA information. Those informations
will be used in conjunction with the group of process historic to
provide the enhanced accounting which is a per-job accounting
information.
o Finally, EAC we will support the project based accounting as describe
in section 3)
Any comments?
I am working on the EAC implementation to check how we can make it work.
Guillaume

I think Andrew removed pagg for lack of code that uses pagg?
I added Andrew to cc on this.
Pagg has completed code review cycle. I am not concerned of this.
The users of pagg are trying to get accepted :) CSA is one user
of pagg and we have other software that needs pagg is working on
release under GPL license.
Thanks,
- jay
Guillaume Thouvenin wrote:
> Enhanced Accounting HOWTO
> =========================
>
> According to discussion on the lse-tech mailing-list, it appears that
> three steps (at least) are required to improve accounting.
>
> 1) Improve accounting structure
> ----------------------------
>
> The current BSD-accounting structure doesn't have enough informations.
> Metrics computed by CSA module can be added to BSD accounting. According
> to other discussion (like Andi Kleen's comment on the patch I wrote when
> I wanted to add CSA IO values in the BSD accounting
> ( http://lkml.org/lkml/2004/8/2/70 ) the current method to get metrics
> about blocks/char read/write is not accurate since most writes can be
> accounted by some pdflush threads. Maybe add a counter in the routine
> mpage_writepages() but I don't know if we can recover a process ID from
> the struct page and I don't know if it will be enough... I'm looking if
> this is the right way.
>
>
> 2) Group of processes management
> ------------------------------
>
> We need to be able to manage groups of processes as it's clear that
> a major accounting improvement is the per-job accounting. I don't know if
> "job" is the right noun. There are several implementation that already exist
> and some of them are already in the kernel. The property needed here is that
> if a process is in a container, its children will be in the same container.
> Different implementations can be:
>
> - PAGG + JOB (job)
> - ELSA (bank)
> - CKRM (class)
> - CPUSET (a cpuset of all CPUs can act as a container)
>
> The interface between kernel and user space application can be a new virtual
> file system (like CKRM does with /rcfs) but it can also be a device driver with
> ioctl operations (like ELSA does). Both solutions are interesting and need to
> be discussed. We can notice that CPUSET is in 2.6.9-rc1-mm2 tree and it seems
> that PAGG has been removed (don't know why) from -mm tree.
>
> 3) Data presentation
> -----------------
>
> We can have several different implementations.
>
> 4) General overview
> ----------------
>
> KERNEL SPACE | USER SPACE (or MODULE)
> |
> ---------------- |
> | BSD accounting | |
> | + | | ------------
> | CSA |<=======>| |
> ---------------- | | Enhanced |
> | | Accounting |
> ------------- | | Core |
> | group of |<=======>| |
> | processes | | ------------
> | manager | |
> ------------- |
> |
>
> Communication between EAC (Enhanced Accounting Core) and kernel space can be
> done via virtual fs or a device. The goal of EAC is to keep a trace of the
> different groups of processes during the accounting period using the group of
> processes management module. The idea is that the group of processes management
> can send a message to EAC when a process ended to indicate the job
> ID if any. The other thing that has to be done by EAC is the data presentation.
>
> Here is an example of what can be done:
>
> 1) First we can add processes into containers using the GPM (group of
> processes manager) module. For example we can add an ftp server with
> pid #123 and a daemon ssh with pid #234. Thus, inside the GPM you have:
> container 1 -> 123
> container 2 -> 234
>
> 2) Now, a user can login via ssh, so, sshd will create new children. Thus,
> inside the GPM you will have something like:
> container 1 -> 123
> container 2 -> 234 333 334 335
>
> 3) As soon as a process is terminated, the EAC must be aware of that fact
> (using a signal from GPM for example). There is just this modification
> that is needed. With the signal, the GPM will transfer information like
> pid, jib (job ID), command, ... to the EAC. This will allow to keep trace
> about what happens in the system. Thus, in the EAC we keep information
> about all processes that are finished.
>
> 4) If the sysadmin asks information about process 234, the EAC will know that
> it belongs to container 2 and there are other processes in this container.
> As all processes are exited, accounting information are written in the
> accounting file (like it is currently with BSD accounting). With those
> informations (accounting + job information) we can do per-job accounting.
>
> I am working on the EAC implementation to check how we can make it work.
>
> Any comments?
>
> Best
> Guillaume
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by BEA Weblogic Workshop
> FREE Java Enterprise J2EE developer tools!
> Get your free copy of BEA WebLogic Workshop 8.1 today.
> http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click
> _______________________________________________
> Lse-tech mailing list
> Lse-tech@...
> https://lists.sourceforge.net/lists/listinfo/lse-tech