On Tue, Feb 26, 2013 at 4:24 AM, Robert Yang <liezhi.yang@windriver.com> wrote:
> * Summary:> [...]now we use the mkfs.ext3/ext4 to create the image, and use mkdebugfs.sh> to copy the files to the image.
For a while this had me confused because I somehow thought you were
proposing using the Linux kernel's debugfs to create filesystems (i.e.
mount -t debugfs none /sys/kernel/debug). Maybe I'm still confused,
but if not I think, perhaps, it would be nice to come up with a better
name for this script since the name clash will probably trip others up
too.

On 02/26/2013 09:53 AM, Trevor Woerner wrote:
> On Tue, Feb 26, 2013 at 4:24 AM, Robert Yang <liezhi.yang@windriver.com> wrote:>> * Summary:>> [...]now we use the mkfs.ext3/ext4 to create the image, and use mkdebugfs.sh>> to copy the files to the image.> > For a while this had me confused because I somehow thought you were> proposing using the Linux kernel's debugfs to create filesystems (i.e.> mount -t debugfs none /sys/kernel/debug). Maybe I'm still confused,> but if not I think, perhaps, it would be nice to come up with a better> name for this script since the name clash will probably trip others up> too.
It is confusing, unfortunately, debugfs is provided by e2fsprogs, not
something of our making. If you would prefer another name for
mkdebugfs.sh, that's fine. What would you propose?

On 02/26/2013 09:55 AM, Saul Wold wrote:
> On 02/26/2013 01:24 AM, Robert Yang wrote:>> ===Contents===>> * Summary>> * Brief design>> * Test info>> * The rootfs generation time changes>> * The rootfs size changes>>>> * Summary:>> The ext3 and ext4 rootfs generation relied on genext2fs and flipping>> some bits to "convert" the ext2 filesystem to ext3 and ext4 in the past,>> now we use the mkfs.ext3/ext4 to create the image, and use mkdebugfs.sh>> to copy the files to the image.>>>> * Brief design:>> - Upgrade the e2fsprogs to the update to date version (the git>> repo) which contains the "symlink" command in debugfs shell that we>> need.> This should really be an RFC> > This work was intended to be a step toward incorporating it into > genext2fs directly.
I think that was a thinko, but I'll correct for everyone else's benefit.
This will eventually be incorporated into libext2fs and mke2fs directly
and genext2fs will be retired as obsolete.
> At this point we do not want to switch to the GIT> version of e2fsprogs, there is an update to e2fsprogs pending on the > patch queue that addes the symlink support (crossed with your request)> >> - Replace the current genext2fs command with mkdebugfs.sh in>> image_types.bbclass.> > The goal is to change genext2fs to support a rootfs option an not > replace it with mkdebugfs.
Again, mke2fs. We can go about this in two ways. One is to just
prototype this in a branch and use it to validate the functionality and
not make any changes to oe-core image generation until mke2fs has
initial directory support. The other is to merge this and get broader
testing of the concept and later move to the full mke2fs implementation
once it becomes available. I understand the resistance to the latter,
but long term I think it will result in a more robust solution as we
will have caught more of the corner cases and have been able to do a
better job integrating into mke2fs the first time.
> >> - The new steps to generate the root filesystem are:>> -> use "dd" command to make rootfs.ext* file> Why the dd?
mke2fs requires an image file to format. We could create it in other
ways, but this seems reasonable. We could probably reduce the time to
create it by using a larger block size. Say 1M?
> >> -> run "mkfs.ext* rootfs.ext*" to create the filesystem>> -> run the mkdebugfs.sh to generate the root filesystem>>> > > >> * Test info: (MACHINE = qemuarm, IMAGE_FSTYPES = "ext2 ext3 ext4 tar.bz2")>> $ bitbake core-image-minimal core-image-sato meta-toolchain-sdk core-image-sato-sdk>> $ runqemu runqemu qemuarm core-image-sato ext3>> $ runqemu runqemu qemuarm core-image-minimal ext3>>>> All of them are OK> > What about booting an ext4 image, part of the goal of this was to enable > ext4.
Seconded.

On 02/26/2013 10:15 AM, Phil Blundell wrote:
> On Tue, 2013-02-26 at 17:24 +0800, Robert Yang wrote:>> Before:>> $ time bitbake core-image-minimal>> >> real 1m10.823s>> user 0m37.108s>> sys 0m15.894s>> >> After:>> $ time bitbake core-image-minimal>> >> real 1m17.501s>> user 0m29.304s>> sys 0m20.731s>> >> # 7 seconds lost.>> >> - For a core-image-sato generation (IMAGE_FSTYPES="tar.bz2 ext3"):>> >> Before:>> $ time bitbake core-image-sato>> >> real 11m10.645s>> user 2m43.503s>> sys 1m1.589s>> >> After:>> $ time bitbake core-image-sato>> >> real 11m53.131s>> user 3m18.988s>> sys 2m8.350s>> >> # 43 seconds lost.>> >> * The rootfs size changes>> - The image size are the same by "ls -h", but different by "du -sh":>> (core-image-sato)>> >> $ ls -lh BEFORE.rootfs.ext3 AFTER.rootfs.ext3 | awk '{print $5"\t"$NF}'>> 357M BEFORE.rootfs.ext3>> 357M AFTER.rootfs.ext3>> >> $ du -sh BEFORE.rootfs.ext3 AFTER.rootfs.ext3>> 238M BEFORE.rootfs.ext3>> 357M AFTER.rootfs.ext3> > So, am I understanding correctly that your patch causes the rootfs> generation to take longer, and the resulting files to be bigger? It> doesn't seem totally obvious to me that this is a good thing, and your> cover letter doesn't provide any other explanation for why the changes> are desirable. Can you expand on the rationale for doing this?
A better motivation description is indeed required.
Basically, genext2fs doesn't support creating ext4 filesystems. It
creates, as I understand it, an ext2 filesystem, then adds a journal,
and sets some bits. It can't support the newer features like extents. So
what we end up with is a bit of a hack for a filesystem.
The ext tools (e2fsprogs) unfortunately don't provide an integrated
solution for generating prepopulated filesystem images as many other
mkfs* tools do. One thing missing was symlink support in libext2fs. I
added that support and demonstrated a script which uses the e2fsprogs
debugfs tool that can populate the newly formatted filesystem from a
directory and without root privileges.
Robert's patches integrate this stage of development into OE-Core. As I
replied to Saul's 0/7 response:
"We can go about this in two ways. One is to just
prototype this in a branch and use it to validate the functionality and
not make any changes to oe-core image generation until mke2fs has
initial directory support. The other is to merge this and get broader
testing of the concept and later move to the full mke2fs implementation
once it becomes available. I understand the resistance to the latter,
but long term I think it will result in a more robust solution as we
will have caught more of the corner cases and have been able to do a
better job integrating into mke2fs the first time."
> Also, when sending patches for review, there is no point in sending a> series which adds a new file and then makes changes to it as separate> patches (in particular when the changes are just whitespace). Please> squash those into a single patch before posting them on the list.
I think Robert was trying to maintain some history by keeping my
original script intact. I agree, it isn't necessary and those three
patches can be merged into one.
Thank you for reviewing!

On Tue, Feb 26, 2013 at 4:52 PM, Darren Hart <dvhart@linux.intel.com> wrote:
> On 02/26/2013 09:53 AM, Trevor Woerner wrote:>> On Tue, Feb 26, 2013 at 4:24 AM, Robert Yang <liezhi.yang@windriver.com> wrote:>>> * Summary:>>> [...]now we use the mkfs.ext3/ext4 to create the image, and use mkdebugfs.sh>>> to copy the files to the image.>>>> For a while this had me confused because I somehow thought you were>> proposing using the Linux kernel's debugfs to create filesystems (i.e.>> mount -t debugfs none /sys/kernel/debug).>> It is confusing, unfortunately, debugfs is provided by e2fsprogs, not> something of our making. If you would prefer another name for> mkdebugfs.sh, that's fine. What would you propose?
mkextXfs.sh ?

On 02/26/2013 02:37 PM, Trevor Woerner wrote:
> On Tue, Feb 26, 2013 at 4:52 PM, Darren Hart <dvhart@linux.intel.com> wrote:>> On 02/26/2013 09:53 AM, Trevor Woerner wrote:>>> On Tue, Feb 26, 2013 at 4:24 AM, Robert Yang <liezhi.yang@windriver.com> wrote:>>>> * Summary:>>>> [...]now we use the mkfs.ext3/ext4 to create the image, and use mkdebugfs.sh>>>> to copy the files to the image.>>>>>> For a while this had me confused because I somehow thought you were>>> proposing using the Linux kernel's debugfs to create filesystems (i.e.>>> mount -t debugfs none /sys/kernel/debug).>>>> It is confusing, unfortunately, debugfs is provided by e2fsprogs, not>> something of our making. If you would prefer another name for>> mkdebugfs.sh, that's fine. What would you propose?> > mkextXfs.sh ?
Likely to be confused with mke2fs itself. Perhaps:
populate-extfs.sh ?