Revisiting the installation process

Recommended Posts

First. Thanks for so great work in the ARMBIAN community. Any word of gratitude will not be enough.

I have been installing Linux on different SBC. Some of them are supported by ARMBIAN, others not. There are some that never will be supported (as the Parallella 16). And the ARMBIAN process it is very clean for the already refined machines. However, when trying to bring up some other machine, it is complicated to understand what to do, how and where, as the process have been evolving for a long period of time.

Recently I began to work with @chwe on the BPI-R2, and my first job is to make the nand-sata-install to work. So, I prepared an image to check this, to study the internal parts of the script and what is the purpose of it. For my surprise, the script do several things, but it seems to accumulate also old stuff, maybe about previous ways to do certain things. How to work this without breaking something else? There, I realized how important is the /etc/armbian-release file, but also the /usr/lib/nand-sata-install directory (for all possible cards, as the ones are not a10 or a20 receive the a20 treatment). And there is also the /usr/lib/u-boot directory with some operations that could be used in ... I suppose u-boot ..., although I am not sure in what way, as u-boot it is the first thing to run in the machine.

As a "reader", my perspective is that the vital system information it is not well organized. At least it doesn't follow a unified standard, and carry the possibility of side effects as some elements could be located in not well known places. I still need to make a catalog for all possible system parameter sources, and I figure that I will find some redundant elements in some place. And redundancy it is not only about saving resources, but about breaking something when not following a referential methodology where any change must be system wide (i.e. universal).

Maybe ... and this is the reason for the discussion ... it is important to create a unified system information backbone where all processes in the system can obtain trustworthy information to work with (/proc only works with the kernel stuff). This could also reduce the complexity of the different processes, as it could provide reusability of some information gathering functions (instead of repeating them in different scripts and or programs). To install a system in a disk looks similar to reconfigure it in an already installed place or even to query how it have been installed.

What is the opinion of people here? There are related discussions but they seems to be a little lost in time. I am not really interested in the historical solutions (this can be written in some book), but in the actual problems and how to resolve them correctly. There is a related thread about naming, although this one is about operating.

Share this post

Link to post

Share on other sites

Great that it finally found it's way into an own thread. Just for others (https://github.com/armbian/build/issues/1144) and the origin of this discussion came up in the BPi-R2 thread. I hope we can keep the information in one place. So hopefully it would be this thread.

1 hour ago, malvcr said:

Recently I began to work with @chwe on the BPI-R2, and my first job is to make the nand-sata-install to work.

It's not a job, there won't be money for it.. But anyway, if we bring up a new script which covers all this, there's no need to support the R2 with the old nand-sata-install.sh. At least for me, it would be fine to have it on the 'Armbian-install.sh' only.

Some thoughts and parts I remember from other threads where it was discussed from my side:

Rermember: We had a 'huge' discussion how the script is named.. Hopefully we can avoid it this time.. As long as there's no script it simply doesn't matter how it is named. Once it's finished most people don't care, as long as it does a decent job.

Opinion: When I played a bit with a second FAT partition which would allow some configurations prior to first boot (idea behind was a 1mb FAT partition which gets properly shredded during the first boot but writes some configuration stuff.. e.g. network settings). I got in favor with yaml files for information, firstly they're easy to read for humans, secondly a bash script reading from such files is actually a 'few liner' and it worked quite well. https://gist.github.com/pkuczynski/8665367 probably this script can be enhanced to edit existing entries, should also be a 'fewliner'.

Opinion: Let's dump all unneeded stuff from nand-sata-install 3.4 kernel will fade out anyway so we don't need NAND,

We need to sum up all the NEEDS we have. And see if we should cover it under one script or split it in different 'services'. E.g. similar to @botfabs 'inject' script where there is a core script and some additional scripts.

Share this post

Link to post

Share on other sites

What could be an Armbian core? ... Maybe basic kernel with file systems and networking? How difficult is to bring only this up on all known machines? How to do it with unknown machines? Is mainline safe enough to play on new devices? The store systems (SATA, SD, EMMC ... etc ... ) must be here, together with the capacity of moving the OS to these types of store.

I need hardware cryptography support, but not everybody needs that. The most of the people can have a happy life with software based cryptography. The same goes on with GPIO and some specialized ports.

HDMI it is not needed everywhere. Although the R2 has an HDMI port, seldom people will use that machine with a monitor. Several Zero - Level devices lack of a video port because their focus it is not to have a desktop. The same happens with LCD, even with cameras. A NAS and many headless systems don't need many of these elements and can work with a web based setup (as openmediavault, mediawiki or openkm do).

And, of course, there is the "desktop", although with the small machines seems to be a specialized desktop because they are not powerful enough to do "everything" a current standard PC usually do.

My point is that maybe it is better to figure how the machines could be used instead of trying to resolve 100% problems at once. The comment about HDMI in the R2 Armbian unsupported page it is right; maybe the HDMI never will work well, because really, almost nobody needs an HDMI port in a router, bridge or these types of machines.

Share this post

Link to post

Share on other sites

This (new) script can and should drop support for NAND install cases. This will remain supported with a current script. What could also add fast(er) install option. If we don't want to change a file system type, SD card can be DD-ed to emmc. This works in most cases, but not all. U-boot will not be a part of the default package base in the future, which means it needs to be installed, applied (to eMMC) and de-installed. Support for separate boot partition must also be fully operational. Install should be parameter for automated install and menu driven as it is now.

4 hours ago, malvcr said:

I need hardware cryptography support, but not everybody needs that.

This is a bit off topic. You need to have your own kernel if you go out of the defaults or ask that some kernel feature is included by default. IMO we don't need to make more complications here.

3) Normal ... same file system, just raw copy to final storage. As option, and for special cases, new file system rsync or similar.

4) boot partition must work always, although, the particular model can change (now u-boot, later other methods) ... this means, boot must be also a sub-script that can be easily replaced or updated as required.

About "extra", "exotic" things as hardware cryptography, as them must be worked in a detailed particular (non generic) way, a chapter for each configuration and customization in the online documentation seems to be enough. The normal method will not use that capacity but a well tested methodology must be available (and the forum it is just a light unstructured reference). To bring-up a new machine, that work it is not required and could be available "later" if it is possible to do so.

In the menu case, the interactive method must be the "default", but also some type of "batch" must be available. Could be possible to install from an image but also to re-create the image from an existing installation. Sometimes it is necessary to install many similar machines as pre-configured images.

Share this post

Link to post

Share on other sites

I think it would be helpful to list all the u-boot + rootfs location combinations possible. Something like that :

Then for each board we define in a definition file what combination is supported. We could use bitmap hex value. For example 0x13 will means the board only support MMC/MMC, MMC/Block Device and Block Device/Block Device.

Regarding the utility feature, for example today on Helios4, it takes 2 steps to completely move away from SDcard

2nd - Move RootFS to other device than USB (menu 6 Boot from SPI - system on SATA, USB or NVMe)

I think it's ok that way, but we could have imagined this to be done in one step, where the user would choose through the interactive menu : U-Boot Location + RootFS Location, then from there the script take care to fulfill the new locations.

As you described it, the new script should be designed to address only the generic use cases, any board specific use case should be address via a plug-in / sub-script.

I also agree with you, when user choose the partition type for the new rootfs target, he should be able to choose LUKS. However, your point on hardware crypto off-load is off topic and only related to kernel crypto framework, it has nothing to do with u-boot | rootfs migration.

I understand the utility name is up to discussion, but if it ends up being just a subset of armbian-config then maybe yes the name doesn't matter. I will still venture to coin a name :uboot-rootfs-manage.sh

Share this post

Link to post

Share on other sites

I just want to linked up an important comment here from ongoing RFC WIP disucssion, this way we know what additional use case we need to address in nand-sata-install (or whatever new name).

Quote

Another issue is u-boot. We will drop u-boot package installation for all boards. Only compiling, packing and writing. But we need to have "write_uboot_platform" definitions for nand-sata-install and/or manual install/update via armbian-config ...