UPDATE 16 Feb 2018: doing a writeup this weekend, talk about a can of worms!
UPDATE: 03 March 2018: updated and open for comment.Last edited by scsijon on Mon 05 Mar 2018, 00:11; edited 3 times in total

After a bit of research chasing people and asking questions on the net, I found that:-

Sourcerer was origonally called Sorcerer GNU Linux (SGL). Nowadays SGL doesn't exist anymore because the original author (Kyle Sallee) retracted his GPL code and distributes it under a non-OSI approved license and is a commercial product.

Sorcerer Linux was, and I quote.

Quote:

Sorcerer is a simple yet powerful source based Linux distribution. Simple and powerful sounds like an oxymoron but in this case it is a reality that you can enjoy on your computer(s). Why is that? Sorcerer provides handful (simple) of commands that give you the total control over the installation, tracking and management of software on your Linux computer (powerful). However, Sorcerer is not intended for beginners in system administration because it assumes that the system administrator (SA) knows how to (and wants to do it) configure and tweak the system and installed software to suit his/her needs.

I was able to find on the wayback machine an old gpl build, it's basically a puppy2 level. I verified it is complicated to build and not for our wants.

I parallel with sourcerer was gentoo, which looks at building from another direction. It still exists but it is too tied into itself for our purposes. I had looked at it in detail some time ago.

Sourcerer had a number of offshoots when it went non-gpl. At least two of these, Lunar-linux and Source Mage survived into time, a lot didn't.

Starting with Lunar, it's just as easy to quote them even if it's a little long, rather than paraphrase, so I do.

Quote:

Lunar-Linux is a source based distribution. This means that lunar is just like any other distribution, except that you compile packages instead of installing precompiled packages. That really is the only difference with so-called "binary" distributions.

This means that you have the control over how you compile packages yourself, instead of relying on the willingness of others to provide you the features of software that you need or want. This control is what allows you to do the following with lunar that other distributions typically don't allow you to do:

* optimize your entire system for speed, or size
* sacrifice stability for speed or vice versa
* insert or remove features of packages that require recompiling
* upgrade core system components without reinstalling

But this is just an incomplete list. There are many more benefits that are not mentioned here.

The downside is that you need to spend time compiling packages. However, with the current power of systems, this hardly stops people from choosing such an approach!

A little history about lunar: Lunar-Linux was born as a fork from Sorcerer GNU Linux (SGL). Nowadays SGL doesn't exist anymore because the original author (Kyle Sallee) retracted his GPL code and distributes it under a non-OSI approved license, and renamed it "Sorcerer". Another group of people also forked SGL into SourceMage GNU Linux, and they work side by
side Lunar-Linux, often sharing concepts and ideas. Lunar exists since early 2002.

Lunar will provide you with an excellent platform for the following tasks:

But be warned: depending on your needs you might need to invest a large amount of time into your system. Lunar is not a hands-off distribution per se, although it can be under specific circumstances.

Looking into it, a lot of lunar is dead and unreachable. I believe also believe that for our purposes it was unsuitable for what I was chasing, it's build process is just not puppyable, which is a shame as they did have a good set of package build tools.
EDIT1:I may revisit this later as i've just received a url for a later iso.
EDIT2: Lunar has gone mixed/proprietory foir a licence so it's now OUT!

On to Source Mage, and I won't keep you long at present.

Source Mage has been in the main systemd'd, but it looks like a non-systemd version is coming using openRC instead. Source Mage has two mediums to work from. One installs via a iso, supposedly a 20min task, but you need to know details including what packages are called, etc first. I left it alone. It was, even with it's instructions complicated. The second and seemingly more popular is a chroot version, this has both systemd versions and a new non-systemd version. Being chroot (think switchroot), it can be loaded on top of an existing linux vaersion and run from there. Their only real problem seems (and they admit it) is documentation is out the door while a few dedicated people have been working and using it, so it's not dead. I had a quick run, enough to make me intend to dedicate one of my test boxes to do a propper workthrough too see what it can do. However, I don't think it will be suitable as once more it's build system seems like it's too integrated for puppying. A shame as I do like the grimmore auto-build system as it's created to build packages with, and it can even rebuild existing packages with configuration changes when needed as part of the process of building and installing anew package.

I'd love to be able to use Source Mage's build system, but it doesn't seem 'splitable' from Source Mage itself. However I shall see, maybe using puppy as a base, chroot'ing into a Source Mage Puppy and working from that will be possible. They seem to mainly use ldxe as desktop at least but i'd like to jwm them.

I do consider them a contender!

and I was working with that WHEN THINGS STARTED GETTING MESSY........

Some of my email replies brought in good comments, some just horid, and a strange reply or three!

However, out of this came a list of other build systems to try out that also build 'dynamically'.

A few were clearly just ideas that were a variation on other existing ones.

Others just a first try to do...

And, a very few, are actually worth following through and looking at.

It all made me go back to consider what I was really attempting to do and to write it all down on a whiteboard and sort it out rather than just in my head.

My next missive will contain a "Proposed Mission Statement".

And so I/we will go on into March with something to do, and a clearer idea of what I want out of this whole idea!

And THEN I shall open the thread up for comments.Last edited by scsijon on Sat 28 Apr 2018, 20:49; edited 1 time in total

Although the title is right and I think will be something like what i'm going to call my Puppy, i'm finding new things while chasing around, that may be of use to others that are of more use in the other Topic than this one. I shall

As an example, the reason most of us can't boot from our sd/microsd card slots is that most of these slots are PCIe NVMe connected. They need both the linux driver enabled and built, as well as the system's 'bios' firmware NVMe aware.http://murga-linux.com/puppy/viewtopic.php?p=983179#983179 talks about it for those who missed it. It seems it's more than just SSD's that use it.

EDIT1: Interesting, I was loaned a ufs PCIe card and tried it in a PCIe slot in one of my laptops and it was seen. This is one of the laptops that won't see a NVME card. Since the ufs cards are still M.2 2280 they fit and when released everywhere should make people happy. They are also suppose to be cheaper than the NVME cards but have a 5yr warrenty rather than a set number of ops (reads/writes) as NVME have.Last edited by scsijon on Sat 28 Apr 2018, 20:58; edited 2 times in total

My origonal idea was to use a simple and very basic puppy, adding to it a Package Builder/Installer system such as sourcerer had. This would have had a small number of auto-build sets available, such as one for our usual Puppy Packages, that would have turned the basic environment into a working Puppy and allow users to install new packages as wanted "on the fly'. Could happen, but not worth the effort and trouble to keep it up to date and bug-free, I "archived" this..

Second idea was to use the simple puppy, switch-root into a Source Mage environment, and using their facilities and package config files, go from there. Could happen, but their structure is multi-level (init Levels) and multi-user (no root users), systemd, and quite a few other differances from puppy, so that has also been "archived".

Third idea was to use one of the other self-build systems called Gentoo and just do an imitation puppy. With a little thought and a lot of 'play', it's not happening, it might look like puppy, but it doesn't act like puppy. "Archived".

Fourth was Lunar. I've already talked about that earlier. "Archived".

Fifth to seventh were similar but using three other good self-build systems, basically all failed at different points to supply what I was aiming for. All "archived". There were a number of others I looked at 'on-paper' that didn't even get to that point.

I was pointed at Void Linux at one point, but was busy on others looking more promising. Came back to it, and after doing some research, it might just work. Build a simple, non-xorg puppy, either just add a few packages to be able to run xbps, void's package manager (builder/installer/updater), use their site package-dataset (or download it and run it local as it is designed to be able to do and work from). The xbps system comes as a self contained package. So I shall use March/April and a spare box to install Void Linux and try it, as well as use BarryK's latest woofq to build a simple puppy with the few extra packages to go from there if it's a 'possible'.

I'll also at this time open the thread up for comments.

I'm not looking for build help at present, I want to work through a number of issues in my head and on my whiteboards first, however if others want to they could always download and play with void and give me their thoughts.

OH yes, Void's build source packageset is nearly 9,000 at present, splits things like gtk2, gtk3 versions of packages as well as those for the various qt versions, as well as being multi-lingual.

Also xbps has a creator that allows you to build it's builders template and add to your set or upload to their database, or any at a site you direct. And it has JWM, so maybe our jwm enhancements can be added back to void as a thank you for allowing us to use their product.

xbps is under a Simplified BSD license (2 clause) that I don't think will cause us too many problems.

Quote:

/*
* Copyright (c) 2008-2016 Juan Romero Pardines and contributors
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
* IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
* IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
* NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
* THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/

I don't intend replacing PPM, just adding xbpm as a source-builder-installer.

And that should do for the moment, I shall no doubt add more "after a thought or two".

Although the title is right and I think will be something like what i'm going to call my Puppy, i'm finding new things while chasing around, that may be of use to others that are of more use in the other Topic than this one. I shall

As an example, the reason most of us can't boot from our sd/microsd card slots is that most of these slots are PCIe NVMe connected. They need both the linux driver enabled and built, as well as the system's 'bios' firmware NVMe aware.http://murga-linux.com/puppy/viewtopic.php?p=983179#983179 talks about it for those who missed it. It seems it's more than just SSD's that use it.

Ah, I compiled kernel 4.14.20 in latest EasyOS, version 0.8, with this nvm driver as a module. Looks like I will have to redo it.

Note, I think that sd/mmc interfaces are not pci-e specifically, and pre-date nvm. The sd interface chip hangs off the pci bus I think.

It seems that bios/uefi firmware just doesn't consider the sd-card interface chip as a possible boot source, at least in all my laptops.

Unless the sd-card chip is connected via the usb bus, not pci. This can be either external or internal. One of my laptops has sd-card slot and it shows up as /dev/sd* rather than mmcblk*, meaning that it is internally connected via usb.
Meaning, it should be possible to boot from it._________________http://bkhome.org/news/

Ah, I compiled kernel 4.14.20 in latest EasyOS, version 0.8, with this nvm driver as a module. Looks like I will have to redo it.

Note, I think that sd/mmc interfaces are not pci-e specifically, and pre-date nvm. The sd interface chip hangs off the pci bus I think.

It seems that bios/uefi firmware just doesn't consider the sd-card interface chip as a possible boot source, at least in all my laptops.

Unless the sd-card chip is connected via the usb bus, not pci. This can be either external or internal. One of my laptops has sd-card slot and it shows up as /dev/sd* rather than mmcblk*, meaning that it is internally connected via usb.
Meaning, it should be possible to boot from it.

The problem in part is that it needs the bios firmware component installed and enabled to boot and most don't even have that firmware for everything hard coded to start from. Also we are also talking PCIe, not pci, a little bit different and yes they predate the NVMe component, but use the same PCIe interface and driver for some reason.

To start with, have a look at what your wan (wifi) device is connected as. These are M.2 2228 cards usually, unless on the motherboard itself and usually connected via PCIe. Most chipsets have 2/4 PCIe ports even if not all used. Also have a look in your bios startup and see if you can startup the notebook from the wifi. If it's off try turning it on if your sd won't boot, it might just be what is needed as some settings are windows-centric in names.

Also, often there is an extra socket for a WWAN Card (think mobile phone data), but just installing a card is not just it, you need it factory installed as the antennae is usually buiilt into your laptop video screen, and it's different from the wifi antenae so you can't just do a quick changeover, just any doesn't work.

I've come across some software that if you have a camera can use eye-movement and your screen to follow, i'll add this next in this message when i'm back at base, weather it's linux or not will depend on what i find.

Package is TOBII, at tobii.com, can someone have a look please, I just don't have enough time.Last edited by scsijon on Sun 11 Mar 2018, 00:37; edited 1 time in total

I've installed Void to a spare partition. Downloaded a live iso with LXDE (32bit). All the desktops/window managers are unconfigured, so the menus don't do a lot. It is really a good idea for anyone trying it out to read the wiki first, as you get no info from the distro itself. Wifi was configured during install, and actually worked, which is good as there is no network manager included. Installation was quite quick and straightforward - just skipped the Partition and Bootloader bits. With Grub4Dos, it just needs "root=/dev/sdaX" adding to the kernel line.
The only gripe I have is that my keyboard selection is not working although it appears to have been set up. [EDIT] My bad - had selected 'uk' but it should have been 'gb'. Changed it with setxkbdmap now.
Also, switching a console to root (using su) results in the arrow keys not working, which can be a pain. eg they just do this:
Code:
# ^[[A^[[C^[[A^[[B^[[D
so no history or moving along the line etc. They work fine as a normal user.
[EDIT] - discovered that the default root shell is sh (linked to dash I believe), while normal user has bash.
Otherwise I quite like it. Been slavishly updating with XBPS, and installed a few things. It lacks the likes of wget or a graphical text editor etc.
I've registered with the forum so will see if I can get the keymap sorted out. [not needed now] Also tempted to just re-install with a barebones version and build it up from there, now that I have a better idea of how it works. I intend to persist with it one way or another anway.

Yes, sh is designed for the old 88key keyboard, everything else had to be individually mapped in a keyboard config file and a setting set to be used to enable them.

1- I was working on using the basic void system (no desktops etc.) and trying to build a 'puppy' on top of it by just adding packages. Something like building a puppy 2.01 equivalent as a starting point. After all we do know most of what is in a puppy thanks to the .packages/woof-installed-packages file. With the rest being in woof's ~/woof-code/~ and the like parts of the woof builder, they can be added when found missing.
It will need some of our puppy 'specials' like a .pet/.pup packager/installer (at present individually in woof's ~/woof-code/~), the PPM, the jwm addons we have and the like packaged in a standard format package. Also, they don't have to actually build, you can set it to just install the package. Directory structure will be king in maintaining things here.

2- On the other side, XBPS (xbps-master) and it's builder side (void-packages-master) are both available as packages themselves. Adding what few other packages are needed. These could be 'reworked' for our 'puppy world' (if must be, and couldn't be used by adding a config file to puppy instead) and added to something like one of barry's latest quirky's as it is an keep-up-to-date system. It seems the easier way, but as i've found in the past, it rarely is and is definately the more time consuming.

I had been considering trying both on different sets of hardware as each will provide different results and different learning information.

-i'll put more in here when i have time, as it's time to suit up

EDIT: And I thought someone was 'having a lean on me', but there is actually a package called B.A.T.M.A.N. The Better Approach To Mobile Adhoc Networking (B.A.T.M.A.N.) is a routing protocol for multi-hop mobile ad hoc networks which is under development by the German "Freifunk" community and intended to replace the link state routing protocol OLSR to quote wikapedia. Came across it from a message to have a look at batman-adv. Not sure whyhttps://git.open-mesh.org/batman-adv.git if anyones interested.Last edited by scsijon on Thu 15 Mar 2018, 13:25; edited 1 time in total

And as quickly as SSD and eMMC are now appearing in laptops and tablets with a decent price, now we have UFS appearing. Suppose to be replacing both AND our favorite sd. Expect to see them appearing in laptops and tablets from now on.
Details are herehttps://www.kernel.org/doc/Documentation/scsi/ufs.txt for those interested. And another setting for the kernel (sorry Barry) to add.

Also Qualcomm is spreading it's wings with a dual phone/tablet.laptop 8core+extras processor https://www.qualcomm.com/products/msm8998. It uses the Qualcomm® Kryo™ 280 CPU so it's not a slouch either. Haven't checked if it's in the kernel yet, but I suspect so. There are 5 brands releasing 2-in-1 laptops/tablets so far, but before you buy one check out the reviews, some of them are horrible but others are great.

Dnsmasq provides network infrastructure for small networks: DNS, DHCP, router advertisement and network boot. It is designed to be lightweight and have a small footprint, suitable for resource constrained routers and firewalls. It has also been widely used for tethering on smartphones and portable hotspots, and to support virtual networking in virtualisation frameworks. Supported platforms include Linux (with glibc and uclibc), Android, *BSD, and Mac OS X. Dnsmasq is included in most Linux distributions and the ports systems of FreeBSD, OpenBSD and NetBSD. Dnsmasq provides full IPv6 support.

The DNS subsystem provides a local DNS server for the network, with forwarding of all query types to upstream recursive DNS servers and caching of common record types (A, AAAA, CNAME and PTR, also DNSKEY and DS when DNSSEC is enabled).
Local DNS names can be defined by reading /etc/hosts, by importing names from the DHCP subsystem, or by configuration of a wide range of useful record types.
Upstream servers can be configured in a variety of convenient ways, including dynamic configuration as these change on moving upstream network.
Authoritative DNS mode allows local DNS names may be exported to zone in the global DNS. Dnsmasq acts as authoritative server for this zone, and also provides zone transfer to secondaries for the zone, if required.
DNSSEC validation may be performed on DNS replies from upstream nameservers, providing security against spoofing and cache poisoning.
Specified sub-domains can be directed to their own upstream DNS servers, making VPN configuration easy.
Internationalised domain names are supported.

The DHCP subsystem supports DHCPv4, DHCPv6, BOOTP and PXE.
Both static and dynamic DHCP leases are supported, along with stateless mode in DHCPv6.
The PXE system is a full PXE server, supporting netboot menus and multiple architecture support. It includes proxy-mode, where the PXE system co-operates with another DHCP server.
There is a built in read-only TFTP server to support netboot.
Machines which are configured by DHCP have their names automatically included in the DNS and the names can specified by each machine or centrally by associating a name with a MAC address or UID in the dnsmasq configuration file.

The Router Advertisement subsystem provides basic autoconfiguration for IPv6 hosts. It can be used stand-alone or in conjunction with DHCPv6.
The M and O bits are configurable, to control hosts' use of DHCPv6.
Router advertisements can include the RDNSS option.
There is a mode which uses name information from DHCPv4 configuration to provide DNS entries for autoconfigured IPv6 addresses which would otherwise be anonymous.

For extra compactness, unused features may be omitted at compile time.

You would think that some laptop makers would be happy with using a single PCIe NVME SSD for their top end laptops wouldn't you!, after all you can get them up to 1tb nowadays. No, they've decided to up the storage and throughput a notch by also running them in raid mode using a pair of ssd's for extra speed.
EDIT: Just had my head blown out! Apparently with an intel chipset you can have up to 20 NVME SSD into one raid system per processor, with AMD it's 10 at present. But considering a single NVME card works with i/o adverageing 3.8 microseconds..... BLOODY FAST is all I can say.

Dell, HP and lenovo all now have them. And they don't tell you that's the configuration until you try them out as the same model may be either. It seems to depend on the processor and video as to which is supplied.

Unfortunately, at present, Barry's latest Easy Pyro64 won't recognize them as having a drive of any type, even with the latest kernel kit. (Sorry barry, something else is needed to be set in the kernel.)
EDIT: OOPS, seems like I added the dev and buuild updates to easy pyro64 but forgot to add the 'main' pet. Will do this evening and test again. Fun tomorrow with the dual screen laptops coming for the 'day'.
EDIT: Seems like there are a few gotchas to set in the bios to have a nvme boot system. I'll document them when I have mine right.

Mind you I do like the 15.5" 4k screens and are seriously thinking of buying one, although the +$3K cost is nothing to sneeeze at for something good enough to handle it all..

With a 'bit' of late night reading (2am) on the web i've found that there are some changes needed in your setup to handle PCIe NVME SSD's. Strangely their mainly in the sata section rather than any pcie related section. Especially when your SSD is set up in raid mode as most seem to be by the manufacturer.

I'll add a detailed topic in the HOWTO Topic when I have it sorted out.

I'll add some links in this tonight that may in the short term help.

Oh, yes, there are a few other settings needed in the kernel. I'll detail them at the same time.

Just a warning about a number of the cheaper laptops listing their Storage just as SSD.

It seems that both 128GB and 256GB SSD sizes are now available in emmc with M.2 2280 pcie and sata connection modules out there, and a few suppliers are using them.

I did see one in a sata 3 2.5" drive format also listed, but it seemed to be only for release in the asian market at present.

However, doing some reading about emmc, i'm not sure i'd worry at present as long as your not running any caches.

For those wondering about Sourcery, i'll be back at it after Australia's ANZAC Day. The break from it has also allowed me 'in my spare time' to formulate some plans for the way to proceed forward without having to build a complete new puppy.

I'm now considering using BarryK's Pyro (64bit for now) as the base as it's using 'modern' releases of packages and since he has it to beta state. It should be the easiest one to work with.

You cannot post new topics in this forumYou cannot reply to topics in this forumYou cannot edit your posts in this forumYou cannot delete your posts in this forumYou cannot vote in polls in this forumYou cannot attach files in this forumYou can download files in this forum