From demeler at biochem.uthscsa.edu Thu Jul 21 08:53:35 2016
From: demeler at biochem.uthscsa.edu (Borries Demeler)
Date: Thu Jul 21 08:53:39 2016
Subject: [SATLUG] diskless ramdisk booting
Message-ID: <20160721135334.GA1937@biochem.uthscsa.edu>
I am hoping someone on this list can explain how diskless booting works.
I have a bunch of high school kids in my lab this summer and we are
trying to build a diskless linux cluster as a summer project. I have a
bunch of generic boxes with quad-core processors, on board video and 8
GB RAM each. We configured one of the boxes with CentOS 6.8 as a head
node. It has dhcpd, tftpd, nfs working fine now. We can successfully
pxe boot various iso images served through the tftp server, the nodes
get private IP addresses, tftp boot the kernels and ramdisks on the iso
images, and the nodes can mount nfs shared volumes from the head node.
All of this works fine.
We now wanted to build our own custom image and wanted to compile a
minimal kernel and build a minimal ramdisk on which we could nfs mount
various shares that hold the software and home directories we want to have
accessible on the slave nodes.
But we so far have been unable to build a proper kernel or ramdisk that
gets past the mounting point where the root ramdisk file system gets
mounted and cannot figure out where we are going wrong. Any help would
be appreciated. We tried to follow this guide to the T:
http://eduardo-lago.blogspot.com/2012/06/ram-only-pxe-boot-smallest-diskless.html
and searched the internet for all kinds of other documentation on the topic.
But we must be missing something essential, like our init is not getting
executed or the kernel cannot find the root file system.
Any help for these kids would be greatly appreciated! Can someone explain
how the boot process works exactly step-by-step so we can see where our
newly built image might be failing? THe boot process always fails at the
point where the initrd specified ramdisk does not mount root.
Thanks, -Borries
From igor.gueths at RACKSPACE.COM Thu Jul 21 10:55:03 2016
From: igor.gueths at RACKSPACE.COM (Igor Gueths)
Date: Thu Jul 21 10:55:09 2016
Subject: [SATLUG] diskless ramdisk booting
In-Reply-To: <20160721135334.GA1937@biochem.uthscsa.edu>
References: <20160721135334.GA1937@biochem.uthscsa.edu>
Message-ID: <4E7BCC7C-821E-4E61-82E5-E67E37D66F12@rackspace.com>
Hello,
based on your initial description of the problem, it sounds as though the kernel is panicking because it cannot find the root filesystem. That being said, at a high level diskless booting works something like this (assuming of course you are in fact PXE booting):
?BIOS passes control to Eprom on the NIC.
?Boot loader on Eprom attempts to get an IP via DHCP, paying special attention to the TFTP-related DHCP options to determine where its TFTP server is.
?Kernel is downloaded from TFTP server, decompressed and executed.
I unfortunately do not recall if one has to hardcode NFS mount points and such into the kernel image, or if as part of passing control to the kernel a separate config file gets downloaded from which further boot options are passed to the kernel. That being said, I would check to see what sort of message (s) you are getting at boot time when the failure occurs and go from there. Hope this helps, and good luck.
> On Jul 21, 2016, at 8:53 AM, Borries Demeler wrote:
>
> I am hoping someone on this list can explain how diskless booting works.
>
> I have a bunch of high school kids in my lab this summer and we are
> trying to build a diskless linux cluster as a summer project. I have a
> bunch of generic boxes with quad-core processors, on board video and 8
> GB RAM each. We configured one of the boxes with CentOS 6.8 as a head
> node. It has dhcpd, tftpd, nfs working fine now. We can successfully
> pxe boot various iso images served through the tftp server, the nodes
> get private IP addresses, tftp boot the kernels and ramdisks on the iso
> images, and the nodes can mount nfs shared volumes from the head node.
> All of this works fine.
>
> We now wanted to build our own custom image and wanted to compile a
> minimal kernel and build a minimal ramdisk on which we could nfs mount
> various shares that hold the software and home directories we want to have
> accessible on the slave nodes.
>
> But we so far have been unable to build a proper kernel or ramdisk that
> gets past the mounting point where the root ramdisk file system gets
> mounted and cannot figure out where we are going wrong. Any help would
> be appreciated. We tried to follow this guide to the T:
>
> http://eduardo-lago.blogspot.com/2012/06/ram-only-pxe-boot-smallest-diskless.html
>
> and searched the internet for all kinds of other documentation on the topic.
> But we must be missing something essential, like our init is not getting
> executed or the kernel cannot find the root file system.
>
> Any help for these kids would be greatly appreciated! Can someone explain
> how the boot process works exactly step-by-step so we can see where our
> newly built image might be failing? THe boot process always fails at the
> point where the initrd specified ramdisk does not mount root.
>
> Thanks, -Borries
> --
> _______________________________________________
> SATLUG mailing list
> SATLUG@satlug.org
> http://alamo.satlug.org/mailman/listinfo/satlug to manage/unsubscribe
> Powered by Rackspace (www.rackspace.com)
From bruce.dubbs at gmail.com Thu Jul 21 10:55:29 2016
From: bruce.dubbs at gmail.com (Bruce Dubbs)
Date: Thu Jul 21 10:55:38 2016
Subject: [SATLUG] diskless ramdisk booting
In-Reply-To: <20160721135334.GA1937@biochem.uthscsa.edu>
References: <20160721135334.GA1937@biochem.uthscsa.edu>
Message-ID: <5790F071.2070605@gmail.com>
Borries Demeler wrote:
> I am hoping someone on this list can explain how diskless booting works.
>
> I have a bunch of high school kids in my lab this summer and we are
> trying to build a diskless linux cluster as a summer project. I have a
> bunch of generic boxes with quad-core processors, on board video and 8
> GB RAM each. We configured one of the boxes with CentOS 6.8 as a head
> node. It has dhcpd, tftpd, nfs working fine now. We can successfully
> pxe boot various iso images served through the tftp server, the nodes
> get private IP addresses, tftp boot the kernels and ramdisks on the iso
> images, and the nodes can mount nfs shared volumes from the head node.
> All of this works fine.
>
> We now wanted to build our own custom image and wanted to compile a
> minimal kernel and build a minimal ramdisk on which we could nfs mount
> various shares that hold the software and home directories we want to have
> accessible on the slave nodes.
>
> But we so far have been unable to build a proper kernel or ramdisk that
> gets past the mounting point where the root ramdisk file system gets
> mounted and cannot figure out where we are going wrong. Any help would
> be appreciated. We tried to follow this guide to the T:
>
> http://eduardo-lago.blogspot.com/2012/06/ram-only-pxe-boot-smallest-diskless.html
>
> and searched the internet for all kinds of other documentation on the topic.
> But we must be missing something essential, like our init is not getting
> executed or the kernel cannot find the root file system.
>
> Any help for these kids would be greatly appreciated! Can someone explain
> how the boot process works exactly step-by-step so we can see where our
> newly built image might be failing? THe boot process always fails at the
> point where the initrd specified ramdisk does not mount root.
Have you made a custom initrd? See if
http://www.linuxfromscratch.org/blfs/view/stable/postlfs/initramfs.html
helps. You will, of course, need to modify the init.in file.
-- Bruce
From mayfield_mark at gvtc.com Thu Jul 21 11:29:16 2016
From: mayfield_mark at gvtc.com (Mark Mayfield)
Date: Thu Jul 21 11:29:20 2016
Subject: [SATLUG] diskless ramdisk booting
In-Reply-To: <5790F071.2070605@gmail.com>
References: <20160721135334.GA1937@biochem.uthscsa.edu>
<5790F071.2070605@gmail.com>
Message-ID:
Depending on how you want to do it there are some kernel compile options
you need to set.
NFS capability in the kernel, NFSRoot, dhcp in the kernel and make sure
you get your network card drivers in the kernel also.
Pretty much all these will not be enabled by default in the kernel
config. If you do everything right you don't even need a ramdisk. You
can do some of this in the ramdisk but I had better luck just doing it
in the kernel. The biggest problem I faced was making sure your network
drivers are in the kernel or ramdisk. If you know you will be using one
type of nic card it isn't a big problem but if you will use different
machines it is more problematic. I've used ramdisk creation options in
both fedora (dracut) and debian (mkinitrd I think) that tell the system
to put all (or at least nearly all) modules in the ramdisk. Also you
have set up your fstab right and if your using a ramdisk make sure the
fstab configured for netboot makes it in the ramdisk. Hope the info
helps, I haven't messed with it in a while but I worked with net boot
systems quite a bit a few years back. At A&M I had a pxe server with a
bunch of distro's it, and a custom boot CD for machines outside of the
dhcp scope that could still hit the NFS servers.
On 07/21/2016 10:55 AM, Bruce Dubbs wrote:
> Borries Demeler wrote:
>> I am hoping someone on this list can explain how diskless booting works.
>>
>> I have a bunch of high school kids in my lab this summer and we are
>> trying to build a diskless linux cluster as a summer project. I have a
>> bunch of generic boxes with quad-core processors, on board video and 8
>> GB RAM each. We configured one of the boxes with CentOS 6.8 as a head
>> node. It has dhcpd, tftpd, nfs working fine now. We can successfully
>> pxe boot various iso images served through the tftp server, the nodes
>> get private IP addresses, tftp boot the kernels and ramdisks on the iso
>> images, and the nodes can mount nfs shared volumes from the head node.
>> All of this works fine.
>>
>> We now wanted to build our own custom image and wanted to compile a
>> minimal kernel and build a minimal ramdisk on which we could nfs mount
>> various shares that hold the software and home directories we want to
>> have
>> accessible on the slave nodes.
>>
>> But we so far have been unable to build a proper kernel or ramdisk that
>> gets past the mounting point where the root ramdisk file system gets
>> mounted and cannot figure out where we are going wrong. Any help would
>> be appreciated. We tried to follow this guide to the T:
>>
>> http://eduardo-lago.blogspot.com/2012/06/ram-only-pxe-boot-smallest-diskless.html
>>
>>
>> and searched the internet for all kinds of other documentation on the
>> topic.
>> But we must be missing something essential, like our init is not getting
>> executed or the kernel cannot find the root file system.
>>
>> Any help for these kids would be greatly appreciated! Can someone
>> explain
>> how the boot process works exactly step-by-step so we can see where our
>> newly built image might be failing? THe boot process always fails at the
>> point where the initrd specified ramdisk does not mount root.
>
> Have you made a custom initrd? See if
>
> http://www.linuxfromscratch.org/blfs/view/stable/postlfs/initramfs.html
>
> helps. You will, of course, need to modify the init.in file.
>
> -- Bruce
>
From demeler at biochem.uthscsa.edu Thu Jul 21 11:41:50 2016
From: demeler at biochem.uthscsa.edu (Borries Demeler)
Date: Thu Jul 21 11:41:52 2016
Subject: [SATLUG] diskless ramdisk booting
In-Reply-To: <4E7BCC7C-821E-4E61-82E5-E67E37D66F12@rackspace.com>
References: <20160721135334.GA1937@biochem.uthscsa.edu>
<4E7BCC7C-821E-4E61-82E5-E67E37D66F12@rackspace.com>
Message-ID: <20160721164150.GA7633@biochem.uthscsa.edu>
On Thu, Jul 21, 2016 at 03:55:03PM +0000, Igor Gueths wrote:
> Hello,
> based on your initial description of the problem, it sounds as though the kernel is panicking because it cannot find the root filesystem. That being said, at a high level diskless booting works something like this (assuming of course you are in fact PXE booting):
> ???BIOS passes control to Eprom on the NIC.
> ???Boot loader on Eprom attempts to get an IP via DHCP, paying special attention to the TFTP-related DHCP options to determine where its TFTP server is.
> ???Kernel is downloaded from TFTP server, decompressed and executed.
Yes, that's exactly how far we get with the pxe boot.
> I unfortunately do not recall if one has to hardcode NFS mount points and such into the kernel image, or if as part of passing control to the kernel a separate config file gets downloaded from which further boot options are passed to the kernel. That being said, I would check to see what sort of message (s) you are getting at boot time when the failure occurs and go from there. Hope this helps, and good luck.
Our first try is to boot a small self-contained initramfs we built by hand that has a
init script as well as busybox built into it. The nsf mount points can be put into /etc/fstab
on the ram image and can be loaded as long as the kernel supports nfs. When we boot an
image from, say CentOS installation media, this part works perfectly well, and we can
access nfs mounted dirs just fine, but our own initramfs doesn't seem to work, or
maybe it is the init.
-b.
> > On Jul 21, 2016, at 8:53 AM, Borries Demeler wrote:
> >
> > I am hoping someone on this list can explain how diskless booting works.
> >
> > I have a bunch of high school kids in my lab this summer and we are
> > trying to build a diskless linux cluster as a summer project. I have a
> > bunch of generic boxes with quad-core processors, on board video and 8
> > GB RAM each. We configured one of the boxes with CentOS 6.8 as a head
> > node. It has dhcpd, tftpd, nfs working fine now. We can successfully
> > pxe boot various iso images served through the tftp server, the nodes
> > get private IP addresses, tftp boot the kernels and ramdisks on the iso
> > images, and the nodes can mount nfs shared volumes from the head node.
> > All of this works fine.
> >
> > We now wanted to build our own custom image and wanted to compile a
> > minimal kernel and build a minimal ramdisk on which we could nfs mount
> > various shares that hold the software and home directories we want to have
> > accessible on the slave nodes.
> >
> > But we so far have been unable to build a proper kernel or ramdisk that
> > gets past the mounting point where the root ramdisk file system gets
> > mounted and cannot figure out where we are going wrong. Any help would
> > be appreciated. We tried to follow this guide to the T:
> >
> > http://eduardo-lago.blogspot.com/2012/06/ram-only-pxe-boot-smallest-diskless.html
> >
> > and searched the internet for all kinds of other documentation on the topic.
> > But we must be missing something essential, like our init is not getting
> > executed or the kernel cannot find the root file system.
> >
> > Any help for these kids would be greatly appreciated! Can someone explain
> > how the boot process works exactly step-by-step so we can see where our
> > newly built image might be failing? THe boot process always fails at the
> > point where the initrd specified ramdisk does not mount root.
> >
> > Thanks, -Borries
> > --
> > _______________________________________________
> > SATLUG mailing list
> > SATLUG@satlug.org
> > http://alamo.satlug.org/mailman/listinfo/satlug to manage/unsubscribe
> > Powered by Rackspace (www.rackspace.com)
>
> --
> _______________________________________________
> SATLUG mailing list
> SATLUG@satlug.org
> http://alamo.satlug.org/mailman/listinfo/satlug to manage/unsubscribe
> Powered by Rackspace (www.rackspace.com)
From demeler at biochem.uthscsa.edu Thu Jul 21 11:42:23 2016
From: demeler at biochem.uthscsa.edu (Borries Demeler)
Date: Thu Jul 21 11:42:24 2016
Subject: [SATLUG] diskless ramdisk booting
In-Reply-To: <5790F071.2070605@gmail.com>
References: <20160721135334.GA1937@biochem.uthscsa.edu>
<5790F071.2070605@gmail.com>
Message-ID: <20160721164223.GB7633@biochem.uthscsa.edu>
On Thu, Jul 21, 2016 at 10:55:29AM -0500, Bruce Dubbs wrote:
> Borries Demeler wrote:
> >I am hoping someone on this list can explain how diskless booting works.
> >
> >I have a bunch of high school kids in my lab this summer and we are
> >trying to build a diskless linux cluster as a summer project. I have a
> >bunch of generic boxes with quad-core processors, on board video and 8
> >GB RAM each. We configured one of the boxes with CentOS 6.8 as a head
> >node. It has dhcpd, tftpd, nfs working fine now. We can successfully
> >pxe boot various iso images served through the tftp server, the nodes
> >get private IP addresses, tftp boot the kernels and ramdisks on the iso
> >images, and the nodes can mount nfs shared volumes from the head node.
> >All of this works fine.
> >
> >We now wanted to build our own custom image and wanted to compile a
> >minimal kernel and build a minimal ramdisk on which we could nfs mount
> >various shares that hold the software and home directories we want to have
> >accessible on the slave nodes.
> >
> >But we so far have been unable to build a proper kernel or ramdisk that
> >gets past the mounting point where the root ramdisk file system gets
> >mounted and cannot figure out where we are going wrong. Any help would
> >be appreciated. We tried to follow this guide to the T:
> >
> >http://eduardo-lago.blogspot.com/2012/06/ram-only-pxe-boot-smallest-diskless.html
> >
> >and searched the internet for all kinds of other documentation on the topic.
> >But we must be missing something essential, like our init is not getting
> >executed or the kernel cannot find the root file system.
> >
> >Any help for these kids would be greatly appreciated! Can someone explain
> >how the boot process works exactly step-by-step so we can see where our
> >newly built image might be failing? THe boot process always fails at the
> >point where the initrd specified ramdisk does not mount root.
>
> Have you made a custom initrd? See if
>
> http://www.linuxfromscratch.org/blfs/view/stable/postlfs/initramfs.html
>
> helps. You will, of course, need to modify the init.in file.
>
> -- Bruce
>
> --
> _______________________________________________
> SATLUG mailing list
> SATLUG@satlug.org
> http://alamo.satlug.org/mailman/listinfo/satlug to manage/unsubscribe
> Powered by Rackspace (www.rackspace.com)
Thanks, Bruce, we'll check it out.
-b.
From demeler at biochem.uthscsa.edu Thu Jul 21 11:45:15 2016
From: demeler at biochem.uthscsa.edu (Borries Demeler)
Date: Thu Jul 21 11:45:16 2016
Subject: [SATLUG] diskless ramdisk booting
In-Reply-To:
References: <20160721135334.GA1937@biochem.uthscsa.edu>
<5790F071.2070605@gmail.com>
Message-ID: <20160721164515.GF7633@biochem.uthscsa.edu>
On Thu, Jul 21, 2016 at 11:29:16AM -0500, Mark Mayfield wrote:
>
> Depending on how you want to do it there are some kernel compile
> options you need to set.
>
> NFS capability in the kernel, NFSRoot, dhcp in the kernel and make
> sure you get your network card drivers in the kernel also.
>
> Pretty much all these will not be enabled by default in the kernel
> config. If you do everything right you don't even need a ramdisk.
> You can do some of this in the ramdisk but I had better luck just
> doing it in the kernel. The biggest problem I faced was making sure
> your network drivers are in the kernel or ramdisk. If you know you
> will be using one type of nic card it isn't a big problem but if you
> will use different machines it is more problematic. I've used
> ramdisk creation options in both fedora (dracut) and debian
> (mkinitrd I think) that tell the system to put all (or at least
> nearly all) modules in the ramdisk. Also you have set up your fstab
> right and if your using a ramdisk make sure the fstab configured for
> netboot makes it in the ramdisk. Hope the info helps, I haven't
> messed with it in a while but I worked with net boot systems quite a
> bit a few years back. At A&M I had a pxe server with a bunch of
> distro's it, and a custom boot CD for machines outside of the dhcp
> scope that could still hit the NFS servers.
Sounds great, thanks for the pointers, we'll look into it.
I believe we have all the drivers in there, but maybe we missed a critical
one. It is also possible our initramfs system, or the init script is not correct.
Thanks again, well keep you posted.
-b.
>
> On 07/21/2016 10:55 AM, Bruce Dubbs wrote:
> >Borries Demeler wrote:
> >>I am hoping someone on this list can explain how diskless booting works.
> >>
> >>I have a bunch of high school kids in my lab this summer and we are
> >>trying to build a diskless linux cluster as a summer project. I have a
> >>bunch of generic boxes with quad-core processors, on board video and 8
> >>GB RAM each. We configured one of the boxes with CentOS 6.8 as a head
> >>node. It has dhcpd, tftpd, nfs working fine now. We can successfully
> >>pxe boot various iso images served through the tftp server, the nodes
> >>get private IP addresses, tftp boot the kernels and ramdisks on the iso
> >>images, and the nodes can mount nfs shared volumes from the head node.
> >>All of this works fine.
> >>
> >>We now wanted to build our own custom image and wanted to compile a
> >>minimal kernel and build a minimal ramdisk on which we could nfs mount
> >>various shares that hold the software and home directories we
> >>want to have
> >>accessible on the slave nodes.
> >>
> >>But we so far have been unable to build a proper kernel or ramdisk that
> >>gets past the mounting point where the root ramdisk file system gets
> >>mounted and cannot figure out where we are going wrong. Any help would
> >>be appreciated. We tried to follow this guide to the T:
> >>
> >>http://eduardo-lago.blogspot.com/2012/06/ram-only-pxe-boot-smallest-diskless.html
> >>
> >>
> >>and searched the internet for all kinds of other documentation
> >>on the topic.
> >>But we must be missing something essential, like our init is not getting
> >>executed or the kernel cannot find the root file system.
> >>
> >>Any help for these kids would be greatly appreciated! Can
> >>someone explain
> >>how the boot process works exactly step-by-step so we can see where our
> >>newly built image might be failing? THe boot process always fails at the
> >>point where the initrd specified ramdisk does not mount root.
> >
> >Have you made a custom initrd? See if
> >
> >http://www.linuxfromscratch.org/blfs/view/stable/postlfs/initramfs.html
> >
> >helps. You will, of course, need to modify the init.in file.
> >
> > -- Bruce
> >
>
> --
> _______________________________________________
> SATLUG mailing list
> SATLUG@satlug.org
> http://alamo.satlug.org/mailman/listinfo/satlug to manage/unsubscribe
> Powered by Rackspace (www.rackspace.com)
From herrold at owlriver.com Thu Jul 21 11:56:27 2016
From: herrold at owlriver.com (R P Herrold)
Date: Thu Jul 21 11:56:36 2016
Subject: [SATLUG] diskless ramdisk booting
In-Reply-To:
References: <20160721135334.GA1937@biochem.uthscsa.edu>
<5790F071.2070605@gmail.com>
Message-ID:
> > Borries Demeler wrote:
> > > I am hoping someone on this list can explain how diskless booting works.
> > >
> > > I have a bunch of high school kids in my lab this summer and we are
> > > trying to build a diskless linux cluster as a summer project. I have a
> > > bunch of generic boxes with quad-core processors, on board video and 8
> > > GB RAM each. We configured one of the boxes with CentOS 6.8 as a head
> > > node. It has dhcpd, tftpd, nfs working fine now. We can successfully
> > > pxe boot various iso images served through the tftp server, the nodes
> > > get private IP addresses, tftp boot the kernels and ramdisks on the iso
> > > images, and the nodes can mount nfs shared volumes from the head node.
> > > All of this works fine.
> > >
> > > We now wanted to build our own custom image and wanted to compile a
> > > minimal kernel and build a minimal ramdisk on which we could nfs mount
> > > various shares that hold the software and home directories we want to have
> > > accessible on the slave nodes.
it may well make sense here to use an older (known working)
PXE, and TFTP client [see LTSP and an older K12 below], to
confirm you have functionality ... iptables restrictions, and
NFS as root (ot access a network swap and /home/) in NFSv4 are
possible points of breakage that would probably not be
accessible as security models ahve evolved
> > > But we so far have been unable to build a proper kernel or ramdisk that
> > > gets past the mounting point where the root ramdisk file system gets
> > > mounted and cannot figure out where we are going wrong. Any help would
> > > be appreciated. We tried to follow this guide to the T:
> > >
> > > http://eduardo-lago.blogspot.com/2012/06/ram-only-pxe-boot-smallest-diskless.html
It has been years since I set one of these installs up (I
built a 200 seat diskless installation for a customer). The
LTSP, and Jim McQuillan's work are probably a trailhead to use
http://www.ltsp.org/
As you can PXE boot, you are almost there. I set one up on
CentOS 6, but have not tried on C7 with the change to systemd
in the boot time scripts
Fedora seems to have taken the torch for the k12 Linux variant
https://fedorahosted.org/k12linux/
for a while, but the last time it seems to have been worked on
there [ltsp-5.1.95-6.fc19.src.rpm] is at F20 (so perhaps three
years ago)
good luck,
-- Russ herrold
From mayfield_mark at gvtc.com Thu Jul 21 13:33:07 2016
From: mayfield_mark at gvtc.com (Mark Mayfield)
Date: Thu Jul 21 13:33:10 2016
Subject: [SATLUG] diskless ramdisk booting
In-Reply-To: <20160721164515.GF7633@biochem.uthscsa.edu>
References: <20160721135334.GA1937@biochem.uthscsa.edu>
<5790F071.2070605@gmail.com>
<20160721164515.GF7633@biochem.uthscsa.edu>
Message-ID: <6ca48864-5348-14bd-9c59-fb1402af5f38@gvtc.com>
On 07/21/2016 11:45 AM, Borries Demeler wrote:
> On Thu, Jul 21, 2016 at 11:29:16AM -0500, Mark Mayfield wrote:
>> Depending on how you want to do it there are some kernel compile
>> options you need to set.
>>
>> NFS capability in the kernel, NFSRoot, dhcp in the kernel and make
>> sure you get your network card drivers in the kernel also.
>>
>> Pretty much all these will not be enabled by default in the kernel
>> config. If you do everything right you don't even need a ramdisk.
>> You can do some of this in the ramdisk but I had better luck just
>> doing it in the kernel. The biggest problem I faced was making sure
>> your network drivers are in the kernel or ramdisk. If you know you
>> will be using one type of nic card it isn't a big problem but if you
>> will use different machines it is more problematic. I've used
>> ramdisk creation options in both fedora (dracut) and debian
>> (mkinitrd I think) that tell the system to put all (or at least
>> nearly all) modules in the ramdisk. Also you have set up your fstab
>> right and if your using a ramdisk make sure the fstab configured for
>> netboot makes it in the ramdisk. Hope the info helps, I haven't
>> messed with it in a while but I worked with net boot systems quite a
>> bit a few years back. At A&M I had a pxe server with a bunch of
>> distro's it, and a custom boot CD for machines outside of the dhcp
>> scope that could still hit the NFS servers.
> Sounds great, thanks for the pointers, we'll look into it.
> I believe we have all the drivers in there, but maybe we missed a critical
> one. It is also possible our initramfs system, or the init script is not correct.
>
> Thanks again, well keep you posted.
> -b.
I never had touch anything in init when I worked with netboot. Out of
curiosity, what are you using for a bootloader? Ive used syslinux, grub
and grub2 for pxebooting.
>> On 07/21/2016 10:55 AM, Bruce Dubbs wrote:
>>> Borries Demeler wrote:
>>>> I am hoping someone on this list can explain how diskless booting works.
>>>>
>>>> I have a bunch of high school kids in my lab this summer and we are
>>>> trying to build a diskless linux cluster as a summer project. I have a
>>>> bunch of generic boxes with quad-core processors, on board video and 8
>>>> GB RAM each. We configured one of the boxes with CentOS 6.8 as a head
>>>> node. It has dhcpd, tftpd, nfs working fine now. We can successfully
>>>> pxe boot various iso images served through the tftp server, the nodes
>>>> get private IP addresses, tftp boot the kernels and ramdisks on the iso
>>>> images, and the nodes can mount nfs shared volumes from the head node.
>>>> All of this works fine.
>>>>
>>>> We now wanted to build our own custom image and wanted to compile a
>>>> minimal kernel and build a minimal ramdisk on which we could nfs mount
>>>> various shares that hold the software and home directories we
>>>> want to have
>>>> accessible on the slave nodes.
>>>>
>>>> But we so far have been unable to build a proper kernel or ramdisk that
>>>> gets past the mounting point where the root ramdisk file system gets
>>>> mounted and cannot figure out where we are going wrong. Any help would
>>>> be appreciated. We tried to follow this guide to the T:
>>>>
>>>> http://eduardo-lago.blogspot.com/2012/06/ram-only-pxe-boot-smallest-diskless.html
>>>>
>>>>
>>>> and searched the internet for all kinds of other documentation
>>>> on the topic.
>>>> But we must be missing something essential, like our init is not getting
>>>> executed or the kernel cannot find the root file system.
>>>>
>>>> Any help for these kids would be greatly appreciated! Can
>>>> someone explain
>>>> how the boot process works exactly step-by-step so we can see where our
>>>> newly built image might be failing? THe boot process always fails at the
>>>> point where the initrd specified ramdisk does not mount root.
>>> Have you made a custom initrd? See if
>>>
>>> http://www.linuxfromscratch.org/blfs/view/stable/postlfs/initramfs.html
>>>
>>> helps. You will, of course, need to modify the init.in file.
>>>
>>> -- Bruce
>>>
>> --
>> _______________________________________________
>> SATLUG mailing list
>> SATLUG@satlug.org
>> http://alamo.satlug.org/mailman/listinfo/satlug to manage/unsubscribe
>> Powered by Rackspace (www.rackspace.com)
From mayfield_mark at gvtc.com Thu Jul 21 13:33:21 2016
From: mayfield_mark at gvtc.com (Mark Mayfield)
Date: Thu Jul 21 13:33:24 2016
Subject: [SATLUG] diskless ramdisk booting
In-Reply-To: <20160721164150.GA7633@biochem.uthscsa.edu>
References: <20160721135334.GA1937@biochem.uthscsa.edu>
<4E7BCC7C-821E-4E61-82E5-E67E37D66F12@rackspace.com>
<20160721164150.GA7633@biochem.uthscsa.edu>
Message-ID: <26ebd8de-361d-cbb1-7bf4-e5a047ec8d7f@gvtc.com>
|
|
On 07/21/2016 11:41 AM, Borries Demeler wrote:
> On Thu, Jul 21, 2016 at 03:55:03PM +0000, Igor Gueths wrote:
>> Hello,
>> based on your initial description of the problem, it sounds as though the kernel is panicking because it cannot find the root filesystem. That being said, at a high level diskless booting works something like this (assuming of course you are in fact PXE booting):
>> ???BIOS passes control to Eprom on the NIC.
>> ???Boot loader on Eprom attempts to get an IP via DHCP, paying special attention to the TFTP-related DHCP options to determine where its TFTP server is.
>> ???Kernel is downloaded from TFTP server, decompressed and executed.
> Yes, that's exactly how far we get with the pxe boot.
>
>> I unfortunately do not recall if one has to hardcode NFS mount points and such into the kernel image, or if as part of passing control to the kernel a separate config file gets downloaded from which further boot options are passed to the kernel. That being said, I would check to see what sort of message (s) you are getting at boot time when the failure occurs and go from there. Hope this helps, and good luck.
When I did it with nfsroot in the kernel, the root argument was passed
from the bootloader
|
grub 1 example
title pxebootclient
kernel path/from/tftp/to/distboot/vmlinuz root=/dev/nfs
nfsroot=192.168.1.2:/nfsshare ip=dhcp rw
initrd path/from/tftp/to/distboot/initrd.img
|
> Our first try is to boot a small self-contained initramfs we built by hand that has a
> init script as well as busybox built into it. The nsf mount points can be put into /etc/fstab
> on the ram image and can be loaded as long as the kernel supports nfs. When we boot an
> image from, say CentOS installation media, this part works perfectly well, and we can
> access nfs mounted dirs just fine, but our own initramfs doesn't seem to work, or
> maybe it is the init.
>
> -b.
>
>
>>> On Jul 21, 2016, at 8:53 AM, Borries Demeler wrote:
>>>
>>> I am hoping someone on this list can explain how diskless booting works.
>>>
>>> I have a bunch of high school kids in my lab this summer and we are
>>> trying to build a diskless linux cluster as a summer project. I have a
>>> bunch of generic boxes with quad-core processors, on board video and 8
>>> GB RAM each. We configured one of the boxes with CentOS 6.8 as a head
>>> node. It has dhcpd, tftpd, nfs working fine now. We can successfully
>>> pxe boot various iso images served through the tftp server, the nodes
>>> get private IP addresses, tftp boot the kernels and ramdisks on the iso
>>> images, and the nodes can mount nfs shared volumes from the head node.
>>> All of this works fine.
>>>
>>> We now wanted to build our own custom image and wanted to compile a
>>> minimal kernel and build a minimal ramdisk on which we could nfs mount
>>> various shares that hold the software and home directories we want to have
>>> accessible on the slave nodes.
>>>
>>> But we so far have been unable to build a proper kernel or ramdisk that
>>> gets past the mounting point where the root ramdisk file system gets
>>> mounted and cannot figure out where we are going wrong. Any help would
>>> be appreciated. We tried to follow this guide to the T:
>>>
>>> http://eduardo-lago.blogspot.com/2012/06/ram-only-pxe-boot-smallest-diskless.html
>>>
>>> and searched the internet for all kinds of other documentation on the topic.
>>> But we must be missing something essential, like our init is not getting
>>> executed or the kernel cannot find the root file system.
>>>
>>> Any help for these kids would be greatly appreciated! Can someone explain
>>> how the boot process works exactly step-by-step so we can see where our
>>> newly built image might be failing? THe boot process always fails at the
>>> point where the initrd specified ramdisk does not mount root.
>>>
>>> Thanks, -Borries
>>> --
>>> _______________________________________________
>>> SATLUG mailing list
>>> SATLUG@satlug.org
>>> http://alamo.satlug.org/mailman/listinfo/satlug to manage/unsubscribe
>>> Powered by Rackspace (www.rackspace.com)
>> --
>> _______________________________________________
>> SATLUG mailing list
>> SATLUG@satlug.org
>> http://alamo.satlug.org/mailman/listinfo/satlug to manage/unsubscribe
>> Powered by Rackspace (www.rackspace.com)
From mayfield_mark at gvtc.com Thu Jul 21 14:00:04 2016
From: mayfield_mark at gvtc.com (Mark Mayfield)
Date: Thu Jul 21 14:00:06 2016
Subject: [SATLUG] diskless ramdisk booting
In-Reply-To: <20160721135334.GA1937@biochem.uthscsa.edu>
References: <20160721135334.GA1937@biochem.uthscsa.edu>
Message-ID: <213a959c-0f0b-65df-a9b8-45b03fae8f1b@gvtc.com>
I didn't catch at first that your cramming you distro into the ramdisk
instead of using an initial ramdisk. I put my whole systems on nfs
shares and just loaded the kernel or kernel+initial ramdisk. The way he
make the ramdisk your network drivers must be in the kernel
I would just go into the kernel drivers section->network cards and mark
all the net drivers with * (in kernel) instead of M (kernel module)
I'd also use lsmod to find the kernel module the systems your using use
and make sure they are options in the kernel. If they can't be compiled
into the kernel you'll have to make sure they can be loaded from the
ramdisk.
On 07/21/2016 08:53 AM, Borries Demeler wrote:
> I am hoping someone on this list can explain how diskless booting works.
>
> I have a bunch of high school kids in my lab this summer and we are
> trying to build a diskless linux cluster as a summer project. I have a
> bunch of generic boxes with quad-core processors, on board video and 8
> GB RAM each. We configured one of the boxes with CentOS 6.8 as a head
> node. It has dhcpd, tftpd, nfs working fine now. We can successfully
> pxe boot various iso images served through the tftp server, the nodes
> get private IP addresses, tftp boot the kernels and ramdisks on the iso
> images, and the nodes can mount nfs shared volumes from the head node.
> All of this works fine.
>
> We now wanted to build our own custom image and wanted to compile a
> minimal kernel and build a minimal ramdisk on which we could nfs mount
> various shares that hold the software and home directories we want to have
> accessible on the slave nodes.
>
> But we so far have been unable to build a proper kernel or ramdisk that
> gets past the mounting point where the root ramdisk file system gets
> mounted and cannot figure out where we are going wrong. Any help would
> be appreciated. We tried to follow this guide to the T:
>
> http://eduardo-lago.blogspot.com/2012/06/ram-only-pxe-boot-smallest-diskless.html
>
> and searched the internet for all kinds of other documentation on the topic.
> But we must be missing something essential, like our init is not getting
> executed or the kernel cannot find the root file system.
>
> Any help for these kids would be greatly appreciated! Can someone explain
> how the boot process works exactly step-by-step so we can see where our
> newly built image might be failing? THe boot process always fails at the
> point where the initrd specified ramdisk does not mount root.
>
> Thanks, -Borries
From sargonemail at gmail.com Wed Jul 27 11:00:20 2016
From: sargonemail at gmail.com (Craig)
Date: Wed Jul 27 11:00:26 2016
Subject: [SATLUG] diskless ramdisk booting
Message-ID:
...
> But we so far have been unable to build a proper kernel or ramdisk that
> gets past the mounting point where the root ramdisk file system gets
> mounted and cannot figure out where we are going wrong. Any help would
> be appreciated. We tried to follow this guide to the T:
>> http://eduardo-lago.blogspot.com/2012/06/ram-only-pxe-boot-smallest-diskless.html
>
> and searched the internet for all kinds of other documentation on the topic.
> But we must be missing something essential, like our init is not getting
> executed or the kernel cannot find the root file system.
...
Kinda sounds like an issue related to switching from initial boot
program area to final OS run area (e.g. root
file system area for "nic card" & switch to root file system area for
"RAMdisk" once everything has been setup in ram)
Since there are several ways under linux to do the swtich, might want
to look in the boot scripts
for commands/programs discussed in
http://tiebing.blogspot.com/2014/02/linux-switchroot-vs-pivotroot-vs-chroot.html
Craig
From demeler at biochem.uthscsa.edu Wed Jul 27 11:21:15 2016
From: demeler at biochem.uthscsa.edu (Borries Demeler)
Date: Wed Jul 27 11:21:17 2016
Subject: [SATLUG] diskless ramdisk booting
In-Reply-To:
References:
Message-ID: <20160727162115.GA19018@biochem.uthscsa.edu>
We managed to boot from nfsroot, but we really need a RAM disk for /var and /etc
for each node to be different.
Yes, you are correct, the init switching is what we have not yet figured out :-)
-b.
On Wed, Jul 27, 2016 at 11:00:20AM -0500, Craig wrote:
> ...
> > But we so far have been unable to build a proper kernel or ramdisk that
> > gets past the mounting point where the root ramdisk file system gets
> > mounted and cannot figure out where we are going wrong. Any help would
> > be appreciated. We tried to follow this guide to the T:
> >> http://eduardo-lago.blogspot.com/2012/06/ram-only-pxe-boot-smallest-diskless.html
> >
> > and searched the internet for all kinds of other documentation on the topic.
> > But we must be missing something essential, like our init is not getting
> > executed or the kernel cannot find the root file system.
> ...
>
> Kinda sounds like an issue related to switching from initial boot
> program area to final OS run area (e.g. root
> file system area for "nic card" & switch to root file system area for
> "RAMdisk" once everything has been setup in ram)
> Since there are several ways under linux to do the swtich, might want
> to look in the boot scripts
> for commands/programs discussed in
> http://tiebing.blogspot.com/2014/02/linux-switchroot-vs-pivotroot-vs-chroot.html
>
> Craig
> --
> _______________________________________________
> SATLUG mailing list
> SATLUG@satlug.org
> http://alamo.satlug.org/mailman/listinfo/satlug to manage/unsubscribe
> Powered by Rackspace (www.rackspace.com)
From bruce.dubbs at gmail.com Wed Jul 27 13:39:22 2016
From: bruce.dubbs at gmail.com (Bruce Dubbs)
Date: Wed Jul 27 13:39:27 2016
Subject: [SATLUG] diskless ramdisk booting
In-Reply-To: <20160727162115.GA19018@biochem.uthscsa.edu>
References:
<20160727162115.GA19018@biochem.uthscsa.edu>
Message-ID: <5798FFDA.4070206@gmail.com>
Borries Demeler wrote:
>
> We managed to boot from nfsroot, but we really need a RAM disk for /var and /etc
> for each node to be different.
I've never heard of /etc being different for each node. For that you need
a different root partition for each node or some very fancy changes in the
initramfs.
> Yes, you are correct, the init switching is what we have not yet figured out :-)
So you got the root partition mounted? Switching to the system from
initramfs is done by the switch_root program from the util-linux package:
ftp://ftp.kernel.org/pub/linux/utils/util-linux/
-- Bruce
From sargonemail at gmail.com Wed Jul 27 15:53:21 2016
From: sargonemail at gmail.com (Craig)
Date: Wed Jul 27 15:53:24 2016
Subject: [SATLUG] diskless ramdisk booting
In-Reply-To:
References:
Message-ID:
*> Borries Demeler, d*emeler at biochem.uthscsa.edu on
*Wed Jul 27 11:21:15 CDT 2016> wrote:*> We managed to boot from
nfsroot, but we really need a RAM disk for /var and /etc
> for each node to be different.
>
> Yes, you are correct, the init switching is what we have not yet figured out :-)
....
Ummm... ok, based on this article & initial article, sounds like some
terminology is getting
in the way. So going to sum up what I think is being attempted.
The remote node initiates a pxe boot off of the nfs server. The nfs
server setsup an OS instance
on the server side (per remote node request) which is remotely
accessed by the remote node (not
certain if the user home directory resides on nfs server, some other
server, or local machine).
The remote node mini-OS can be obtained either via pxe or local
HD/floppy/usb/cd/dvd from pxe server.
Mini-OS is run entirely in RAM.
The mini-OS running in RAM is used to (gpxe, grub2, uboot/netboot):
?configured to access a complete distribution running on a different box,
the traditional terminal server, but with access to a remote VM instance.
?configured to access specific app(s) running remotely on a different box
where apps are run as a container/docker instance on the server side.
?contains the environment to run a specific app or apps on current box,
where apps are run as a container/docker instance locally.
?setup the complete distribution to run in RAM or install to box,
where live image is pulled from server
e.g. netboot.xyz but with the nfs server as the permenant storage
point for the user
home directory
?? other approach(es) not related to traditional terminal service(s)
Craig
From sargonemail at gmail.com Wed Jul 27 16:02:35 2016
From: sargonemail at gmail.com (Craig)
Date: Wed Jul 27 16:02:38 2016
Subject: [SATLUG] diskless ramdisk booting
In-Reply-To:
References:
Message-ID:
Methods for creating a mount point for /var and /tmp in RAM :
http://www.satlug.org/pipermail/satlug/2012-July/066579.html
Craig
On Wed, Jul 27, 2016 at 3:53 PM, Craig wrote:
> *> Borries Demeler, d*emeler at biochem.uthscsa.edu on
>
> *Wed Jul 27 11:21:15 CDT 2016> wrote:*> We managed to boot from nfsroot, but we really need a RAM disk for /var and /etc
> > for each node to be different.
> >
> > Yes, you are correct, the init switching is what we have not yet figured out :-)
> ....
>
> Ummm... ok, based on this article & initial article, sounds like some terminology is getting
> in the way. So going to sum up what I think is being attempted.
>
> The remote node initiates a pxe boot off of the nfs server. The nfs server setsup an OS instance
> on the server side (per remote node request) which is remotely accessed by the remote node (not
> certain if the user home directory resides on nfs server, some other server, or local machine).
> The remote node mini-OS can be obtained either via pxe or local HD/floppy/usb/cd/dvd from pxe server.
> Mini-OS is run entirely in RAM.
>
> The mini-OS running in RAM is used to (gpxe, grub2, uboot/netboot):
> ?configured to access a complete distribution running on a different box,
> the traditional terminal server, but with access to a remote VM instance.
>
> ?configured to access specific app(s) running remotely on a different box
> where apps are run as a container/docker instance on the server side.
>
> ?contains the environment to run a specific app or apps on current box,
> where apps are run as a container/docker instance locally.
>
> ?setup the complete distribution to run in RAM or install to box,
> where live image is pulled from server
> e.g. netboot.xyz but with the nfs server as the permenant storage point for the user
> home directory
>
> ?? other approach(es) not related to traditional terminal service(s)
>
> Craig
>
>
From mayfield_mark at gvtc.com Wed Jul 27 22:42:36 2016
From: mayfield_mark at gvtc.com (Mark Mayfield)
Date: Wed Jul 27 22:42:39 2016
Subject: [SATLUG] diskless ramdisk booting
In-Reply-To:
References:
Message-ID:
On 07/27/2016 04:02 PM, Craig wrote:
> Methods for creating a mount point for /var and /tmp in RAM :
> http://www.satlug.org/pipermail/satlug/2012-July/066579.html
>
> Craig
I've used tmpfs for /var and /tmp, as Bruce said having a different /etc
for different workstations is not very feasible. It also shouldn't
usually be necessary. The machines all having the same hostname could be
a problem, but you can set static hosts in dhcp by mac address so they
all get different host names.
Even cramming /etc in a ramdisk, all hosts still get the same /etc.
Unless you give each host a different ram disk based off mac address.
>
> On Wed, Jul 27, 2016 at 3:53 PM, Craig wrote:
>
>> *> Borries Demeler, d*emeler at biochem.uthscsa.edu on
>>
>> *Wed Jul 27 11:21:15 CDT 2016> wrote:*> We managed to boot from nfsroot, but we really need a RAM disk for /var and /etc
>>> for each node to be different.
>>>
>>> Yes, you are correct, the init switching is what we have not yet figured out :-)
>> ....
>>
>> Ummm... ok, based on this article & initial article, sounds like some terminology is getting
>> in the way. So going to sum up what I think is being attempted.
>>
>> The remote node initiates a pxe boot off of the nfs server. The nfs server setsup an OS instance
>> on the server side (per remote node request) which is remotely accessed by the remote node (not
>> certain if the user home directory resides on nfs server, some other server, or local machine).
>> The remote node mini-OS can be obtained either via pxe or local HD/floppy/usb/cd/dvd from pxe server.
>> Mini-OS is run entirely in RAM.
>>
>> The mini-OS running in RAM is used to (gpxe, grub2, uboot/netboot):
>> ?configured to access a complete distribution running on a different box,
>> the traditional terminal server, but with access to a remote VM instance.
>>
>> ?configured to access specific app(s) running remotely on a different box
>> where apps are run as a container/docker instance on the server side.
>>
>> ?contains the environment to run a specific app or apps on current box,
>> where apps are run as a container/docker instance locally.
>>
>> ?setup the complete distribution to run in RAM or install to box,
>> where live image is pulled from server
>> e.g. netboot.xyz but with the nfs server as the permenant storage point for the user
>> home directory
>>
>> ?? other approach(es) not related to traditional terminal service(s)
>>
>> Craig
>>
>>
From sargonemail at gmail.com Thu Jul 28 21:21:00 2016
From: sargonemail at gmail.com (Craig)
Date: Thu Jul 28 21:21:03 2016
Subject: [SATLUG] diskless ramdisk booting
In-Reply-To:
References:
Message-ID:
One approach to avoiding the pivot root issue :
http://www.cs.vu.nl/~herbertb/misc/writingkernels.txt
On Wed, Jul 27, 2016 at 4:02 PM, Craig wrote:
> Methods for creating a mount point for /var and /tmp in RAM :
> http://www.satlug.org/pipermail/satlug/2012-July/066579.html
>
> Craig
>
> On Wed, Jul 27, 2016 at 3:53 PM, Craig wrote:
>
>> *> Borries Demeler, d*emeler at biochem.uthscsa.edu on
>>
>> *Wed Jul 27 11:21:15 CDT 2016> wrote:*> We managed to boot from nfsroot, but we really need a RAM disk for /var and /etc
>> > for each node to be different.
>> >
>> > Yes, you are correct, the init switching is what we have not yet figured out :-)
>> ....
>>
>> Ummm... ok, based on this article & initial article, sounds like some terminology is getting
>> in the way. So going to sum up what I think is being attempted.
>>
>> The remote node initiates a pxe boot off of the nfs server. The nfs server setsup an OS instance
>> on the server side (per remote node request) which is remotely accessed by the remote node (not
>> certain if the user home directory resides on nfs server, some other server, or local machine).
>> The remote node mini-OS can be obtained either via pxe or local HD/floppy/usb/cd/dvd from pxe server.
>> Mini-OS is run entirely in RAM.
>>
>> The mini-OS running in RAM is used to (gpxe, grub2, uboot/netboot):
>> ?configured to access a complete distribution running on a different box,
>> the traditional terminal server, but with access to a remote VM instance.
>>
>> ?configured to access specific app(s) running remotely on a different box
>> where apps are run as a container/docker instance on the server side.
>>
>> ?contains the environment to run a specific app or apps on current box,
>> where apps are run as a container/docker instance locally.
>>
>> ?setup the complete distribution to run in RAM or install to box,
>> where live image is pulled from server
>> e.g. netboot.xyz but with the nfs server as the permenant storage point for the user
>> home directory
>>
>> ?? other approach(es) not related to traditional terminal service(s)
>>
>> Craig
>>
>>
>
From bruce.dubbs at gmail.com Fri Jul 29 01:23:24 2016
From: bruce.dubbs at gmail.com (Bruce Dubbs)
Date: Fri Jul 29 01:23:31 2016
Subject: [SATLUG] diskless ramdisk booting
In-Reply-To:
References:
Message-ID: <579AF65C.3040206@gmail.com>
Craig wrote:
> One approach to avoiding the pivot root issue :
> http://www.cs.vu.nl/~herbertb/misc/writingkernels.txt
Craig, that file is from 2009 and uses Grub Legacy and is talking about
creating some sort of non-Linux kernel. A standard Linux kernel build is
bootable by default. Adding Grub2 is a lot easier as you only need to do
'grub-install '
-- Bruce