perhaps my wording was not clear enough..'and have a not save option' might be more clear.

In other words it truly loads all to ram including saves (sfs part only if enough ram) ..... and there is no persistent save file on a hard drive/usb. It inherently, and as part of the system has a no save option. My point was usb DOES have a save file which is mounted in use...just has the extra tmpfs for session changes which then gets merged. Hence the inability to remove the usb flash stick. The impression was given that puppy loads to ram saves and all regardless of mode...which is not the case EXCEPT for multisession. PUPMODE=13 was designed reduce save file writes rather than be independent of one.

Another thing mentioned was decompressing of sfs.... not so...only the required data read is decompressed on the fly when needed but the sfs remains the same size sat in another tmpfs.

Thing is if talking about the system and optimising then it needs to be clear what the system is including my wording of it
Its important to know what goes where....

There is a nice set of diagrams of the various modes somewhere.... a picture definitely paints a thousand words.

As an aside the sfs save method I made is based on multisession's total ram approach but usable on flash and hard drives. In other words achieve what all these hacks with PUPMODE=13 are trying to do...ie be free of ANY drives once booted and discard all at the end if desired.

Just mentioned it because I run pupmode 5 all of the time and modified the shutdown code to skip asking whether to create a savefile or not (defaults to not) i.e. I run with no savefile and just remaster any changes I want preserved across reboots (increasingly infrequently). I now have all of the sfs's that I use regularly saved inside the puppy sfs (and I drop puppy sfs into initrd), so initrd holds everything and is loaded into ram at bootup. I've dropped using a swap partition as well, so nothing touches the disk at all - except a remaster if I use the HDD as a temp storage area (or if I intentionally mount a drive).

Can't recall the exact detail without looking, but seem to remember seeing/reading a suggestion of switching from pupmode 5 to pupmode 12 I think it was - as part of shutdown to equally not save. (I think it was because that would be assumed to already have been saved - or something like that ??? Or could have been pupmode 13 ???).

wouldn't it be nice to have saved configs and data and changes all in ram with option to not save whenever you want and all done automatically and with no need for a symlinks or other such methods ...

oh it is....

yes mode 12 would do nothing at shutdown.

PUPMODE=13 should have been dropped / replaced many moons ago.

Mike

I think you've either (a) mis-spoke, reversing the roles of PUPMODE's 12 and 13; or (b) intending to continue the sarcasm of the first part of the quoted text, may have left the reader confused.

If you are serious that some PUPMODE should have been dropped, than it should be 12; and I'd agree with you.

The attached screenshot is from Lupu 5.28, playdaz's "original", now about 4 years old. Its a frugal install to a hard drive. I don't know if older Pups can be configured using jpeps boot parameter modification, [pmedia=ataflash] or whether their SaveConfiguration can be set to ONLY "ASK AT SHUTDOWN."

On my setup a Save Icon appears on the desktop [to enable manual Saves]; and at Shutdown a dialog appears offering the choice to Save or No Save. I'll discuss the effects of No Save below.

mikeb wrote:

Quote:

Pups operate entirely in RAM.

This is only partially true...indeed addition sfs are not necessarily loaded to ram at all.

Correct, Also discussed further below.

Quote:

For usb the current session is in ram but the save file is not... the ram session is saved at shutdown and periodically... seems like this is what you are talking about and in this case you can opt to not save iff you add an addon script.

Almost correct. IIRC, your Pup of choice is a highly modified version of one of, I think, the 4 Series; Lupu being among the 5 Series. I don't have a working Wary, Racy or Saluki to test. The only "script" I'm certain of is jpep's modification of the boot parameter. However, shinobar has been very active in modifying PupSaveConfig, http://www.murga-linux.com/puppy/viewtopic.php?p=457081#457081. His recent modification, standard in Carolina and Carolina-Vanguard, includes the choice to Never Save with even the dialog at Shutdown being absent.

Quote:

For hard drives and other fast media there is no ram for changes...the save file is used exclusively so any changes are instantly added.

That's the default setting you get on a frugal install when Grub4dos writes the Menu.lst. jpep's parameter modification and the user's configuration of SaveSession can change that.

The following has been my experience with Lupu and several of this year's litter of pups, frugally installed to hard drives, boot parameter changed to pmedia-ataflash, and SaveSession set to 0:
Change Wallpaper, don't save, old wallpaper appears on reboot.
Load SFS-on-the-fly, don't save, SFS isn't loaded on reboot.
Install pet, restart X if necessary, run pet, don't save, pet not present on reboot.

I admit being rather confused regarding how Puppies handle SFSes. From experience when an SFS is loaded, but not as yet opened, some portion of it exists in RAM. Usually there's a listing on the Menu. If, in fact, it has been loaded, even if there is no listing on the Menu, typing the name of its executable or wrapper in a terminal will either open the application or produce an error message other than "command not found."

Some time ago, in attempting to compare resource usage of installed pets, SFSes and Program Folders --applications decompress outside the SaveFile linked to the OS via scripts-- I used the then current LibreOffice as the test subject. As an SFS (with, I think gz compression) it required about 175 Mb of hard drive. As a decompressed Program Folder it took, IIRC, about 500 Mbs of hard drive. As an installed pet, it required the SaveFile use 500 Mbs of hard drive in addition to anything else in the SaveFile. As a loaded, but not opened, SFS it required about 35 Mb of RAM.

The "Urban Dictionary" gives other definitions, but the one I first learned for "blivit" was "ten pounds of shit in a 5 pound bag."

So here's my problem with how Puppies handle SFSes. I don't have a Swap File. And my computer has 4 Gbs of RAM --3.5 recognized running 32-bit OSes. So I can't explore the problem. What happens when you only have, say, 512 Mbs of RAM, your OS is using, say, 100 Mbs of that, and you attempt to open an application which requires 500 Mb? System crash? Application doesn't open? Something else?

The more important question is this: When an SFS is loaded and opened, the files you can observe if you had decompressed the SFS appear when you browse to their location. Or to be clearer, if you decompressed an SFS you can browse thru its files and folders and observe and delete or modify say its executable at /usr/local/bin. if, instead, you load the SFS and browse to /usr/local/bin you'll also see that SFS's executable. But what you are seeing is then only be in RAM. What happens if you delete that executable and --being in PUPMODE 13 configured not to automatically save-- shutdown without saving? My guess would be that the "deletion" would not have been written. That on reboot, the SFS will be decompressed from its "pristine" compressed file; still have an executable; and still be functional. But that's a guess.

"I'll be back" after testing.

mikesLrLast edited by mikeslr on Mon 09 Feb 2015, 19:09; edited 1 time in total

I've messed around with 'loading' using sym links i.e. mount the sfs somewhere and then

cd /
cp -rs /mnt/somewhere/* .

.. and you end up with thousands of sym links under / and sub folders/files into the sfs

Really quick way to get a sfs up and available asap.

I've since found a way to mount sfs more directly into aufs and now use that. i.e. more correctly layers in the sfs - but doing so in the blink of a eye (thanks to a large part due to MikeB's guidance/pointers).

Not sure whether this is strictly correct/ok, but it works ok for me :

Which loads in around 1.5GB of non compressed, 400MB of compressed (SFS's total file size) of extra apps/stuff in around a second or two (I have a fast version of fixmenus that someone kindly pointed me to but can't recall the exact details off-hand).

Note that code is specific to PUPMODE 5 i.e. I boot pupmode 5 all of the time and have disabled the shutdown save menu (run with no save file ever). In that mode pup layer is always set to be /pup_ro2 - and I have all of the sfs's that I load stored in /OFFICE on the ramdisk (so accessed in the above code as /initrd/pup_ro2/OFFICE/<sfs>

Posted: Tue 10 Feb 2015, 12:57 Post subject:
The Invulnerability? of an SFS on a HarddriveSubject description: On a Pup without an Automatic Save to SaveFile/Folder

Yesterday, having posed a question at the end of this post, http://murga-linux.com/puppy/viewtopic.php?p=827188#827188, I spent an unreasonable amount of time testing what effect, if any, operating without an automatic save making changes to the “copy of an SFS loaded into RAM” might have upon the SFS on the storage media that had been selected for loading.

Prior to these tests, based on prior experience, I had guessed that the SFS on the storage media was invulnerable. There's a difference, however, between a conclusion based on general prior experience, and an actual test. For a couple thousand years our observations lead us to conclude that the Sun revolved around the Earth. It is only within the last century that we've had evidence that the opposite was, in “fact”, true.*

I took detailed notes as I ran those tests. If someone is interested, I'll email a copy of those notes. If several people are interested, I'll post them. My test SFSes were Calibre and Libreoffice. My test Pup was TahrPup 6, both in its condition with a SaveFolder I've built up over about a month, booted pfix=ram, and after creating a new SaveFolder. In all instances, I used the boot parameter “pmedia=ataflash” to put the frugal install of Tahpup on a harddrive into PUPMODE 13; and, as soon as possible, initiated Menu>System>Puppy Event Manager, clicked the Save Session Tab, and configured Save Session to never automatically save but to ask at shutdown.

Changes I made to the SFSes loaded on the fly included deleting its config file, changing the icon identified in its desktop file and even deleting its executable.

The conclusion I reached is that operating under PUPMODE 13 with PupSaveConfig configured not to perform automatic Saves, if a manual Save is not executed, changes made to a loaded SFS are only to its condition existing in RAM and are neither preserved on a reboot nor written to the SFS file on the harddrive. If a Save is executed, changes are only to files and folders in the SaveFile/Folder, which “over-ride” any corresponding files in the SFS when loaded: not to the files and folders in the SFS on the hard drive, itself.

After practicing Law for 35 years, and drafting innumerable contracts, it would be inconsistent to my nature not make definitive statements, without “wiggle room.” Therefore, I've included the following which, of course, is more extensive than the above “conclusion”.

Caution 1: My computer has 4 Gbs of RAM, 3.5 of which are recognized even when operating under 32-bit systems. Having no apparent need for one, it's been many years since I devoted hard drive space to a Swapfile/Folder/Partition. I have little knowledge of how a Linux system uses one, other than that in some instances hard drive space is being used as “substitute/additional” RAM. I don't believe data consigned to the “Swapfile” is preserved between boots, but that's a guess. If wrong, I don't know under what circumstances it would.
Caution 2: I don't know how a Puppy handles more data than it has RAM available to contain. I have been advised that where a computer's RAM is only, say 512 Mb, and there is no Swapfile, and the OS, opened applications, and opened data files exceed 512 Mb, it does not “cache” the excess by writing to the SaveFile. IIRC, some Pups are configured to only use 256 Mbs of RAM for the OS and opened applications, leaving sufficient RAM to swap data in and out. But I don't know. Nor do I know whether there is a routine to first measure the available RAM and Swapfile space before imposing that restriction. Nor am I going to physically remove memory cards to find out.
Caution 3: I use several applications as “Program Folders”. That is, they are decompressed to my hard drive and linked to my OS via symlinks. Any change I make while running those applications are written immediately to the external files. On the next bootup, even if I shutdown without Saving, those changes will be reflected.
Caution 4: Several of the “Portable Applications” shinobar and the Japanese Team have created utilize SFSes, but don't have them load via bootmanager or SFS-Load-on the fly. Rather, they are built as RoxApps, somehow linking to the OS. RSH's Lazy-Puppy and Lassie utilize the same or similar technique. It is possible that changes to such applications are effective immediately; or if not to the entire application, at least to their “external-to-the-OS” configuration files.
Caution 5: When remastering a Pup, you have the opportunity to include in your remaster both any loaded SFSes and the files reflecting your Pup's present state/configuration. Either or both may effect the condition of the now “builtin” application.
Caution 6: Although the files within an SFS on storage media can not be modified by making changes to its copy loaded into RAM, the SFS on storage media can be deleted. And, of course, there are applications for mounting and modifying SFSes on storage media.
Caution 7: There is a difference between SaveFiles and SaveFolders. SaveFiles are compressed. SaveFolders aren't. Just as you can write data to files in folders on any mounted partition and such “changes” will be preserved when running under Pupmode 13 even if you choose not to Save, so also can you or malware running on your computer write to the files contained in your SaveFolder and those “changes” will be preserved when running under Pupmode 13 even if you choose not to Save.
Caution 8: We know what we know. We know what we don't know. But we also don't know what we don't know we don't know.

mikeslr

* I know most of you think that this “fact” was established some 500 years ago by Copernicus. Copernicus's “proof” was, however, only a mathematical model which provided a better fit to observations, made from Earth, than its predecessor. Mathematics is a language: one which evolves as our need to describe our observations becomes greater, or where a mathematician's flight of fancy takes it. Sometimes, utilizing such “flight of fancy” our “conclusions” are erroneous. James Clerk Maxwell, one of the greatest mathematicians of all time, argued that light was a wave traveling through aether. Subsequently, Einstein's mathematics “proved” that there was no aether, leading to the currently prevailing belief that “Space is Empty”. Unfortunately, that belief is also in error. It ignores one of Einstein's other conclusions: that mass curves space. How do you curve Nothing? Add that there is evidence that constantly popping in and out of existence in such “Empty Space” are “virtual” particles-- virtual only because they don't last very long. Add that light can be either a particle or a wave depending on what we choose to test for suggests that we're really making things up as we go along.
Only with the development of Space Flight have we been able to make observations which were not from Earth. But a true skeptic might doubt the validity of even those observations. Conspiracy theories aside, observations made other than from Earth still have to be transmitted to Earth for analysis and integration into our conception of Everything. I find it peculiar that we are roughly 93 million miles from the Sun –the body which warps space the most of any body “close” to us-- and that our measurement from here of the speed of light is roughly 93 x 2 thousand miles per second. Like the game of “Telephone”. we have no idea how much the content may have warped between its original and its received.

But lest we fall into utter despair, it is best to apply the advice given by the Judge to the Prisoner being sentenced for two homocides:
Judge: I sentence you to two life terms, to be served consecutively.
Prison: But, Judge, I won't live that long.
Judge: Well, do the best you can.

I don't believe data consigned to the “Swapfile” is preserved between boots, but that's a guess. If wrong, I don't know under what circumstances it would

It's not in the operational sense, but does persist and can be read via forensics. A disposed of PC/HDD can be inspected and potentially confidential data read from swapfiles (pagefile under Windows).

Quote:

I don't know how a Puppy handles more data than it has RAM available to contain

A loaded sfs is a squashed (compressed) file system. My largest is around 1.5GB uncompressed, 450MB compressed - which includes Libre Office, Audacity, Inkscape, Blender ...etc. When puppy accesses that data it only uncompresses what it needs at the time i.e. part of the whole compressed sfs, so for example if Libre is being read/loaded it will pull out just that part of the compressed data and load that into main memory. A bit like having a zip file of many files and where you extract just one or a few of those files. Part of the operation is that fixed sized pages are loaded into/out of memory - potentially swapped in/out of the swap file as/when needed. If there's no swap file then its loaded from disk (where the file normally resides), which is much the same thing however disk transfers tend to be variable sized files whereas from swap its fixed sized blocks (and a block transfer can be a single CPU instruction i.e. fast). So even though my PC only has 1.5GB of ram, and my biggest sfs uncompressed size is also 1.5GB, it only takes up the 450MB compressed size, leaving over 1GB of ram still free, part of which is then used to uncompress parts of the sfs into memory as and when required (and delete out of memory other parts that no longer needs to reside in ram at the time (it can always go back and re-get the data again if needed later)).

SFS's are read only and as such puppy sets up pointers when the sfs and its contents are loaded so it knows that the data/programs exists and whereabouts they're located. Those pointers are called inodes. Part of the loading often involves merging/joining two or more sources to a single point. For instance the main puppy will have a /usr directory as may a sfs that's being loaded, so the two are merged together (union file system (unionfs)) and considered as being one. There is another choice to unionfs that's called another union file system (aufs). When you set up unions you can define which of the two is read only and which is read/write so that any changes are recorded on the correct medium.

A non full install puppy in effect maintains a difference list. Any changes are recorded in /initrd/pup_rw, so if you add a file its added there. If deleted again its removed from there. If a permanent file is deleted then it creates a .whiteout reference to indicate that the file has been deleted. The layering of puppy is such that it can determine the status of each file i.e. if its in pup_rw then that is considered the source to be read, otherwise it looks at the next level (pup read only level).

Posted: Tue 10 Feb 2015, 14:24 Post subject:
Re: The Invulnerability? of an SFS on a HarddriveSubject description: On a Pup without an Automatic Save to SaveFile/Folder

mikeslr wrote:

Maxwell, one of the greatest mathematicians of all time, argued that light was a wave traveling through aether. Subsequently, Einstein's mathematics “proved” that there was no aether, leading to the currently prevailing belief that “Space is Empty”. Unfortunately, that belief is also in error. It ignores one of Einstein's other conclusions: that mass curves space. How do you curve Nothing? Add that there is evidence that constantly popping in and out of existence in such “Empty Space” are “virtual” particles-- virtual only because they don't last very long. Add that light can be either a particle or a wave depending on what we choose to test for suggests that we're really making things up as we go along.

Proof that nothing doesn't exist (Thank God).

Mass is created out of change, the common denominator is energy and changes in energy levels. But we tend to focus studies on matter/mass, a product of energy changes. A light particle having broken free of the energy level changes that produced it is self sustaining through its interaction of electrical and magnetic energy levels. As the electrical level peaks and magnetic level troughs so it pulls itself back towards a reversal in those energy levels until magnetic peaks/electrical troughs. Somewhere during those repeated transitions mass appears out of nowhere and then disappears again. Studied from a mass (particle physics) focused aspect it looks confusing with mass/particles appearing and disappearing and transitioning through two-slit tests etc (seen as both a wave and a particle).

Pointless looking at how to create something out of nothing when nothing doesn't exist.

Grab the Web-Server directory out of /root and drag that to outside of puppy, such as to /mnt/home so that its in effect 'deleted' out of /root. Look in the /initrd/pup_rw rox window and click the eye icon so you see hidden files and there's a whiteout file .wh.Web-Server that indicates that the Web-Server folder was 'deleted'. Move (drag/drop) the Web-Server folder back in again into /root and the /initrd/pup_rw whiteout file disappears (to indicate that the folder hasn't been deleted).

You can play around in a similar manner with other tests, such as create a file in /root and see how it also appears in /initrd/pup_rw. For new/added files a copy is also created in /initrd/pup_rw and if later that new file is deleted its also just deleted in /initrd/pup_rw (no need for whiteout files).

Mikeb guided me towards how you can create a savefile using the pup_rw layer. Copy that at shutdown, reinstate the copy at bootup and you have persistence of changes. Not all of the content however needs to be preserved, i.e. ignore /tmp ... and others. If you make a copy of /initrd/pup_rw using mksquashfs with no compression it copies (and restores) really quickly even if many changes have been made. Periodic remastering can enable the changes to be more permanently installed and the 'savefile' (sfs made using mksquashfs of the /initrd/pup_rw folder) reset to a zero size again.

I did use that approach for a while, but then changed things less and less as I got closer to the puppy I preferred so I resorted to not bothering and just remastering as/when I wanted to install a permanent change i.e. I just ram boot pupmode 5 each/every time and don't save (save option eliminated) at shutdown. I store all data outside of puppy space, use either a freshly downloaded new version of firefox, or a portable firefox on HDD, and use a online email account - so the core puppy remains the exact same from one reboot to another (read only so any virus can only persist for a single session). The main benefit for me is that I can play around, potentially totally screw up the puppy session/files - and then just reboot to be back at a pristine version of puppy again.

You cannot post new topics in this forumYou cannot reply to topics in this forumYou cannot edit your posts in this forumYou cannot delete your posts in this forumYou cannot vote in polls in this forumYou cannot attach files in this forumYou can download files in this forum