The Script and RPI 3 Update

Having started putting together “the script” a long time ago, long before the Raspberry Pi 3 came out – and having developed it to handle a wide range of boards and scenarios including of course the Pi3, I found it interesting to return today to installing “the script” on the Raspberry Pi 3 – and of course I wanted to ensure that nothing had changed recently which might stop others doing the same.

Oh! While I’m on – the ESP8266 code – just updated it – got LOTS of RAM back, added in a message on power up to indicate why the board is starting… and re-thought a power up message that wrote to FLASH (and hence could have worn the FLASH out after long periods of using deep-sleep restarts. All updated and OTA available.

Right – where was I… yes… Raspberry Pi 3

From a convenience perspective, most of the small boards I’ve dealt with lately have SSH enabled at start as well as root access. The Pi3 setup (Raspbian) currently has neither. So in order to install “the script” easily, I found myself having to plug in a monitor and keyboard and use the desktop to enable I2c and enable SSH. Simple enough.

I then used Mobaxterm (could have used WinSCP) to access the board from my PC – I find that easier than messing around with a small keyboard on the actual PI and swapping screens over etc. At that point I realised that remote ROOT access was disabled. I know there are those who will say this is a security issue but personally I’d be happier with (as happens elsewhere) initial access and THEN an offer on first setup – to disable it unless instructed otherwise.

Anyway, off I went to sudo nano /etc/ssh/sshd_config to place/update an entry “PermitRootLogin yes” then reboot to ensure I could remote in as root and then change the password for both pi and root.

I then found “apt-get update” failing miserably because the guys at Raspberry Pi seem to have decided to use IPV6 without a fall-back position. Adding “ipv6.disable=1” to the first line of /boot/cmdline.txt file seemed to fix that. I took the opportunity to remove the serial debug entry while I was on.

I know, IPV6 etc is long overdue etc. etc. but I’m certainly not about to spend hours adjusting my router and talking to my WIFI provider just for one project that insists on IPV6. Life is short and I simply wanted to install some software and not get side-tracked – no other board I know of currently presents this challenge.

So two reboots later I was on my way to installing “the script” which has operated flawlessly on just about every other setup for some time now.

Now, the script started up just fine (installing as root, only to be quickly told to run again as user Pi – this is deliberate as this is handy for those systems which initially don’t have a PI user which needs to be set up with a range of group access permissions and we want to keep things consistent).

While installing Node-Red I noted a whole boatload of warnings (not errors) to do with the serialport – you would think that on the most popular of boards on the latest operating system setup, this would have been resolved a while ago – not that deprecation warnings do a lot of good except look awful. Unless you want to get into obtaining the source and fixing it (hence turning an evening’s project into a day’s project) for not much apparent benefit, all these warnings seem to serve little purpose but to confuse the beginner.

non addable to script as they're needed before running it, but you can make them before even starting, with a notepad 🙂
just remember to enclose "ssh" in quotes when saving, otherwise notepad will add .txt to the name...
which maybe works anyway, but i never tested this way...

I'm just a new user of your script and I used it the first time today, didn't notice any errors when installing BUT when using it afterwards I didn't have Webmin (It's in the list when you browse to the ip adress of the Raspberry Pi 3, but it didn't work).
I used an image of Raspbian Stretch

Second thing I noticed was when I tried to follow a video of Andreas Spiess (I believe 126), I tried to use his flow.txt(found it on github) and when importing it in node red (I am also new to this things so maybe I do something wrong) Node Red complained that it missed 2 nodes, both with sqlite, I then removed the 'node-red-node-sqlite' node via manage palette and re-installed it, this gave me a new node in the sidebar called 'sqlite' (I attached a screenshot). I then used node red for several hours but then node red complained again that it was missing the sqlite node.

Webmin is turned off by default as it takes WAY too long to load and is of limited value, so you have to TICK if you want Webmin. Just did Raspberry Pi 3 yesterday - see blog you HAVE to disable IPV6 - everything installed for me...

MrShark, that was me that updated NPM (via the checkbox at the beginning of the script) but you and Peter set me straight on that one. And in that case, it wasn't SQLite that was having problems, it was that NR would crash after trying to load any new module after initial boot.

And one additional side rant:
The IPV6 thing sounds peculiar that there would be no fallback. Surely i'm not that out of touch to have missed some tipping point where the technical world has now accepted IPV6 as the default (and certainly not for a consumer targeted device like an RPI)...

i know, but we have to see why all of the sudden the raspberries (usually with the original raspbian) seem to decide to go for ipv6 for naming resolution: which SHOULD work, but instead we had many problems with and miserably fails almost everytime it goes that path... could even and probably be an ISP issue, of course... maybe their dns cannot resolve in ipv6, but for now, the safest path to have all working is just disabling ipv6, with no appreciable consequences...

If you put an empty file in the boot partition with the name "ssh" it will start ssh automatically. And if you create a valid wpa_supplicant.conf file and put it in the root of the /boot partition it too will get your rPi onto the local WiFi network.

I wrote a script which runs in a bootable Ubuntu ISO which handles all that and even punches in a changed hostname so it isn't just another "raspberrypi" host on your network.

The hostname stuff needs to have access to the EXT filesystem and therefore it's easiest to do a bootable Ubuntu ISO to handle all that. I did this for the DIYRoboCar project.

I used the script now first time on Pi 3 with an image maybe older because of the touch screen implemented (aliexpress). For the tool i select all except the password change. It works great!
All what I can in the moment not regonize are the login credentials for the node red ui desktop. The try of pi, root or admin doesn't bring me forward.
Some suggestions from your side?

Pete, A couple of comments on your script. First and foremost -- thanks a lot for this work.

Second a couple of questions about some of your choices. Most of the traffic to the server is MQTT and Node Red (which presents its own web service). The LAMP and fileshare demands are minimal -- a home page and a few PHP services. So why use Apache2 which is resource hungry for a RPi -- especially with the out-of-box Prefork MPM defaults and use of the mod_php5 option. For this usecase wouldn't lighttpd + php5-cgi be a better fit?

Secondly Samba. I know it makes Win$ PC access easier but again it is pretty resource hungry and SFTP is absolutely fine for most usecases, (and seamless if you use Linux). OK, this needs open-ssh-server rather than Dropbear, but the resource hike for this is a *lot* less than running Samba.

i had a setup of lighttpd+php5 for a while, now lost somewhere as we did not embed it in the script, ever... maybe a better choice today, with Stretch and Xenial out there, is to move to nginx+php7... which would be even easier and more useful, as you can use it even as reverse proxy so to have the different web services all on port 80... maybe a work for the christmas holidays 🙂

As for the second, I do use openSSH, don't like Dropbear at all. I used Samba as I wanted to be able to handle the PI's by name for the likes of WinSCP and Samba does a good job of that. I've never understood why that isn't standard - it's not as if the PC is a minority player or anything.

See the comments by Mr Shark - he likes nginx, you'd like lighttd. Incidentally - I've been using the same service provider for the blog for years without issue and the moment they moved to lightpd I started having problems 🙂

You can disable any of the options and add in your own - indeed if anyone wants to ponder changes to the script - perhaps offering Apache OR lighttpd OR NGINX might be nice AS LONG AS such an installation works with all of the boards this script works with (it is not by any means just for the PI, Antonio and I have put a lot of work into making it work with FriendlyArm, Orange Pi, Roseapple Pi etc etc using a variety of different operating systems (see the list at the top of the script in the comments) - so any changes need to be considered very carefully for compatibility - and that of course demands that someone test all of the boards, some of which I for example don't have any more. Therein lies the problem.

Apache or Apache + Varnish is also my preferred engine choice for the LAMP servers / VMs that I administer, though I've also used nginx. In fact this is the first time that I've used Lighttpd. I was a PHP engine contributor for a couple of years working on Opcache and the VM internals so I also understand the strengths and weakness of PHP5 vs PHP7 and the use of Opcache, etc.

My main point is that the RPi and equivalent SBCs have limited resources -- particularly RAM and the primary job of the SBC here is to act as an MQTT broker and Node-Red server, so there are few benefits of having fat LAMP components taking up a third of the free RAM resources just to serve up the odd admin page when alternatives will do this with significantly less resources.

The same argument applies for WinSCP and Samba: heavy on resources, and for little gain in _this_ usecase. Modern routers offer DCHP and will add any DCHP clients to their local DNS cache and so you should have no resolution issues. Using MS-specific protocols is also questionable when most home browser clients run iOS or Android these days.

You are using SQLite as your DB engine. This is probably fine for this application, but it really needs a decent amount of RAM allocated to VFS buffer cache; replacing the Apache, WinSCP and Samba services frees up a lot of RAM for this purpose at no material performance hit on the small admin page use.

Thanks very much to Peter and his collaborators for their work in this arena. The
information, inspiration, leadership and helps you provide are invaluable to those of us with limited time, resources and knowledge. Enabling us to undertake projects otherwise beyond our scope. Your work is greatly appreciated!

Sorry that this is rather long. It is so because I tried not to leave anything relevant out. My problem is that I am having trouble getting the script to install properly on emmc of Nano PI NEO 2 Plus.

the script fails consistently at this point: CC(target) Release/obj.target/sqlite3/gen/sqlite-autoconf-3150000/sqlite3.o

When it fails the heat sink (oem) gets rather warm and stays that way (hours later) but console is unresponsive. Normally the unit is cool to touch. I even tried applying fan to processor during install in that case, the sink stayed cool but lockup occurred in same place. For last fail, I turned fan off after lockup and as expected temp on sink rose quickly to the 'warm' state. If memory starvation was the problem I would expect that after say, 8 hours, the process would fail but no. When I come home from work the core is still screaming away doing something to generate that heat! Also from an 8g transflash install, I am seeing 2.3G or so of available storage space which should be far more than enough – even with the 'extras' I selected.

When I began my attempts, I had added a few options (webmin, log2ram, HA-bridge), removing one at a time and ended up with only webmin (still fails). Each time I would get a fail I would re-flash eMMC and then copy over script and run again.

I have been able to get the transflash release of Ubuntu (same FriendlyARM release date) to run script to completion (log file). I am using a 16G transflash card for development and it is showing 12G out of 14.5G free so the 8G eMMC should not be an issue.

But wait - that's not all!

This occurs once during script this occurs and I have no clue what best response is:

Installing new version of config file /etc/issue ...

Configuration file '/etc/issue.net'
==> Modified (by you or by a script) since installation.
==> Package distributor has shipped an updated version.
What would you like to do about it ? Your options are:
Y or I : install the package maintainer's version
...

Remember, in every case the script is run immediately after flash and reboot of the os – no updates or other changes. The clip above ( /etc/issue )from script log / boot is from 16G transflash with same Nov 3,2017 release by Friendly ARM (not the eMMC version) but same occurs on eMMC install.

Any insights please?

P

If memory serves, doing a D seemed to break the script. It may be that I don't know the secret things every 'user' - read that - “sysadmin” should know. [The concept of 'users' in Linux is a fiction from my experience.]

Peter thank you for all you do to help those just starting out. I'm having a little operator headspace issue. Installed the latest Raspbrian on a RPi 3 which creates a pi user, droped the script.sh in pi's home directory gave it exacute and tried to run it and this is what I get. Any help would be greatly appreciated.

Hi Everyone, I ran the script on my new RaspberryPi3 and all seems OK except for the Mosquitto installation. I cant get the broker and client parts to work together. Is there any issues with installing mosquitto on to Raspbian stretch desktop?

I'm a slow learner here and keep picking up and putting this thing down after I get a head ache. I'm trying to test the MQTT out by opening two separate Putty windows and typing the following in to each, my problem is that I don't receive the "message" but see an error:

I am pretty sure you know, but in case you do not - The script does not install Node-red any more. Was using it few weeks earlier without an issue - did you changed a lot in it or it's due to new Node-red or due to Stretch updates or...? Node-red is not installed by default in Stretch anymore as well - maybe it causes all the issues?

I am using Raspberry Pi 3B+ and 2018-06-27-raspbian-stretch. Getting the same Error all the time:
script.sh: line 1153: npm: command not found

and this is not related to you board, as you said in previous comment that you're using a raspberry 3b+ so the lines that should be triggered by script are the ones between 809 and 820 of actual version... BUT, there are chances that as you have the 3b+ version, something in it could be changed and so script tries to execute wrong install...

use latest version of script, don't know which one you're using, as line 1153 has NOT npm commands in it... and those lines are ok and works, just tried in a linux shell... they have to download the correct binary version for the given architecture

############################################################
Installing Nodes (could take some time)
script.sh: line 855: npm: command not found
script.sh: line 863: npm: command not found
sudo: npm: command not found
Reading package lists...
Building dependency tree...
Reading state information...
python-rpi.gpio is already the newest version (0.6.3~stretch-1).
python3-rpi.gpio is already the newest version (0.6.3~stretch-1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Created symlink /etc/systemd/system/multi-user.target.wants/nodered.service → /lib/systemd/system/nodered.service.
############################################################

something failed in installing nodejs or npm, so everything else related failed, too... someone else please, on a raspberry, possibly a 3b+, try latest script and let me know... just a few days ago we tested same script on some boards with Peter, with no issues...

try if you can, i don't know, as said we tested everything last days with Peter, only problems where with caddy because me deleting the wrong file on my github, we didn't touch nodejs and peter didn't have issues...

start again, same script, but BEFORE RUNNING, change on line 812 that 8 to a 6, so to install previous nodejs version... this v8 is causing too much problems, but in weekend with peter we have NONE on this version... unbelievable... let us know if with v6 is fine...

Interesting - I am messing with a pair of Raspberry Pi 3 B+ and they have the very latest Stretch installed, the script with the latest Node is on there - 8.12 and npm version 6.4.1 - Node-Red version 0.19.2 and all is working well.

Thanks Antonio! V6 is working as supposed! 72hours lost but at least it works now! Thanks to pointing out! Would love to donate you some Rpi 3B+ for you just in case ... 😉 PM me your address if you agree

Welcome to the Blog

Hi - I'm Pete and this is my technology blog. It is BIG. It is also the home of "ESP-GO". "The Script", Node-Red-Contrib-Bigtimer and many other useful toys. Use the search box below or check out the archives and other links below - be sure to SHARE what you like using the social media buttons and please subscribe to my YouTube channel http://www.youtube.com/PeterScargill

Please register or log in - the top menu changes a lot when you are part of the party... and once in, don't forget to tick the box so you will get email follow-ups to comments.

If you want to buy me a coffee or help fuel my gadget habit, use the Paypal donate link below.

Disclaimer: Because I have no idea of your level of technical skill or the requirements of your country laws in terms of electricity supply etc., I accept no responsibility for any damage caused through following advice in these pages. When dealing with mains voltages you should satisfy yourself that whatever you are doing is safe and if unsure, seek advice from someone who is sure.

Notice: I'm always happy to offer advice on stuff I've written - and indeed take advice. If I can help in any way just let me know but PLEASE don't ask me how to program in C/PASCAL/NODE/etc. There are many resources out there - if you want to program and can't - there's always Google.

Search for:

Email Newsletter

Enter a name and your real email address if you would like to receive occasional summary emails - sorry but invalid-looking or automated names and email addresses will be removed to help protect others.

Links

EE Times | Electronic Engineering TimesEE Times connects the global electronics community through news, analysis, education, and peer-to-peer discussion around technology, business, products and design