Web, server, command-line and more: solutions log

Anyone who has to solve problems regularly is advised to keep a solutions log, so they never have to think ‘how did I do that last time?’ I haven't yet put up mine in a searchable or indexed way, so here's a massive long page.

Set up Apache2 for local website testing on Mac OS X 10.5

Relevant files:

in /etc/hosts
add a line for each test site:

127.0.0.1 mytestsite.lc
127.0.0.1 myothersite.lc
...

in /etc/apache2/httpd.conf
Uncomment the include to httpd-vhosts.conf# Virtual hostsInclude /private/etc/apache2/extra/httpd-vhosts.conf

…and make local Apache only look for local files - change: Listen 80
to: Listen 127.0.0.1:80

Create and enable a new MySQL user

Set up new database (e.g. wedb) and assign to a MySQL user (e.g. webdev); if the user doesn’t yet exist (or to change their password): mysql> grant all on wedb.* to webdev@localhost identified by "webdevpass";if the user does exist: mysql> grant all on wedb.* to webdev@localhost;
In either case, you should see e.g.: Query OK, 0 rows affected (0.08 sec)
then log out… mysql> quit;

Download/update and build NodeBox2 from trunk

Open the dist folder under the NodeBox project folder, right-click the mac folder and select Compress. Rename the archive to also contain the version number, e.g. NodeBox-2.0.963.zip. Note that using the zip command from terminal creates an invalid executable.

Links and resources:

Tags:

Compile Java classes into a separate directory from their source files

cd to a dir where the ‘src’ and ‘bin’ files reside.

$ javac -d bin src/dir1/dir2/source.java

The -d switch recreates the dir tree to ‘source.class’ in the ‘bin’ dir.

For classes that call others, ‘bin’ should be in the CLASSPATH because classes are used during the actual compile as they’re created. Then you can jar up the classes, or recompile a new version. You can use -sourcepath and -classpath.

If there’s a file that uses source.java in: src/dir1/dir3/another.java then this path is required too: src/dir1/dir3/ (separate multiple paths with a colon :).

If working with multiple projects or versions, keep paths away from CLASSPATH or the default Java class directory to avoid old/new class collisions, and consider using Ant or Maven which use XML to manage projects.

Use grep backrefs (in TextMate, etc.)

Links and resources:

Tags:

regex, textmate, grep, backrefs

Set up a new domain/subdomain on a Debian Linux server under Apache2

General (domain):
setup the domain in Fasthosts/Rimu DNS control panel BEFORE setting Rimu name servers at the domain name registrar:
https://rimuhosting.com/dns/zone.jsp?zone=your_domain_here Dallas 72.249.38.189London 94.76.200.197
Rimuhosting (ecoconsulting.co.uk): https://rimuhosting.com/dns/records.jsp
Fasthosts (subdomain): add new subdomain ‘A’ record in ‘Advanced DNS’ and point to the server IP address.

In this example there is a global link style of pale grey with a dark grey hover:

a:link,
a:visited { color: #999; }
a:hover { color: #666; }

If JQuery is used to fade in the hover state colour at this stage, the first hover over a link will show the sudden css hover - the fade will only appear on the second hover.

So, to preserve this css hover rule without javascript, one solution is to create a css class that matches the css :hover colour attribute to the :link state, allowing the JQuery color fade to show first time. This is only appended to the ‘wrapper’ div by JQuery if javascript is available:

/* if javascript is available the .js class is only added with JQuery, so
css hover and link colours match and the JQuery fade shows instead */
.js .menu a:hover {
color: #999;
}

Now, hovering over the links won’t show an initial sudden css colour change (to #666) instead of a fade. Adding the .js class to the wrapper div means that this effect can also be applied to other links inside it - just add their selectors to the $("#menu li a") part like this: $("#menu li a, #submenu li a").

‘View source’ will not show the JQuery-generated class="js" addition to the ‘wrapper’ id - you will need to inspect the generated source code (right-click > ‘inspect element’ in Safari [other browsers?]).

Links and resources:

Tags:

Why document.write isn’t the best method

There are better ways to incorporate strings into HTML content, such as .innerHTML and .createElement or HTML cloning patterns - document.write should be avoided because:

it is recklessly dependent on timing, so if called before the onload event, it appends or inserts text into the page. If it is called after onload, it completely replaces the page, destroying what came before.

document.write encourages bad structure, in which script and markup are intermingled. A cleaner structure has minimal interaction between markup and script.

Make sure included files themselves are saved with utf-8 encoding from the text editor. Otherwise (even if the parent html page is utf-8) they’ll be imported with their file encoding, and won’t even respond to Apache directives like AddDefaultCharset.

Links and resources:

Tags:

PNG file with alpha transparency and opacity fades to black in Internet Explorer

Don’t use an ie-specific opacity filter in css for elements having a background png image with alpha channel transparency. There is a fix (see link) using an IE filter and rgba, which can be activated using Javascript/JQuery or placed in a CSS file.

Links and resources:

Tags:

CSS, IE, Internet Explorer, png, alpha, transparency, opacity

Border-radius fails for images in Firefox

wrap the image in a span with the same height and width as the image, apply border-radius to the span and use the image as a background to the span. The xhtml-only SVG clipping path method works, but fails with box-shadow (adding the path the the outer border of the shadow), and isn’t future-proof (i.e. FF 4 supports border-radius for images).

Links and resources:

Tags:

number theory, maths, digital root, Ruby, Python, Perl

When to use Python xrange() over range()

xrange() is more efficient than range() especially for larger iterations: range creates a list, so if you do range(1, 10000000) it creates a list in memory with 10000000 elements, while xrange is a generator, so it evaluates lazily and keeps only one number in memory. When testing this I found the following:

iTunes won’t accept album artwork or wma files

Links and resources:

Tags:

Install MongoDB on OS X

The ‘quickstart’ on the MongoDB site could be clearer. Here’s a condensed version with more info.

Get the URL (copy the URL, don’t click) for the latest download for your system from: http://www.mongodb.org/downloads

Open Terminal and go to where you keep source files (good to create a directory if you don’t have it already) e.g.: cd ~/src then type curl, the URL you’ve copied, > (between spaces) and the filename you’ll save it as e.g. (for the 64-bit version of Mongo 1.8.1): curl http://fastdl.mongodb.org/osx/mongodb-osx-x86_64-1.8.1.tgz > mongo.tgz

Make the default directory for MongoDB to store it’s data - type: mkdir -p /data/db
(-p also creates the intermediate ‘data’ directory)
If you want see what’s there after you’ve used MongoDB, type: cd /data/db;ls -al

To use these new aliases, close the Terminal window and open a new one. You can now test MongoDB by starting the MongoDB server with: mongod
then open another Terminal window and type: mongo
In the MongoDB shell that’s now started up, create and retrieve a test document ‘foo’: > db.foo.save( { a : 1 } )> db.foo.find()

Tags:

fcgi, Ruby, gem, fcgi-devel, Rack, install

Microsoft Word keeps reverting to US English/a default font/whatever

solution:
Find and edit the basic template Normal.dotm, then save it. All new documents will reflect the changes. In OS X you’ll find it here: [home]/Library/Application Support/Microsoft/Office/User Templates/Normal.dotm

Links and resources:

Tags:

Word, template, default, Normal.dotm

Set logs to rotate in Apache

Find rotatelogs, then add the following (e.g. depending on your paths) to the VirtualHost block for that domain:

This function can be run periodically to create a tags collection from the posts collection. It’s not quite real-time, but … close enough for most uses. (…for real-time, there are other techniques) …the query is more complicated than just selecting out of Tags, but inserting and updating an individual post (the main use case) is simpler.

Add the remaining, and last 3 links, to my NoSQL Google doc:
if you don’t know how to use document database specific features such as map/reduce, or how to model your data … to take advantage of them, you won’t be able to adequately evaluate those databases. Similarly, if you don’t know how to use pessimistic locking or referential integrity in a relational database, you will not see how much time and effort it could be saving you over trying to implement such robustness in a NoSQL database that wasn’t designed for it.

“the most common interpretation of “NoSQL” is “non-relational”, although NoSQL is not meant as anti-RDBMS, but emphasizes the advantages of Key-Value Stores, Document Databases, and Graph Databases” “Typical RDBMS implementations are tuned either for small but frequent read/write transactions or for large batch transactions with rare write accesses. NoSQL on the other hand, services heavy read/write workloads” - WIkipedia

Links and resources:

Tags:

bash, shell, version

Make changes to currently running bash shell take effect

You’ve changed a variable in a bash configuration file (e.g. with: export VARNAME=the-new-value), and you want it to take effect without leaving the shell - enter: source .bash_login
(or .bash_profile, or whatever the name of the changed bash config file is)

Tags:

Mongo 32-bit won’t run on OS X 10.4.11 (Tiger)

“We’ll end up having a separate build for tiger - so the 32-bit regular os x build isn’t expected to work. We should be getting the tiger build up and running for people to test sometime today. Building from source should also work though.” - Michael Dirolf

Require a specific Ruby gem

Links and resources:

Tags:

ruby, gem, require

Get Safari to show a new favicon

Safari tends to hang on to the version of a favicon loaded when a website is first visited. Go to ~/Library/Safari and move the file ‘WebpageIcons.db’. Reload the page, then replace the file. If this doesn’t work, you may need to go to History > Show All History and delete all pages showing the icon you want to replace, and try again.

Links and resources:

Tags:

Safari, favicon

Calculate one number as a percentage of another

Excel: =((smallerNumber/largerNumber)%)*10000
e.g. for 7.84 as a % of 29.75 = 26.35 =((7.84/29.75)%)*10000

Links and resources:

Tags:

Add rar and unrar capability to OS X

Firstly, download RAR for OS X from RarSoft, double-click to archive and drag the resulting rar folder to Applications.

Leopard 10.5: go to your Home folder and open Library/Workflows/Applications/Finder.

Snow Leopard 10.6: go to Home/Library/Services.

Create any of these folders if they don’t exist.

Download this Automator script and put it in the Finder folder above (the megaupload.com download is not available).

You can now right- or ctrl-click on a folder or file in the finder and (from the bottom of the resulting menu) choose More > Automator > RAR selected files. It’s slower than creating a zip archive (so if it’s a large file or folder, give it time), but it works.

Links and resources:

Tags:

javascript, constructor

Setting RVM default Ruby

View your installed version of Ruby with rvm list, and copy the version you want as a default.

rvm alias create default ruby-1.9.2-p290
or in latest RVM docs(?): rvm --default use 1.9.2-p290
which sets the default Ruby for all new terminal windows (the alias method didn’t do this on OS X 10.5.8 for new windows?).

Go back to system Ruby: rvm use system

Switch to RVM default Ruby: rvm default

Remove rvm default Ruby: rvm reset

Links and resources:

Tags:

rvm, ruby, default

Debian apt-get cheat sheet

aptitude is the text-based interface to the Debian GNU/Linux package system. Debian package names end in .deb

Links and resources:

Tags:

Java: compiling and directory hierarchy

dot-separated classnames need separate directories - in the src directory make a hierarchy that mirrors your package hierarchy e.g. given a class named com.elharo.math.Fraction put Fraction.java inside the math directory:

src
com
elharo
math
Fraction.java

Compiling Java code is tricky because you need to keep track of several related but different things:

The target file you’re compiling.

The directory where the compiler looks for .java files that the target file imports.

The directory where the compiler looks for .class files the target file imports.

The directory where the compiler puts the compiled output.

By default, the javac compiler thinks these are all the current working directory, which is almost never what you want. Consequently, you need to explicitly specify each of these elements when you compile.

If you use the Java CLASSPATH (because you’re only compiling to the same directories every time, say if you only have one Java project), it needs both the source and target directory because the compiler needs to find the .class and .java files each target file imports (check this).

Installing Fortran on OS X as a dependency of scipy (or of R)

error: SandboxViolation: open('/dev/null', 'w') {}
The package setup script has attempted to modify files on your system
that are not within the EasyInstall build area, and has been aborted.
This package cannot be safely installed by EasyInstall, and may not
support alternate installation locations even if you run its setup
script by hand. Please inform the package's author and the EasyInstall
maintainers to find out if a fix or workaround is available.

the issue for me was that scipy requires Fortran, which doesn’t come with OS X and wasn’t installed on my (Leopard) system with Xcode 3.1.4.

Links and resources:

Tags:

python, OS X, scipy, GNU Fortran, GFortran, [R], CRAN, virtualenv

How to scale SVGs for responsive web design

Make sure the SVG root element has a viewBox attribute e.g. <svg version="1.1" viewBox="0 0 300 185">
where first two parameters (0 0) are the origins of the viewBox, the second two are the width and height.

Links and resources:

Tags:

css, geometry, triangle, mathematics

Using RSA for ssh, sftp, rsync, etc. without a password

If you’re developing websites and frequently logging into the server via a command line to upload files (e.g. for static site generator nanoc’s nanoc deploy command) you can set things up so you don’t have to enter (or remember) the user’s password every time.

Open a command-line terminal while logged as the user who will run scp, ssh or rsync (i.e. when making/uploading websites).

To find out if you already have an rsa keypair enter cd ~;ls -al - if you see an .ssh folder, enter cd .ssh;ls and check for the keys: id_rsa and id_rsa.pub

If there is no .ssh folder, make sure you’re at the root level, create it and enter it:

cd ~
mkdir .ssh
cd .ssh

Create the rsa keypair

Now you’re inside the .ssh folder, create a private and public keypair by typing:

ssh-keygen -t rsa

When prompted ‘Enter file in which to save the key’, press return or enter to name the pair with the default name ‘id_rsa’.

Do the same for ‘Enter passphrase’, otherwise you will need to enter a passphrase each time you log in, which defeats the password-free purpose.

You normally won’t need the randomart image, so it can be ignored, or you can copy and save it in a text file.

Create a public key folder for a user on your server

You will need to do this for each user on the remote machine where you upload website files via a terminal. If you’re managing multiple websites, your server will likely have a user account for each website. If you only manage one, you only need to do this once.

Make sure you’re still in the .ssh folder and can see the keypair:

cd ~.ssh;ls

If you see ‘id_rsa.pub’ and ‘id_rsa’ both listed, login in to the remote user’s account on the server:

ssh username@servername_or_ip

Enter the usual password, make sure you’re at the user’s root directory, and check for an .ssh directory:

cd ~
ls -al

If there isn’t an ‘.ssh’ directory, create it, set permissions that restrict everyone except that user, and cd (change directory) into it:

mkdir .ssh
chmod 700 .ssh
cd .ssh

Upload the public key and save its contents

Create an ‘authorized_keys’ file to hold your public key data and set the file’s permissions:

touch authorized_keys
chmod 600 authorized_keys

Confirm the exact path from the server’s root folder to the user’s .ssh folder:

pwd

Copy this path (something like /home/username/.ssh) and use it in the following command, which will securely copy (scp) your local public key file to the user’s .ssh folder. Use the same ‘username@servername’ you used to log in via ssh:

scp id_rsa.pub username@servername:/home/username/.ssh/id_rsa.pub

Use ls to check that the key has uploaded—if you see the two files ‘authorized_keys’ and ‘id_rsa.pub’, enter the following command to copy the contents of your public key to the ‘authorized_keys’ file:

cat id_rsa.pub >>authorized_keys

If you enter less authorized_keys you should see a string of text, which means that your public key is now stored in ‘authorized_keys’. Type q to exit less, and delete your public key file:

rm id_rsa.pub

You should now be able to logout of ssh and login again as before, but without being prompted for a password.

Note: if the user’s home directory on the server is accessed via NFS, then permissions for .ssh has to be 711 and ‘authorized_keys’ 644.

Like rvm, you can now install gems without sudo for your user (not system-wide):

gem install (gem_name)

Homebrew advises:

NOTE: By default, gem installed binaries will be placed into `/usr/local/opt/ruby/bin`

You may want to add this to your PATH.

so you will probably need to add /usr/local/opt/ruby/bin to the end of the colon-separated PATH variable in .bash_profile (if you already have one—in your home folder—create the file if you don’t already have it) e.g.: export PATH=/usr/local/bin:(more paths here):/usr/X11/bin:/usr/local/opt/ruby/bin

Export multiple layers as separate images

Links and resources:

Tags:

photoshop, adobe bridge, multiple images, layers

Share an iCloud calendar in iCal 4 (OS X 10.6 Snow Leopard)

If you use Snow Leopard and want to access/edit calendars that you have in iCloud (say from more recent OS X versions or iOS devices), older advice is far too complex—the most recent iCal 4 in Snow Leopard makes this possible. Follow these steps:

Update your calendar either in iCloud or on a more recent OS or iOS device

From your iCloud account click the ‘share’ icon (right of the calendar name) on the calendar you want to access in Snow Leopard.

Check ‘Public calendar’ and note the first part of the URL, something like: webcal://p06-calendars.icloud.com/[…]

In iCal in Snow Leopard, open Preferences, click ‘Accounts’ and click the + (bottom left) to add a new account

In the dialogue box ‘Account type’ choose ‘Automatic’ and enter the email address and password you use to sign in to iCloud e.g. my.name@me.com

Once your login is authenticated, in ‘Server settings’, modify the first part of the URL above follows: p06-caldav.icloud.com and check ‘Use SSL’

There's an option to animate style changes, but this seems excessive when it's enough to see the changes updated automatically. The ember-cli console outputs Reloading saves-client.css only when you change the stylesheet.

Regex to replace LaTex quotes (`') with typographic quotes

Add whatever other characters you need inside the square brackets [], making sure to escape special characters such as brackets, forward slashes, addition signs, etc. with a backslash \ like this \/\)\+.

Links and resources:

Tags:

latex, atom, atom editor, regular expression, regex

Uninstall the version 3 or older Xcode ‘Developer’ Folder and Contents

XCode 4.x is installed in /Applications/, while XCode 3.x is installed in /Developer/.

If you've installed a newer version of Apple's XCode but an older version (3.x) is still in the /Developer/ folder, run the following command in Terminal, rather than manually deleting it through the Finder:

sudo /Developer/Library/uninstall-devtools --mode=xcodedir

This keeps other aspects of Xcode intact but removes everything within the /Developer/ folder:

Variations of this command uninstall other components, or the whole of XCode—see the links.

Links and resources:

Tags:

node, homebrew, downgrade, version

To open “CS5ServiceManager” you need to install the legacy Java SE 6 runtime

Trying to launch any Adobe CS5 app brings up a warning about Adobe CS5Service Manager wanting the old Java 6. Installing OS X (from Lion onwards) comes without Java, which is needed for Adobe's update manager. If you don't need to update Adobe software (e.g. because your're running CS5 on OS X 10.11) you can remove it in Terminal:

sudo rm /Library/LaunchAgents/com.adobe.*

Warning: opening an older Adobe app will still bring up a dialog warning that it needs the legacy Java 6 version. You can install that from Apple at Download Java for OS X 2015-001 and restart. You may need to re-enter your licence number.

Links and resources:

Tags:

MySQL password has expired

Typically, in a script shell, you get the message: "Your password has expired. To log in you must change it using a client that supports expired passwords."

From MySQL 5.7.4 to 5.7.10 (and beyond?) the default value for the default_password_lifetime variable is 360 (a year). For those versions, if you make no changes to this or to individual user accounts, all user passwords expire after 360 days.

To prevent automatic password expiry log in as root:

mysql -u root -p

then, for clients that automatically connect to the server (e.g. from scripts.) change the password expiration settings for those clients:

ALTER USER 'script'@'localhost' PASSWORD EXPIRE NEVER;

or just disable automatic password expiration for all users:

SET GLOBAL default_password_lifetime = 0;

To make this permanent, add it to the/a MySQL my.conf file. This is usually at /etc/mysql/my.conf but may not exist by default. Mac OS X Homebrew MySQL installs add a default conf file in a different location but MySQL may need it in a more familiar location. In my case it's /usr/local/etc/my.conf so I added:default_password_lifetime = 0

In your project directory, if you check the installed version with ember -v it should be different to your globally installed Ember-cli (to verify the Ember version, either do Ember.VERSION in the browser console or look inside /bower_components/ember.js where you'll find the version in the comment at the top.

Links and resources:

Tags:

mariadb, mysql, homebrew, os x

Installing Python 2.7 on OS X with Homebrew

For a clean version of OS X (10.13 High Sierra), with XCode and Homebrew installed.

The OS X system Python has some shortcomings and it's better to manage programming language software yourself rather than depend on system permissions, updates, etc. You can check the system Python and its location:

python --version
Python 2.7.1
which python
/usr/bin/python

Return to these after installing Python with Homebrew. So, do some housekeeping before any new installs (will generate long readouts):

The instructions on the Homebrew site say brew install python@2 but (without thinking) I did:

brew install python2

which worked fine. Homebrew will do some housekeeping, then tell you:

Installing dependencies for python@2: gdbm, openssl, sqlite
…

Now check the Homebrew-installed Python:

python --version
Python 2.7.14
which python
/usr/local/bin/python

which python2 will give you:

/usr/local/bin/python2

because they both link to /Cellar/python@2/2.7.14_3/bin/pydoc2.7 so just python is fine, unless you also installed Python 3 with Homebrew, in which case be explicit: python2.

Upgrade pip and setuptools

You'll also see that pip and setuptools are also installed, in the location where Homebrew keeps everything tidy:

which pip
/usr/local/bin/pip

Update them:

pip install --upgrade pip setuptools

Now pip install <package> will install the chosen package into the Homebrew-installed site-package directory /usr/local/lib/python2.7/site-packages

Install virtualenv

No Python development environment is complete without virtualenv and (to make using it easier) virtualenvwrapper. Note: DO NOT sudo—it's not required for Homebrew installations—the whole idea is to keep the system-wide environment clean:

If you want to continue working in the same Terminal window, reload the .bash_profile file.

source ~/.bash_profile

Conflicting information on the web

There are several opinions/options about:

how to determine the location for the virtualenvs folder

what to name the folder for virtualenvs

which bash file (.bash_profile, .bashrc, …)

what path to use for the source property

e.g. export WORKON_HOME=$HOME/.virtualenvs where either $HOME or ~/ will ensure that the folder for your virtualenvs is created in your home directory e.g. on OS X: /Users/yourusername/.virtualenvs (The leading . hides the folder in desktop windows).

Links and resources

Tags:

python, virtualenv, virtualenvwrapper, bash, bash_profile

pip install MySQL-python fails

MySQL-python is outdated and can fail to install with a stack trace that ends error: command 'clang' failed with exit status 1. It appears mysqlclient (forked and updated from MySQLdb and also with python3.3+ support) is a good replacement: