Tally-ho! (begin…)

This manual is a pre-release preview. Please follow Moonstalk on Twitter or Facebook to be notified of a public release. If you like running with scissors, you can pull from our public repository on Bitbucket.

Installation

Whoah!

Please be aware that Moonstalk is not suited to the following scenarios. See our Pivotal Tracker project for tasks and fixes in upcoming iterations.

HTML files that are frequently updated on disk and predominantly static content with templates. Moonstalk is designed for dynamically generated pages. Static content (e.g. .html) can however be simultaneously served using the default web server configuration, optionally enabling in-memory caching.

Large datasets exceeding RAM. Not suitable for use with the included Teller database, but you can use another DBMS.

User-accessible site filesystem. No site sandboxing is provided, and sites requiring sandboxing due to user-modifiable code should be deployed on separate [virtual] machines. Access to user-specific data/ subfolders for file import routines is a safe alternative, and exported files should probably be served through a controller not from the filesystem.

Choose a server

You should be able to run Moonstalk on pretty much any Linux or UNIX-based computer.

Moonstalk is not currently compatible with the Windows OS, and has no roadmap for it.

For best performance when hosting busier sites with VPS hosting, choose a package that includes use of 4 or more cores (1 core is adequate for most sites). This site is being served by Moonstalk running on a 4-core Linode (in London). You can take a peek at the response headers to check this (each request is likely to be served by a different backend, indicated by the last number in the node of the X-Powered-By header).

If default log rotation is not disabled Lighttpd will periodically be restarted without its Moonstalk configuration (it's also started immediately upon install with its default config, which we don't want).

On Debian apparently if you want to run using non-root privileges, you currently need to change the permissions on /sbin/sysctl, or copy it to /bin.

Mac OS X

lighttpd is the default webserver. libfcgi and spawn-fcgi are required to interface with the webserver. lua5.1 is the language runtime that Moonstalk uses. libuv is a library for handling connections used by the database .LuaRocks is an optional package manager for Lua modules. mercurial (hg) is required only for installing and updating Moonstalk, but you may use other systems for your own repositories (see Updating). You may manually install any of the above if you prefer.

Install Moonstalk

cd /usr/local

hg clone https://bitbucket.org/moonmill/moonstalk moonstalk

cd moonstalk

./runner

The ./runner command won't start anything now, but will check for required Lua modules, just hit return to install them (using LuaRocks) when it prompts you. (This may take a while.)

If you prefer you may install modules manually, but ensure you install them in a directory prefix configured in your $LUA_PATH.

If you're setting up a production server, be sure to edit data/configuration/Host.lua before proceeding. Or simply: ./runner hostname=server.example.com logging=0

Ensure that your moonstalk/ folder is not in a public path (e.g. an existing web or FTP server root) and that it is not readable by any non-adminstrators.

Enable the Manager

Moonstalk includes a simple CMS system named Manager, you'll need an administrator login to use it, simpy run ./clerk scribe "manager.Operator()" and it'll give you a new password, on your development machine you can then access the web interface at localhost/Manager.

Settle-in

That's it, now you can start coding or use the installed applications! If you're fairly new to web development or would like a simple introduction to Moonstalk development, read on — otherwise if you're a daring adventurer leap ahead to the learn section.

Editing

Your primary tool for hands-on development with Moonstalk should be a text editor or IDE having syntax hilighting for Lua, HTML and CSS.

If you'll be using localised text it must support a 'no-BOM' encoding option for UTF-8 which most do.

Collaborating

If you're working with multiple colleagues, you can use the Mercurial (hg) distributed concurrent versioning system (dCVS), as used by Moonstalk itself for installation. If you're new to Mercurial, there's a site for that.

Tutorial

Ready for a quick hands-on tutorial?! I'm assuming you've never traded your soul for a copy of FrontPage, and know how to hand-code HTML—or at least won't be too scared reading some. If not, open the source view for this page or any other, start reading it and I'll see you back here later.

Before we do anything, you must change your working directory to your moonstalk install folder.

cd /usr/local/moonstalk

Define a site

Sites are created as sub-folders inside the sites/ folder, each named with the site's (primary) domain.

Create an empty folder inside sites/ named as the site's domain (e.g. example.com/).

If you don't have a domain make one up, or use example.com.

Configure DNS

For development and testing on your own computer you must be able to access your site at its domain, but you probably shouldn't change its public DNS records yet.

Add an entry to your /etc/hosts file to temporarily point the domain to your computer's public IP address (if others need to access it), or 127.0.0.1 (if only you will be accessing it).

Rember to disable or remove this entry when you need to access the actual server again. You may need to flush your DNS cache or restart your web browser for changes to take effect.

Instead of editing /etc/hosts you could create a new DNS record like test.example.com and point this to your computer so that you can easily swap between test and production servers.

If you're using OS X I can recommend using HostsWidget to manage your /etc/hosts file.

Start the server

After defining a site, you can start Moonstalk. The following command will start all the servers (web, pages, and db), and enable development mode which disables Moonstalk's page caching — allowing you to see changes as you refresh a page in your browser during development. As you're not yet running a production server to handle heavy loads, only a single page-generation (Scribe) instance is required.

./runner start logging=4 instances=1

Add static files

Drop any HTML files and any images or other assets e.g. logo.jpg or page.html into the folder and access them in your browser with their full filename e.g. http://example.com/logo.jpg. Static requests are served by the webserver directly, without any processing by Moonstalk.

Any request containing an extension is considered a request for static content, to be served by the webserver directly.

Add dynamic files

With a variable

Let's try a simple page that just displays details of your web browser. This uses dynamic markup inside an HTML file, which we call a view. Dynamic requests are processed by Moonstalk's Scribe (pages) backend.

Any request not containing an extension is considered a request for dynamic content, to be processed by the Scribe.

Create a new file sites/example.com/hello.html for the view with the following contents, (it's not a complete HTML page but we won't worry about that for now).

<p>Your browser is: ?(request.browser)</p>

The ?( wibble ) markup is called an expression tag (or write macro). When a page containing this is requested, Moonstalk evaluates the variable or expression inside the tag (e.g. wibble) and replaces the tag with the result.

Moonstalk has many variables organised into groups called tables. When values are contained within parent tables their names are delimited with '.' (period). In the above example the table is named 'request', and the value 'browser'.

An expression tag is similar to the «field» placeholder in Microsoft Word mail merges.

Because you've just added a new page, before you can access it you must restart the pages backend to include it in the site. ./runner restart pages

Now go to http://example.com/hello to try it.

Note the lack of a file extension in the address. Views (in sites) are automatically mapped to addresses without extensions. Remember, a request with an extension is a request for a file's static data.

With a form

Now let's try something a little more interactive. We'll use a form to input our name and display it upon submission, using an alternative if no name has been submitted yet.

Because you've just changed an existing page and are using developer mode, you do not need to restart anything. Just refresh your browser window to see the change, and try it out.

Define templates

The hello.html file lacks an <html> tag to make it a complete page, because we're now going to put it in a template. This will apply a common layout to all of our test site's files when Moonstalk serves them.

Create a template.html file in your site folder with the following contents.

The <? ?> markup is called a server processing tag. Inside this you can run any valid Lua code.

scribe.Insert "content" is a function you can use in templates that indicates where in the template to put the content from the requested view, in this case hello.html.

Whenever we create a new page (including templates) we must restart the pages backend so that it can cache it. ./runner restart pages

Restarting just the pages backend should not interrupt traffic, providing multiple instances are running—which is the default on a multi-core CPU.

Refresh the /hello page in your browser to see it inside your new template.

Save data

So now let's try something a little more clever. We'll save a value from a form upon its submission, and then display its saved value. We'll do this by putting code into an additional Lua file which we call a controller.

Create a settings.lua file in your site folder with the following contents, which tells Moonstalk we want to use a database table called ‘mydata’.

Create a new file called hello.lua for the controller with the following contents.

if form.myinput then
save ( mydata.isaved, form.myinput)
end

Because we defined a database, and new controller, we need to restart both. ./runner restart pages db

Refresh the /hello page in your browser to try the new functionality.

Use the log

If you ever lose track of what-on-earth you think should be happening during development, it's useful to see operations step-by-step. In fact you might like to keep the log open whilst developing and it will show what's happening with each request.

Linux

OS X

double-click the temporary/moonstalk.log file

Resubmit the form on the /hello page with a new value, then look back through the log to see where the save occurs and confirm the input value.

Learn…

If you're not too sure about any of the jargon used up til now, you might like to take a ganders at the following glossary, otherwise jump ahead to the learn page to dig into some usage guidance, and then the develop page to for reference and more advanced topics.

Glossary

Although we attempt to avoid using jargon and proprietary terminology, it's useful to understand the naming conventions used to distinguish the various components and concepts.

Server terminology

The [World Wide] Web

The collective identity of one part of the Internet comprising a multitude of server computers that all perform the same function — like Email allows the exchange of messages, or FTP the exchange of files — the web allows the exchange of pages (which may be plain documents, or interactive software interfaces), collected together into zillions of separate web sites. So-named as a web because most of these pages and sites are linked together, enabling association and exploration amongst them.

A 'server' is simply something that accepts and responds to instructions from a ‘client’. These can be computers (hardware), or combinations of operating system and application programs (software) for which the term is used interchangeably (perhaps confusingly).

Web site

A collection of pages that are accessed via a web browser application (i.e. Safari, Internet Explorer, Firefox, Chrome) on your computer, either corresponding to files in a folder from a web server (a statically generated site), or a web application. [Moonstalk hosts web sites, and runs web applications.]

A 'dynamic' page (or site) may refer to either a dynamically generated page, or a page with dynamic and interactive elements (i.e. using JavaScript in the web browser), but not necessarily one that was also dynamically generated by a web application.

HTTP

The protocol—akin to an envelope—used to send instructions between one place on the network, and another. HTTP is the protocol of the Web, and is thus used by a web browser (on a client computer) to send a request to a web server application (on a server computer).

Request

The letter inside the HTTP envelope, is a request. It contains the instructions sent from the visitor's browser, asking the server for a page at a particular address, and if submitting (posting) a form, the contents of each field.

Address

All web sites and their pages have addresses. These comprise a domain name (e.g. example.com — representing a server), and an optional ‘path’ (e.g. /mystuff — representing a resource on that server, such as a folder or file). Together these form the address example.com/mystuff yet this address could be for somewhere other than the Web, so to identify it as a web address it can be further combined (prefixed) with the Web's protocol, to form a URL, becoming http://example.com/mystuff.

Although a full URL properly represents a web address, some web sites use a server with a domain name starting www. (such as www.example.com) and in this case the http:// prefix is somewhat redundant as it is clear that the domain is a ‘world wide web’ (thus HTTP) address. Further, because the web is the most popular part of the Internet, it is almost given that any address without an identifying prefix (such as example.com/mystuff or just example.com) is probably a web address and not, for example, an FTP address.

Web server

This application is the frontend of a server which accepts and responds to HTTP requests (often many simultaneously). With static websites, content is hosted (served from disk) by the web server itself, however with dynamically generated sites, the pages are generated by a web application through a backend of which a server may have one, or many for the same or different web applications. Some web servers may integrate the backend, whilst others may use separate backend application server programs or even separate server computers.

Web application

Software (a set of instructions) that dynamically generates pages, such that they appear different for every visitor, or every time they are accessed, by incorporating values from a database (a data storage application) into the page according to various criteria defined by the application developer. Such applications are built using a programming framework. [Moonstalk is used to build web applications.]

Framework

The integrated components of a programming language and libraries of useful software commands. This aids performing specific tasks, and in the development of web applications, which are run by a backend. [Moonstalk provides a framework.]

Backend

A server application that understands and interprets the language of a web application (e.g. PHP, Ruby, C#, Lua), to generate its pages on-demand when the web server forwards requests to it. [Moonstalk provides a backend.]

FastCGI

Another protocol, used to forward HTTP requests from the web server application, onto the backend application. In effect the request sent in a HTTP envelope between two different computers (client and server) is taken out, and put in a FastCGI envelope to be sent between the two different programs (frontend web server and backend page server—typically on the same computer).

Hosting

The service and functionality of delivering something such as a web page from somewhere, such as a web server, to someone, such as a visitor—thus web hosting. Both a server or service provider offering hosting are called a host. [Moonstalk configures and provides hosting.]

Server node

This is a host, that is either a dedicated (physical), virtualised (VPS/cloud), or a shared server (shared hosting) that runs the operating system (e.g. OS X or Linux) on which a web server application is installed (a web server). The server operating system (OS) provides bottom-layer functionality such as networking and storage. A server node is managed by an operator, or service provider, and provides hosting services to multiple site owners.

Moonstalk terminology

Lua

This is a programming language and a runtime (application) for interpreting software code written in its language, so that it can be run (executed) on a computer. [Moonstalk and its hosted web applications use Lua.]

Pages

HTML ‘view’ (layout) files, containing database field markers or server processing tags, and optional counterpart Lua ‘controller’ (logic) files. When we refer to 'pages' we may be referring to either or both of these components.

Sites

Folders of pages and images, etc.. A site’s pages are specific to one or more domain names. Sites may be managed by individual site owners.

Applications

As with sites, but comprising additional Lua files for functions. Applications may be accessed across multiple sites. Applications are installed by the server operator and not site owners.

‘Scribe’ Backend

The FastCGI server program that receives requests from the web server, and returns pages from a site after generating them, and typically by querying an application, database or folder for their content. Multiple scribe processes ('backends') run simultaneously on a single server node, and the web server queues requests, passing them to the next available backend.

‘Teller’ Database

Moonstalk's database system is a server program that handles requests from scribe servers to fetch or save data. The teller uses native Lua as both the protocol and query language (not SQL), and does not use a defined schema. The teller also provides persistence (automatically saves data) and replication (synchronisation with other nodes).

‘Manager’ Application

This is an included web application, available by default on all your sites, that enables per-site access to a page and content management system (CMS).

‘Runner’ Utility

This is a command-line launcher program that provides the functionality to start, stop or restart the Moonstalk components, including the web scribe.

‘Watchdog’ Utility

This application keeps an eye on Moonstalk's applications and can tell the runner to restart them and notify someone if problems are detected.

Libraries

Moonstalk relies on a variety of shared third-party software for specific functions. Applications may in turn utilise these, or their own. In Lua libraries are referred to as 'modules'.

Dig-in! (learn…)

This manual is a pre-release preview. Please follow Moonstalk on Twitter or Facebook to be notified of a public release.

Create

This section provides an an overview for the use of Moonstalk, and the basics of adapting it to common needs. See the develop page for detailed attributes, behaviours and functionality.

Lua

If you've not learned the basics of the Lua language yet, now might be a good time. Programming In Lua is an easy to understand and straight-to-the-point guide that'll introduce you to the essentials in just a couple of hours. If you know other languages you may well be able soldier on and extrapolate much of its syntax.

Bundles

Moonstalk hosts both sites and applications, and we refer to both generically as ‘bundles’. Each are defined by folders containing much the same set of files for their resources.

Sites provide content and functionality that is specific to one or more domains.

Applications supplement one or more sites with shared content and functionality.

Creating a bundle is as simple as creating a folder, and putting some files in it. The following are files common to bundles (both sites and applications). All are optional and may define functionality indepedantly of each other, or in combination.

view.html
All HTML files are loaded as views (pages). HTML files may either be static (containing browser markup tags), or dynamic (containing both browser markup and server processing tags).

controller.lua
Except for the named files below, all other Lua files are loaded as controllers (containing server processing code).

template.html
The default template for your views (a view itself).

settings.lua
Defines Moonstalk's hosting behaviour for the bundle.

functions.lua
Defines a library of common functions that may be used in any of a bundle's views and controllers.

database.lua
Defines database procedures and indexing functions (if used with the Teller database).

Any file not otherwise noted and having an extension, is considered to be static content and will be served directly by the web server. See the design page, and bundles reference section for specific details.

You may create subfolders and organise your files as you wish. However top-level sub-folders having the name of another application will extend (supplement) or override (replace) that application with its own contents.

Sites

Site bundles are created inside the Moonstalk sites folder, and each bundle (folder) defines an individual site, named with its primary domain.

The primary domain my be specified with or without the www prefix, but both are always handled, therefore ensure your DNS is setup with records for both.

See the Tenants application for details of how sites can be stored in a database (e.g. for SaaS).

Features

Multiple domains
In the settings you may specify an array of additional domains on which the site should be hosted:domains={"example.net","www.example.net",}

Primary domain redirection
If your site has multiple domains, they are automatically redirected to the primary. This may be disabled by specifying redirect=false in the settings.

I recommend keeping redirection enabled, and ensuring that all links and promotional materials reference only one domain name, as this benefits your site's search rank, and reduces the potential for confusion.

Canonical tag
If your site has multiple domains, and redirection is turned off, a canonical metatag is inserted into your page instead.

Files

front.html
The site homepage, it should be present except for a site where you are using an application that defines it.

Applications

Application bundles are created inside the Moonstalk applications folder, and each bundle defines an individual application, named with its ID.

Applications provide shared or generic functionality (such as a blog system) for use on multiple sites (disparate domains). Applications must typically be enabled in a site's settings in order to use them, refer to an application's own documentation for details of its behaviour, configuration, and usage. Moonstalk includes several bundled applications.

Features

Global
Access to the entire Moonstalk environment, and thus all tables, sites, users and other applications.

Customisation
You may customise application's views and controllers by replacing them with your own copy. See bundle behaviours.

Files

myapp/*
As applications are shared they have no common domain on which to serve static content. Static content is therefore hosted from a subfolder in the application bundle having the same name as the application itself (e.g. applications/myapp/myapp/file.jpg). This is mapped to all domains on a URL with the same name (e.g. /myapp/file.jpg). You may thus use relative references instead of absolute for your HTML in your application's root folder.

The generic server not-found and error pages are provided by the Generic application, see its documentation to customise them.

Applications are for installation by a server operator only, and should be audited for security.

Static resources

Moonstalk configures the webserver to serve any file accessed with a filename extension on a site's primary domain, straight from disk (or RAM if cached) and without processing by Moonstalk's Scribe backend.

If your visitors primarily come from more than a single country (or state) the use of a Content Delivery Network (CDN) is highly recommend for improved performance in preference to hosting your assets from your server itself.

Assets

These are files such as images, CSS, and Javascript that are referenced in and enhance HTML pages, and also PDFs, video and other downloadable files.

Typically you would group all your assets together inside an assets subfolder of your site. When you link to them in HTML you simply use the relative src or href "assets/myfile.type" and the assets will work both on and offline.

Pages

Moonstalk's primary purpose is serving content that is at least partly dynamic, however you may serve static pages (i.e. complete HTML files) in the same manner as other static assets.

Dynamic resources

Any requests for an address not containing a filename extension is considered a dynamic request, to be processed by Moonstalk's Scribe backend, which runs a sequence of functions to handle the request and provide a response.

Both views and controllers are transformed into functions that run with access to the same Lua environment and Moonstalk functions—the difference lies only in the file format (tags in views, code in controllers). See the develop page for details of the environment.

See the Content application for details of how static pages can be stored in a database.

Views

These are simply .html files, either complete HTML pages or fragments of formatted content for display, such as within a template. Views may utilise the following two forms of dynamic markup.

Expression tags

?(expression)

The expression tag is a macro for the write() function and is replaced in the page by the resulting value of a Lua expression.

An expression may be a variable name, function call (but only if returning a single parameter), concatenation operation, or a combination of these (grouped inside brackets). Code blocks such as if and while cannot be used. Expression tags may be used anywhere in your HTML file (including inside attributes).

Processing tags

<?code?>

These may contain any valid Lua code including blocks, and are stripped out of the page, except for the result of any calls to the write() function which may be used to append output to the page. Code may be split across multiple tags such as to define conditional output.

<?if mytest then?><b>mytest is true</b><?end?>

Controllers

These are predominantly used to separate and protect the logic (Lua code) from the design layout in a view, thus avoiding placing complex server processing tags within a view's markup.

When paired with a view, a controller's code is placed in a .lua file of the same name. I.e. myfile.html is the view and myfile.lua is the controller. Controllers run before views and are used to process and prepare variables (such as from the request, or a database) for insertion as strings into a page, using expression or processing tags in the corresponding view.

Controllers may provide output before a view is run, using write[[<p>content</p>]] (typically using a Lua 'long-string' to avoid issues with escaping), and may thus also be used in place of a view entirely, particularly where a non-HTML response is required (e.g. a JSON or XML API).

Templates

With templates you can define the common layout or features of all pages, or groups of pages, such as providing a navigation menu, header, footer and so forth. A template wraps the content or layout (e.g. from a view) with its additional markup.

Templates are identical to dynamic pages, having both a view and an optional controller, but are run after the page controller and view. Templates may thus have both layout and logic.

In a template you must specify where to insert the page content generated by the page controller and view, using the following command.

<? scribe.Insert "content" ?>

Templates may be defined in the following manner, the latter methods take precedence over any prior.

Default
Create a template.html file in a bundle. This template will apply to all addresses that do not otherwise specify a template.

Per-address
Add a template="name" attribute to an address.

Ad-hoc
Set page.template="name" from a controller or view.

Sections

By default all output is placed into a section container named content, from which the page response is generated. You may define your own output sections, to capture output to insert into the page later in an arbitrary order. Use the following command to define a section.

<? section "name" ?>

All output following this will be captured, until another section is defined (or the page is generated). Templates use this same mechanism.

Once output is collected you must concatenate any custom sections into the page content. To combine your sections with the page, use the scribe.Insert command with the name of your previously defined section.

<? scribe.Insert "name" ?>

Chaining

You may include or run any view or controller in any other, by using the following commands, however bear in mind that the output response will be affected by the order that they run unless you use sections.

scribe.Page "name"

This runs both controller and view or whichever is present.

All application view and controller names are prefixed with the application name, e.g. appname/viewname. The current site's is accessed with just its name.

Chaining behaviour is akin to forwarding in J2EE, and should not be confused with HTTP redirection.

Redirection

To cause the user's browser to send a new request for a specified address use the following command, the redirect will occur after the current request has finished unless explicitly abandoned.

scribe.Redirect "address"

This is often used after successfully processing a submitted form to change the URL to a bookmarkable action ID, instead of simply displaying a different view at an unchanged address. The given address may be a path or an absolute URL.

Routing

To provide a response, a request's URL path must be mapped to a function (typically a controller) or content (typically a view). Moonstalk provides built-in mapping through its addressing mechanism, and ad-hoc routing through the chaining mechanism.

By default views in site bundles (but not applications) are automatically mapped to an address corresponding to their file names (without extension). E.g. the file sites/example.com/mypage.html would be accessed as http://example.com/mypage.

You may define your own mappings, by declaring an addresses array in a bundle's settings file. Each address in the array takes the following form, specifying one selector, one or more handlers, with optional attributes inherited by the page. (See addresses on the develop page for full usage.)

{selector="path", handler="name", attribute=value}

Authentication

Moonstalk has built-in generic handling for user access control. You may employ this either on addresses, or within your own functions (see develop). Addresses and pages may be defined as having restricted access using locks, and users may be granted access to such locked pages using keys. Locks and keys are simply names that denote membership of certain user 'groups' as desired.

An additional application is required to define behaviours for user and key management (i.e. assignment, retraction). Moonstalk includes the people application for managing users within the Teller database. No support is provided for hard-coded lists such as from .htaccess files, but would be easily implemented yourself.

To restrict a view to a particular key holder, add a locks attribute to the page, or to desired addresses in the settings.

{ matches="private/page", view="private-page", locks={ "Private", } }

A user requiring access to that address must then be granted a key with the corresponding name in their keychain.

By default all signed-in users have a default "Guest" key, you may therefore require sign-in on pages by adding a lock of the same name.

Internationalisation

Moonstalk includes built-in support for translated vocabularies (including handling plurals, and functions such as for ordinals), easily used via a simple key-prefix on localised names. Support for locales is also built-in and may be selectively used via formatter functions. Additional built-in behaviours such as Latin transliteration, and a report for untranslated terms, facilitate localisation improving both visitor and developer experience with projects requiring ‘i18n’.

When serving localised and translated pages, a user's preferred or profile-derived locale and language is used wherever possible, followed by the site's specified default locale and language, with fallbacks to the first or only language defined (such as if translation is incomplete or only has a single language defined).

If enabled GeoIP lookups are made using an in-memory dataset to supplement an unprofiled user's locale, and completely avoiding the additional latency of third-party API calls.

Address paths are normalised using case-insensitivity, and transliteration of latin accented characters to ASCII. Pages may be addressed with or without such characters, whilst the original (i.e. with accents if your file or page names contain them) is preserved as the canonical name (and address).

You may use any desired encoding for your HTML and strings in Lua (but UTF-8 is recommended as this used by all Moonstalk's bundled applications), simply be sure to use the same encoding throughout, and ensure that the corresponding encoding is declared in your HTML.

Lua does not handle Unicode sorting, and you may need to use your own or third-party routines for this if needed.

Non-ASCII (e.g. UTF-8) encoded files must be saved without a BOM, otherwise they will fail to load. See the utilities section for some editors that won't muck this up, if yours does.

Views

Views saved as files may have multiple versions created to provide separate content for different target cultures (translations/localisations), simply by using the following naming convention for the files (of any supported view format).

view.culture.html
The view name must be the same for all the shared versions of the same content file. Culture can be any of a language identifier (e.g. fr), a locale identifier (e.g. ca), or a culture pair identifier (e.g. fr-ca).

Views that are not provided in this manner are assumed to either use the site's default culture, or to be dynamically translated and/or localised using vocabularies and formatters.

Views generated from other sources may have cultures specified directly, see the Content app and view table for details.

Locales

These define the formatting of numbers, currency, dates and so forth, with country/language variations. Moonstalk will always attempt to identify and honour a user's locale and language (see vocabularies below).

Values are displayed respecting a locale if you output them using Moonstalk's format functions.

Several locales are included with Moonstalk, whilst additions or corrections may be submitted to us. You may override them in the global environment from a settings file.

Vocabularies

These enable localisation, and store the translated text values (strings) for each translated language. If you don't expect to need to use localised content you may ignore this functionality and it will not affect the display of pages.

You may set localise=false for a site, address or page to completely disable all locations functionality for corresponding requests.

Applications, sites and pages may specify their own individual vocabularies (translations) that supplement each other.

Pages are displayed using the language matching a user's locale or request when available, otherwise it will be substituted with a regionally acceptable language, the site or application default, or finally English, when these are available in the vocabularies.

Partial translations are undesirable, as missing translation are replaced with alternate translations according to the user's locale, which results in pages being displayed in a mix of languages, therefore it is important to ensure all applications, sites, and pages support the desired languages.

In developer mode missing translations are not substituted with alternates, but are instead displayed as ⚑name to assist in identifying missing translations, whilst the /Manager/Localisation view provides a complete report of untranslated terms.

Individual translated words or phrases are represented by keys (untranslated names) for which one is defined for each language (using its language code as defined in globals/Attributes.lua) in the bundle settings file.

In settings files vocabulary is typically abbreviated as v to simplify typing, by defining local v = vocabulary.

Vocabulary item names (keys) should be globally unique (amongst all applications) else will be replaced. Per-site and per-page vocabularies may be used to supplement or customise these definitions.

You may replace a key in any application scope from anywhere, however you may only replace a key for the site in a page, and you cannot replace a key defined in a page as it is the final scope.

Translated words from the active vocabulary (the current user's language) are accessed using a variable named correspondingly but prefixed l. (for the original word) or L. (reformats the word with an initial cap).

when user.language == "fr"
l.aword → un mot
L.aword → Un mot

In views the translated value's variable may be used in a placeholder as with any other variable.

?(l.aword)

Plural forms for translated words are also supported, these are defined in the same manner as words above, but with the number of plural forms corresponding to the language per the Mozilla rules (all of which are supported) e.g. 2 forms for Germanic languages (defined as an array of two strings), or 1 form for Asian languages (defined as a single string not an array).

en.items = {"item","items"}

Once defined you can access the translated plural by calling the localised vocabulary function with the identifier and a number that specifies the correct plural form to be used.

l("items", number)

Additional functions may also be localised and assigned to the vocabulary and called simply using the

l.

prefix. Some are included such as the

l.Ordinal(number)

function that will return an appropriate suffix for an ordinal number.

Manage

Each individual server (virtual or dedicated) on which Moonstalk runs is called a node, and comprises the components inside the install folder. No other location is used for configuration or temporary files, withstanding the startup script.

Runner

This shell utility is your interface for managing a Moonstalk node. Run without any command it will provide you with its status. It will always report any application or site compilation (syntax) errors. All instructions are optional, accepted in any order and with several synonyms. For further usage instructions issue the help command.

cd /usr/local/moonstalk

./runner

The server control commands are:

start, stop, restart, kill

These control commands may be accompanied by a server name:

all, web, pages, db

To ./runner start web you must use super-user priveleges (sudo or login as root).

Configuration

The node configuration is stored in the data/configuration/Node.lua file and contains node-specific options for both Moonstalk and other applications. You may edit this directly, or for node attributes, you may use the runner to change it.

./runner attribute=value

In table values use shell escaping of commas and spaces, and escaped quotes for strings.

For production (non-development) servers, set a unique node ID that is between 101 and 999.

./runner id=101

For development computers, you can enable development mode.

./runner logging=4

You may also need to set a FQDN if the default hostname is not adequate.

Manager

Moonstalk includes a web administration application called the Manager. This is enabled on the localhost site, which is also accessible on the server's default hostname. The Manager's user database is created with a default operator user, having a unique password that is the secret from the node configuration. The runner will notify you of these upon first use.

Updating

Moonstalk can be updated to the current release version with a single command on a non-development server (id~=100).

./runner update

Your sites, applications and data will be untouched by this command, but be aware that updates to applications may require adjustments to your code or settings, therefore it is advisable to check the release notes before updating.

During the pre-release development phase please review the changelog for changes that may affect you. Subscribe by RSS, open your ~/moonstalk directory [repository] with a Mercurial GUI app after updating, or use your shell:
cd ~/moonstalk; hg log | grep summary | more

Repositories

You may create your own repositories using your preferred CVS for each of your application and site folders, or one for the entire sites folder—but not the entire applications folder, unless you are not using Mercurial and exclude moonstalk.* from being tracked.

If you use Mercurial for your site and application repositories, Moonstalk's update command can also pull from them for you. Simply push your changes before attempting an update, and add your repository folder names to the node configuration.

repositories = { "applications/example", "sites/example.com" }

Forking

As the moonstalk directory is a version-managed Mercurial repository, if you change any of Moonstalk's managed files, the changes will be lost when Moonstalk is updated.

You may however fork/clone Moonstalk's repository, and change Moonstalk's default respoitory in .hg/hgrc to your own forked repository. Once an initial installation pull has been made from your respository on each of your deployment servers, the Moonstalk update command may be used to pull from your repoisitory thereafter.

Develop…

Now that you've got a hang of the fundamentals, read the develop page to learn about data tables, configuration, functions and more that will enable you to develop your own solutions as desired.

Addendum

Questions

Why use the PHP tag<? ?>instead of ASP/ROR<% %>or something else?On most keyboards the 'PHP-style' tag is easier to type, but more importantly it is XML/XSLT compliant. You may also use the full language-declared <?lua ?> syntax.

How can I add a hidden comment in a view?Use a Lua long-comment inside a server processing directive <? --[[ your comment here --]] ?>

What's wibble, wobble, wubble?Foo, bar, baz in British-English. These are meta-syntactic variables to be replaced with something that makes more sense in context.

Useful libraries

Moonfall
«A program to generate CSS, either dynamically as a cgi script, or run from the command line. Used in its simplest way- you get variables in css. Used in its most advanced way- you can write a powerful Lua script to create and modify CSS.»

The nity-gritty (develop…)

This manual is a pre-release preview. Please follow Moonstalk on Twitter or Facebook to be notified of a public release.

Tables

These data-structures provide access to the Moonstalk environment, the request, and the response. You may check and change the values in these tables to determine how to handle a request—and to generate a page for response.

All tables exist in either a global or ephemeral (per-request) scope. The latter are discarded after each request is completed and may be modified freely, whilst the prior exist across all requests and should not be modified. Additional reference (pointer) tables are provided in the ephemeral scope for convenient access to some global tables.

Global tables should not be changed during a request, and are intended to only be modified by application load-time functions. Failure to adhere to this will result in random results when using more than a single backend instance. Whilst reference tables change with each request, they are dynamic pointers, or aliases, to the contents of global tables and therefore should also not be changed. No tables are persisted (use a database such as the Teller).

request.

This predominantly contains details of the HTTP headers, with parsing into a more Lua-friendly format where appropriate. Use of this table is discouraged as its format and structure is dependant upon the web server API. See form and page instead.

Ephemeral table

time

cputime

method

url

secure

protocol

domain

path = "/My/Account"

query

browser
HTTP User-Agent header.

client
Originating IP address. Use of user.ip is advised instead.

get = { name="value", }
Present only if method="GET". Use of form is advised instead.

post = { name="value", post_data="data", }
Present only if method="POST". Use of form is advised instead.

referrer

cookies

language

languages

locales

form.

Contains merged values from both request.get and request.post (latter has precedence), but is always present regardless of method.

Ephemeral table

name = "value" or {name="name", size=bytes, ['content-type']="mime/type", contents="data"}
For every input sent as either a GET or POST. If the value is a file the value is an upload table.

uploads = { {name="name", size=bytes, ['content-type']="mime/type", content="data"}, }If a multi-part POST this will contain an array of the parts, this is primarily useful for multi-file uploads in particular using an HTML5 file input with the multiple attribute.

Table (global)

scribe. (global)

Functions

Content

write

Appends the passed value to the current output section. Values may only be strings, numbers or nil (which is ignored).

A table or function value will invoke an error as they cannot be respresented as text in a response, for debugging you may however output them with tostring().

In views, do not call write from a ?()expression tag, instead call it from a <? ?>processing tag.

l.

key

Returns the given language key's value from a translated vocabulary, in its original case. Use L.key for an initial capital.

This is a proxy table—a function with table.key syntax. (Achieved using a metatable __call overload.)

macro

{ [[text]], key="value", }

Replace defined ?(key) macros in the specified text, with their correspondingly specified values.

Whilst macros in views (expression tags) accept expressions (thus, function calls) for replacement, text macros do not, and where no corresponding key is specified a macro is not replaced.

Text macros are typically used for customising strings, such as translated language keys from vocabularies, where pre-defined grammar is customised using embedded variables.

Page

scribe.Section

"name"

Changes the current output to a new container, as named. (The default page response output is named content.) See also Cut.

scribe.Mark

"name"

Marks the current position in a view to later Insert a section or other value.

scribe.Insert

"section_name"

"mark_name"

Inserts a previously defined section into the current view/section. Takes an optional mark_name parameter to insert at the specified Mark instead of the current location. See also Paste.

scribe.Page

"name"

Run a specified controller and/or associated view and append it's output to the current page.

scribe.View

"name"

Run a specified view only and append it's output to the page. If the view provides versions for different cultures a matching translation/localisation will be served.

scribe.Controller

"name"

Run a specified controller only.

scribe.Cookie

("name", value)

{name="", value="", expires=, httpOnly=true, path="", domain=""}

Sets a cookie in the user's browser.

Abandonment

The following functions abandon page processing in some way, to display an alternate view or response.

The current function from which these functions are called, and any chaining from it, will continue to run but their output will be surpressed. No further functions will run except for the default template (if any). To prevent the current function from continuing, follow the function call with a return termination.

scribe.Redirect

"address"

Abandons the page processing to instead returns a 302 redirect response to the user's browser. The address may be a path or URL.

scribe.Authorised

{ "name", }

Checks that the current user has one of the specified keys, otherwise abandons the page processing. If not signed-in the generic/signin view is displayed, or if signed-in but without a specified key, the generic/unauthorised view is displayed.

For a conditional termination use if not scribe.Authorised{"name"} then return end. For conditional content check user.keychain directly.

scribe.Error

"synopsis"

{title="synopsis", detail="explanation", realm="page"}

Abandons the page processing with the generic/error view using the specified details.

scribe.NotFound

()

("name")

Abandons the page processing with the generic/not-found view. Accepts an optional user-friendly name to be displayed instead of the page.address.

Utility

The following functions abandon page processing in some way, to display an alternate view or response.

Lua Modules

All standard Lua libraries are available, with the following, and some other less commonly used libraries that are documented on the credits page.

You may use load other libraries (.lua or .so) saved in the same folder as a view or controller, or within an application folder by using the full path.

require "libraryfile"

require "appname/libraryfile"

Moonstalk uses its own package.path and cpath, therefore if you wish to use other libraries elsewhere on your system (such as from LuaRocks), you must append those paths to Moonstalk's values. Applications may do this in the settings or functions files.

package.path = package.path .. ";/my/lib/dir/?.lua"

Components

Web server

Moonstalk uses the data/configuration/lighttpd.conf as its base configuration, with which its own configuration is combined at launch-time, and then saved to the temporary/lighttpd.conf file to launch Lighttpd.

Behaviours

No address may end in .lua.

Large uploads are rejected if no token cookie is set.

All addresses on any domain, starting application.id/ serve the corresponding static file (if any) from an application's assets folder.

Any other address is handled by the Moonstalk pages backend unless an additional application specific configuration applies:

For folder sites, on the primary domain (not tenant sites) any address (except as above) containing a period but not having a query string, serves the corresponding static file (if any) directly from disk without any processing by Moonstalk. This includes the source of .html views. If a static file needs to be served from the primary domain with a query string it may be placed in an application, or the web server configuration supplemented.

Bundles

Behaviours

Views

If the view defined in an address contains no dynamic markup, then the page.modified value (and thus Last-modified header) is automatically set using the last modification date of the corresponding view's file.false

If a default template exists it is applied to all addresses, to prevent this explicitly set an address template to or the name of anothet template.

If a view contains an <html> tag, page.template is not set to the default template. If you wish to run a template controler with such a view, specify the template using an address, or collator().

Other files

If present any views and controllers placed inside a subfolder named with the application.id of an application, will override its views and controllers with those having the same name in this folder.

File and folder names starting with _ (underscore) in the site root (but not in subfolders) are ignored by Moonstalk, but may still be accessed as static files via the webserver directly. You may thus use this naming to temporarily prevent views and controllers from loading.

File or folders whose names start with . (period) are both not loaded and are inaccessible from the web server. This allows a site folder to be defined as a version-controlled project where the versioning data is in a hidden folder (i.e. Mercurial's .hg/ folder).

Table

name

id

views

controllers

Applications

A bundle whose table exists in the global environment, and having handlers that run at launch-time.

Functions

Initialisation

Loader ()
Run once for each application once it has loaded (before sites are available).

Enabler ()
Run for each site and each application, after all applications and sites have been loaded. The site table in the global environment reflects each call's site. Provides an opportunity to modify sites.

Starter ()
Run once for each application after a database connection has been established by the server. Provides an opportunity to modify the global environment, such as to modify or replace another application's settings, views, functions, etc. Be aware that such usage affects every site.

Reader (data,format)
Run when a view is requested for the first time (with every request during development). Allows changes to the raw file data from disk before being served. Should always return data.

Sites ()
Invoked on startup to populate the node.sites table with sites. This can be used to define permanent sites (i.e. with guaranteed availability) that are constructed or stored in any manner, such as on disk (as in the Sites Folder application). Not to be used for sites that are ephemeral (use the curator function).

readyCurrently application initialisation functions are run regardless of dependancies or the success of preceding functions, this key provides a boolean that allows you to inspect the state of any application's initialisation and exit when required applications are not ready.

Requests

Curator ()
Invoked for each request where no matching site was found in node.sites. This provides an opportunity to define sites and behaviours that may vary with every request, such as ephemeral sites stored in a database but that may be arbitrarily deleted (as in the Tenants application).
The curators of all installed applications are called in succession until one provides a site. The Generic application provides a curator that runs last, to handle the unknown domain condition. When using multiple applications with curators (e.g. both Tenant and Sites Folder) the curator order should be specified with node.curators.

Collator ()
Default collator for an application's own addresses. Invoked per-request after addressing, localisation and authentication have occured, but before controllers and views. Provides an opportunity to perform content retrieval or review that is not specific to a view or controller.
As the same function is used upon many addresses, it should include conditional handling as necessary.

Editor ()
Called after the response has been set and concatenated, but before it is returned. Enables modification of the raw output response value.

Behaviours

View and controller names specified in addresses are assumed to be in the current application, unless prefixed with another's application.id and a slash e.g. appname/name.
If prefixed as ~/name the view or controller is assumed to exist in the current site only.

Bundle names define the application.id and application.name. Any part of the name before a . (period) is ignored.

When present, a bundle subfolder named application.id/ is used to serve assets via the web server, under the path application.id/ on all domains.

Table

Performance

Enabling applications has no per-request performance overhead except where the following are defined by the application, where the overhead will be specific to the functionality provided. Memory overhead for applications with multiple sites is minimal (sites are populated only with references to the application).

curator

pattern-matched addresses (negligable)

collator

editor

Sites

Sites are defined by the sites() function of applications. See the Sites Folder or Tenants applications, or any other application with a sites function, for details on how sites may be defined and configured outside the Scribe environment.

Within the Scribe environment, sites are stored in the node.sites table, with references from the global domains table (used to lookup a site for each request) and ephemeral (per-request) site table.

Behaviours

A canonical metatag using the primary domain is inserted into any HTML page served on a non-primary domain (i.e. when redirect=false). This cannot be disabled without hacking as it is considered a best practice.

Site assets are currently served only from a site's primary domain. If you use multiple domains without redirection, you must currently use absolute URLs for all your asset references.

Table

id = "example.org"

name = "My Site"
This is displayed as the title of a site when using the generic template, if not specified request.domain is used.

domain = "example.org"
By default this is your primary domain and is the domain used for redirection, and canonical links, etc. Manually specifying it is not recommended.

domains = { ['example.org']={} }Alternate domains that serve the same content as the primary domain (included in this table). Each of these domains is added to the domains table as a reference to the site.

redirects = { ['www.example.org']=true, }
Domains that redirect to the primary domain. Any domain specified here will never receive requests, unless intercepted by a curator function.

language = "en"
If you have specified multiple vocabularies or language-specific content, you should specify your site's default language for cases where a visitor has no preferred language, and to ensure addresses for language specific content can be indexed by search engines.

applications = { [application.id]=application, }
If you wish for applications to be enabled in this site, you must specify a list of their names. Order is unimportant unless more than one application attempts to define the same resource (i.e. address) in which case the first specified will be used.

Views

Features

Expression tags
Markup that functions as an in-line macro for the write() function.

Processing tags

Behaviours

White space such as line-breaks inside expression tags is ignored, unless within a defined string.

The scope of a local declared in a view is limited to the current processing tag only, and will be inaccessible from subsequent tags or blocks. Use the page table for disposable values that are used across blocks.

In folder sites (but not for applications or tenant sites) the source code for views (not controllers) is accessible if it's filename is known. It may therefore be undesirable to place processing tags inside .html files as the unprocessed source code for these files, by default, is accessible to anyone simply by changing the address to include the .html extension (assuming the filename is the same as the address). It is desirable in any case to place such directives separately in a controller. You may also customise the web server configuration to disable this mapping.

The modification date of a view mapped by an address, is used for the Last-Modified HTTP header when serving the response. This enables search engines to keep track of your site's changes to reindex pages, and improve their relevancy in search results. If your view displays content that changes in a database you should specify an appropriate value for response.headers["Last-Modified"] in your controller or view accordingly.

Local functions defined within a view may not contain output if also contained within a section (define the function elsewhere).

Functions

Table

Controllers

Behaviours

.lua files that start with #!/usr/bin/env lua (for a CLI command) or contain a line starting module (for a library) are ignored and not loaded as controllers.

White space such as line-breaks inside expression tags is ignored, unless within a defined string.

Features

Addresses

Features

Linked views and controllers
Any address defining a view or controller will also have a corresponding controller or view defined, if one exists with the same name.

Normalised path matching
Freely use proper capitalisation and accents in your URLs—paths are both case-insensitive and transliterated for latin accented characters.

Addresses may not contain the '.' (period) character, as this denotes static content to be served directly by the web server.

For file names that are automatically mapped to addresses (the default) the source code of views is publicly discoverable. (Logic should be placed in controllers, rather than views; if you have to place sensitive logic in a view you may manually map their addresses to obfuscated file names and set autoaddresses=false.)

Table

Selectors

Defines how to match a request. Only one may be used.

matches = "path" or { "path1", "path2", }

starts = "prefix/"

ends = "/suffix"

contains = "value"

pattern = "value"

Handlers

Defines what to do with the request. One or more may be specified per address. You need only specify both controller and view if they are different.

view = "name"

May be set to false to prevent a view from being displayed.

controller = "name"May be set to false to prevent a controller from running.

template = "name"
May be set to to prevent the default template (if any) being used. The default template is never used for complete HTML files.false

collators = {"name"}The named or referenced functions will be run in the specified order, and may set or change the controller and view or perform other functions prior to their execution. Names may reference functions in the current application or site, or other applications. If specified the default site or application's collator function will not be added. May be set to false or an empty table to prevent inheritance of a default collator function.

redirect = "address"Must be used exclusively (without any other handler). May be a URL or address.

locks = { "key", }
May specify an array of keys, if none of these matches user.keychain then the user is denied access or (if signed out) asked to signin.

form = {"…"}
Set default values for the form table (emulates user submitted input).

action = "name"
The name of an action function to run (emulates user submitted input).

Attributes

Addresses may specify any of the attributes of a page, such as template or title.

User

Represents the anonymous visitor that made the request, a profiled visitor (cookie-tracked but anonymous), or a registered user. Managed by the built-in authentication mechanism but may be extended by applications.

Features

Locale and languages
Provides access to

GeoIP
For upplements the typically unreliable browser-based locale with a network-location dervived locale. Ideally enabled only when most visitors are likely to originate from multiple countries and localised value formatting or translation is required.

Behaviours

Table

ip

token
The encrypted value of the token cookie, if signed-in or tracked.

id
A unique ID number for the user, if signed-in or tracked.

languages
An array of the user's preferred locales, derived from the request and user's settings (if any).

locales
An array of the user's preferred locales, derived from the request and user's settings (if any).

keychain

Extend…

To learn how you can extend moonstalk, and make use of pre-packaged functionality, read the applications page, or to understand the Moonstalk's design jump to the architecture page.

Extend (applications…)

Generic

Provides default generic behaviours, including authentication and session handling. Unlike other applications, it is always loaded for all sites.

Views

generic/not-found.html
The 404 error page, shown when any user-requested page cannot be found. The not-found view must specify page.error=false to prevent fallback to the error view.

generic/unknown.html
Shown when a domain does not match any site. Displays the node.provider.name and

generic/error.html
Shown when any internal error occurs, such as an error in the code syntax of a view, or a database error.

node.provider.domain if defined.

generic/signin.html
Shown when users who are not signed-in, attempt to access a locked (restricted) page without permission.

generic/unauthorised.html
Shown when users who are signed-in, attempt to access a locked page without permission.

generic/account.html
Default page for signed-in users.

generic/verify.html
Shown after a login reminder is requested.

Behaviours

Creates a localhost site on which it enables itself.

If used to authenticate a user with one of its authentication functions, will add the user.sessions table, maintain user.last with the timestamp of their last session, and also populate client.session.

Functionality

action.Signin ()
Accepts a form with email (containing an valid identification, including telephone or userid) and password, if valid, establishes a user session, otherwise displays the generic/signin page. Also provided at the /Signin address which accepts an email or id parameter as a default value.

generic.Password ("password")
Returns a secure, salted and hashed version of the password string for storage in a user record. This is required for use with the Signin method. Note that the salt used is node.secret therefore if this ever changes, all user passwords will be invalidated.

user.sessionuser.session.visits=count
user.keychain.Guest
These are defined
for any sucessfully signed-in user

generic.Session (token)
Creates a persistent cookie-based sign-in session for the current user using the specified token, the current user.token, or assigns a new one. See people.Adopt and people.Session for assigning sessions to other users.

generic.Signout ()
Terminates the user session.
Also provided at the /Signout address.

generic.Email { to="recipient@example.com", cc="recipient@example.com", bcc="recipient@example.com", subject="Test", body="content", server="smtp.example.com", username="login", password="secret", from="sender@example.com", headers={} }
Sends an email. Defaults for all values except to, cc & bcc, may be specified as per settings below. Recipient addresses may be specified as arrays of addresses, and each address may be specified using the complete [["Name" <address>]] format. Note that this function blocks the scribe from processing other requests whilst communicating with the SMTP server, and should therefore ideally be run as a task, else response to the user will be delayed.

Delegates

NewUserSessionDelegate (newtoken)
Called after a new session token is assigned.

DeleteUserSessionDelegate (existingtoken)
Called before a session token is removed.

Settings

smtp = { server="smtp.example.com", username="login", password="secret", from="sender@example.com", headers={} }These settings for the Email function may be specified in the message, site settings, or Node settings, where a prior is used in place of a latter.

Sites Folder

Loads bundles in the sites folder, as sites.

Behaviours

The bundle name defines site.domain (the primary domain).

If the primary domain is not prefixed with www an additional domain with the prefix is added, or if prefixed, a domain is added without it. Thus both are always handled.

Views are automatically mapped to addresses using their filename (without extension), see settings below to disable.

Settings

Any attributes of a site bundle may be specified in the site's settings.lua file, however the following are specific to the settings file, providing simplified syntax. All attributes and their values defined in the site settings file (including your own) are accessible in the site table with each request.

applications = { "manager", }
If you wish for applications to be enabled in this site, you must specify a list of their names. Order is unimportant unless more than one application attempts to define the same resource (i.e. address) in which case the first specified will be used. Dependancies will be resolved and those applications also enabled.

autoaddresses = true
Addresses are automatically added for your views, if you wish to specify addresses manually you must set this to false.

redirect = true
The default behaviour with multiple domains is to redirect requests for alternate domains (as listed in domains, or for the 'www' variant of your primary domain if none) — to your primary domain. I.e. if your primary domain is example.com then www.example.com will be redirected to that, or vice-versa. If you specify false, domain redirection is disabled and all domains function as aliases for the same site content. A canonical link is also added to pages on any non-primary domain, referencing your primary domain.

Kit

Provides helper functionality for working with HTML pages, loading and running JavaScript, and facilitating the use of third-party integrations such as Google Analytics.

Kit utilises an editor() and thus its page modifcation functions are retroactive, and may be called at any time (e.g. before any view has even provided page content to modify).

Behaviours

The javascript-dominoes library (5k) is loaded with every page to allow asynchronous loading of any additional script files.

A kit and vocabulary library is loaded with every page.

A language metatag is added with the value of page.language (if any).

Inserts site.css and page.css into every request.

In views, all src attributes are made absolute. The root may be specified (such as to use a CDN) with node.base, or this may be set to false to disable this behaviour entirely. This feature enables you to develop views locally with relative asset references, and they will be changed automatically upon deployment.

Functionality

Javascript

page.javascript.varName = value
Assign a value to a variable in the client-side JavaScript environment.

kit.Script ( script )
Asynchronously load and run a script. The script value may be either a path to an external script file, e.g. script.Load "/assets/script.js", an assignment e.g. script.Load "myvar = 'foo'", or a function to be called after loading e.g. script.Load "myfunc()". You may delcare dependencies to be loaded simultaneously as supplementary parameters, e.g. script.Load ("/moonstalk.kit/jquery.js", "/assets/script.js", "myfunc()") or sequential dependencies as a single string using greater than as a seperator, e.g. script.Load ("/moonstalk.kit/jquery.js>/assets/script.js", "myfunc()"). Expressions should not be terminated with semi-colon.

Page

nocache = true
Adds an HTTP cache-control header with the value no-cache. May be specified in address or page.

robots = "none"
Adds an HTML robots meta tag with the specified value. May be specified in address or page.

Assets

Kit includes a number of useful JavaScript libraries.

kit.Script "/moonstalk.kit/jquery.js"

kit.Script "/moonstalk.kit/jquery-ui.js"

kit.Script "/moonstalk.kit/jquery-hoverIntent.js"

kit.Script "/moonstalk.kit/jquery-format.js"

Forms

For any key in the page.flags table corresponding to a form input name created using a tag function, it's class is set to error, and if the value of the key is a string an error message is displayed in a span with the iderror. The tag functions should be used in server tags not expression tags.

tag.select ( "name", list, selected, localised )
Creates an HTML formselect input with the specified name, and an option corresponding to each item in the list—having integer values and display names from the list item value, either as a localised string name, or (if localised is false) the value itself. Specify selected using or syntax to provide a default e.g. form.name or 2.

tag.checkbox ( "name",selected )
Creates an HTML formcheckbox input with the specified name, and checked as per selected, which may be specified using or syntax to provide a default e.g. form.name or true.

tag.input ( "name", value ) or tag.text { name="name", value=value, attribute="value", ... }
Creates an HTML formtext input with the specified name and value, or the attributes and their values specified in a table parameter. Specify value using or syntax to provide a default e.g. form.name or "value".
If no value parameter is specified it will default to form[name]

page.focusfield = "name"
Sets the browser focus to the form input field with the specified name.

tag.Error ("name", "message")
Sets the browser focus to the form input field with the specified name, but preserving any prior focus, and assigns an optional message to be displayed for the corresponding input created with one of the above tag functions.

Formatting

format.Date ( datetime, format, toggle )
Format a date using a current locale's named date format, or an arbitrary format string. The date may be provided as a date table (e.g. from localtime), or as a reftime to be converted to the site's localtime. Named formats are "numeric", "long", "short", "abbreviated", and "mini", however not all locales differentiate these. Except for "numeric" (the default if unspecified), dates will be returned as semi-relative dates (weekday and year added/omitted according to age) unless the toggle parameter specifies one of "complete", "year" or "weekday". The associated format.ReferenceDate may be used to output a reftime without conversion to site localtime.

format.Time ( datetime, format )
Format a time using the current locale's time format, datetime is handled in the same was as in format.Date. Similarly the associated format.ReferenceTime may be used to output a reftime without conversion.

tag.time ( datetime )
Outputs an HTML 5 time tag that will be refreshed client-side with a relative local date (e.g. "just now", "1 week ago", "13th Sept."). This is recommended as it provides a reliable and continuous adjustment using the browser's current timezone.

format.Number ( number, decimals, locale )
Format a number or 0, as HTML with the specified decimal places or 2, using the specified locale or user.locale.

format.Money ( number, decimals, locale )
Format a monetary value as HTML using the specified locale or user.locale. If no number is provided, displays n/a.

format.TelPrefix ( number )
Adds an international prefix or localises the prefix of an internationally-formatted telephone number (with or without punctuation) to the user.locale.

CAPTCHA

captcha.Generate ()
Renders a simple math-based captcha question into a form. Includes the text and input field. Should be used in a server tag.

captcha.Validate ()
Call this in your form validation, if there is an error the form behaviours above will apply.

Geo

The geo application provides geographic reference and manipulation functionality, including RAM cached (<1ms) GeoIP user location lookups, using free country and city level databases. In future this application will also provide GeoNames data.

Configuration

Node

Providing the data files are available, the application is enabled by default to provide a country for the user locale in cases where a request language is not country-specific. To disable specify geo=false in the Node settings, to force lookups for all requests, specify geo=true or geo={lookup=true}.

You may enable country-level resolution for all requests by specifying geo={countries={'cc'}}, and/or city-level resolution by specifying geo={cities={'cc'}} in the same manner, an empty table will load all available data. At the time of writing per-scribe RAM use is 10MB for country level data for ambigious languages, 40MB for UK city-level data, and 250MB RAM for the US.

Sites

geo = nil
Lookup for ambigious languages only.

geo = false
Never lookup.

geo = true
Always lookup.

Data

The following unpacked third-party files must be placed in the data/library/geoip folder.

Tables

Manager

The manager application provides an adminstration interface for the Moonstalk servers, extensible by other applications. Requires the User application.

Behaviours

Enables itself on the localhost site.node.secretoperator

Creates a default operator user if none exists, with a default password of the and a key named .

Access is possible by any registered user, allowing applications that extend the Manager to restrict access to their pages as appropriate. The Manager's own functions are restricted to users with the operator key.

Content

Enables the retrieval of localisable pages (content objects) from a database (the Teller), and extends the Manager application with functionality to manage these pages. Requires the Tenants application.

Page tables are composed of keys referred to as pieces, each having any valid value(s), including localised tables. A localised table is a table containing a key for each language identifier and a table or string value, but should also contain _localised=true. When a piece is requested corresponding to a localised table, only the value matching the user's preferred language or site language, is returned. If no language matched we also set _localised=false in the response. If the localised table does not contain a _localised flag the corresponding piece name should be specified as "name[localised]".

Configuration

For site folders only, to avoid disassociation of data in the case of a folder name change, it is desirable to provide an ID that is unique and permanent. Define one to the site settings.

id = guuid

Behaviours

page.pieces = {"piece"}
Specify the peices to be retrieved with a specific page. Must be specified before collators are run; from an address would also require collators={content.PageCollator} to be specified.

page.content_urn = "pattern"
Specify a match pattern that will be used by the content.PageCollator in place of page.address (the URN) to retrieve a corresponding page.

Scribe Functions

Pieces ("urn", {"piece"})
Retrieves the localised named pieces of a page having the given urn. The default pieces are {"meta", "vocabulary", "title", "body"}, however you may specify true to retrieve all (i.e. the same as the page table itself, but without any redundant languages). The following attributes are always returned: created, modified, controller, view, template, css.

Delegates

NewPageDelegate (page, page_id, site, site_id)

Tenant

This application virtualises sites, storing their settings and content in the Teller database and extends the Manager application, where settings may be managed.

Primarily intended to facilitate developing SaaS applications where users have individual domains, it also enables support for the Content application (or any other per-site/per-user content in a database) with disk-based sites.

Multi-tenanted SaaS

Tenant (virtual) sites have similar functionality as disk-based site folders. They can have per-site application settings and content (using the Content application) but they cannot define addresses (beyond those supported by the Content application), controllers, views, or databases. Nor may they use variables in content, or disable/enable applications.

All virtual sites use a foundation site (node.sites.tenants) with which a virtual site is merged, enabling a configured set of applications for every tenant site. All CoreAPI site attributes are valid within a virtual site and will replace those defined in the foundation site, and it is therefore necessary to ensure arbitrary attributes cannot be specified for virtual sites from any management interface exposed to a tenant.

Generic foundation

Specify the following in Node.lua to customise the generic/unknown view.

provider = { domain="example.com", name="Example SaaS", }

Custom foundation

Specify a site (loaded elsewhere, e.g. from disk using Sites Folder) to be used instead using tenant in Node.lua.

tenant = { applications = {}, template = "name", subdomain = "", }

Features

Front pages
Sites can specify their own front pages, either as views or redirects, and may randomise amongst several.

Caching
Employs very short-term virtual site caching (1-3 seconds) to optimise for heavy traffic, reducing database queries. Requests under load are thus typically fractionally faster than when under light load. This ensures the ability to respond to requests with fresh data is never compromised.

Behaviours

Page changes
As with sites, but the modification time is the time you edit the content of a page in the Manager. This cannot be manipulated.

CuratorThe tenant application's curator always returns a default site when a site doesn't match, it should therefore always be run last, after any other curators that may provide sites (e.g. the Sites Folder or Match Domains applications. The curator order may be set using node.curators={"sitesfolder","matchdomains","tenant"}.

Teller Events

event.sites (site, site_id)

event.domains (domain, site, site_id)

People

Provides storage of user records in the Teller database, with handling for multi-tenant user.keychain.

Functionality

people.DisplayName (user)Concatenates the first and lastname for output, either for the specified user, or the current user if none.

people.SplitName ("name")Returns firstname, lastname for a given user-input name string where the latter may be nil.

people.New {name=name, email="address", telephone="number", }Returns a new user object with ID. Accepts any valid keys from the user table, in addition to the above convience keys.

people.Adopt ()
Same as New but also updates the current user.

Teller Events

event.user (user, user_id)

event.email (email, user, user_id)

Match Domains

Adds support to sites for wildcard sub-domains (i.e. DNS CNAME) and pattern-matched domains. To enable add the following to settings.lua.

applications = {"matchdomains",}

To enable matches for a domain in the domains list of settings.lua, specify their names in one of the following manners. Matching is not supported on primary domains (site folder name).

Subdomain

Simply specify a period as the first character of the name.

domains = {".example.com",}

This example would catch all subdomains of example.com, but not example.com itself unless specified as the primary domain / site folder name.

Pattern

Notes

This application functions simply by parsing all site domains upon startup for a valid pattern, bulding an array of these domains, and providing a curator function which is utilised to return a corresponding site (if any) for each request.

The Tenants application already supports multi-tenant sub-domains for users via its database, however both applications may still be used together but node.curators should be specified such that matchdomains appears before tenant e.g. node.curators = {"matchdomains", "tenant"}.

Teller

Runs a new server process providing data persistence (storage) and queries (retrieval). For general usage see the tutorial.

This application supports and is required by many of the bundled Moonstalk applications. Use of an alternative database system with Teller-compatible applications would require an application implementing the Teller database interface functions.

Features

Transparency
Built-in support for connections to the Teller database process. No configuration or connection management is required. Additionally table keys may be specified using standard Lua table-name syntax.

HierarchicalRetrieve and assign not just specific key values, but entire Lua table structures.

Functions
Define and call Lua functions to run on the database process, as if they were inside your application. Selected attributes of the Scribe environment (e.g. user, site) are available to these functions.

Tables

To define a new database table (as opposed extend another application's tables, such as users or tenants), declare it in an application's settings file.

database = { "name", }

Teller database tables are global keys (names) and therefore a key for any database value will always begin with a database table name, and which cannot conflict with any other global name (i.e. application names, and Moonstalk tables or functions).

Scribe Functions

These are called in your pages (views, controllers) to access the database from Scribe backends.

save (database.key, new_value, original_value)
Stores the new value in the database for the specified table key (whose parent tables, if any, must already exist), replacing any previous value. original_value is optional but should be specified when saving tables otherwise only the specified (non-nil) keys are updated (to erase or replace a table simply specify the original value as an empty table).

lookup (database.key)
Efficiently checks if a key exists, returning only true or nil.

fetch (database.key)
Returns the value of the target key, or nil if it or any of its parents do not exist.

filter {table=database.key, key=name, value=value, max=200 }
Returns subtables in the specified table, having the named key and corresponding value (which may be nil to return records without that key).

teller.Pieces (database.key, keys, localise)
Returns a truncated copy with only the named keys from the table. Keys must be specified as an array of strings or name=key values, and the key may be namespaced. Localise should be true if any key is a localised table.

teller.FetchRecord (database.key)
Returns the table of the target key as a recordobject, or nil if the key does not exist. Records facilitate working with tables in the Teller, in particular for CRUD operations, by maintaining references to their key (record._key) and original value (record._original). Record objects behave the same as oridinary tables but are enhanced with the following methods for handling the record.

teller.MakeRecord (table)
Converts and returns the specified table as a record object, or a blank record if none.

record:Save ([database.key,] change)
Stores the record's table in the database, whether new or changed. If key is specified, the record's _key is ignored and the table will be saved to the new key (a 'Save a copy'' operation keeping the current record open). If change is true then the _key will be updated to the specified key (a 'Save as' operation closing the prior record). Equivalent to save(database.key or record._key, record, record._original).

record:Delete ()
Deletes the record from the database. Equivalent to save(record._key, nil, {}).

record:Fetch (database.key)
Allows retrospective population of a record with a fetched value if the value was not fetched during the record's creation, the record's _key remains unchanged. Equivalent to newvalues=fetch(key or record._key); copy(newvalues, record).

record:Key (database.key, create)
Set or change the record's _key.

teller.Sum (database.key, value, return)
Adds (increments) a postive number value or subtracts (decrements) a negative value to the value of the specified key. Value is optional and defaults to 1. Returns the new value.

teller.Delegate ("function", ...)
See Teller functions below. Typically only called from the Scribe when carrying out an operation to be delegated to other applications, such as changing database values directly instead of through application-specific functions.

Application Functions

Applications may define database functions that run in the Teller (procedures), and thus have unrestricted access to the database. This provides the ability to work with records, such as to iterate over all records (e.g. for search) without the overhead of fetching [all] values into a page first. You call and define these functions as you would a standard function, but you create them in the database.lua file.

Database functions accept a maximum of 3 parameters, and may return a single value only (use of a single table is recommended in both cases). Returning nil,"message" invokes the error view with the provided message.

Database functions are not suitable for long-running complex queries as they block queries from other backends. You may however run procedures in separate threads (see Tasks application), or you may periodically release the CPU (such as with every x iterations) using coroutine.yield() to allow the database to interleave the servicing of other requests with your procedure.

Teller Functions

These are used with your database functions to work directly in the database. Keys must be specified as serialised Lua strings (using the varkey function as necessary).

Loader ()
Run once for each application after the teller has restored persisted data from disk. Typically used to invoke delegated functions and populate indexes.

Starter ()
Run once for each application after all loaders have executed. Typically used to perform data integrity, sorting and finalisation routines that may depend upon actions carried out by other application's loaders.

delegate ("name", ...)
Runs database functions having the specified name in all applications (in an arbitrary order) with the specified parameters. Returns only true or nil, errmsg. Typically called only be a high-level function to notify other applications of an operation that is to be carried out so that any dependencies may be resolved by them.

All delegates are called regardless of the applications enabled for the current site.

save ("key", new_value, original_value)
Stores (asynchronously replicating and persisting as necessary) the new value in the database permanently as the specified table key (whose parent tables, if any, should already exist), replacing any previous value. original_value is optional but should be specified when saving tables, and must be specified when removing a table. You may be specify original_value simply as as an empty table {}.
Failure to do this will result in removed values being restored the next time the Teller is started. To change a table value to some other type, you must first delete it and only then assign the new value, failure to do this will corrupt the database.
These behaviours are handled automatically when you specify both new and old values.

Delegation

Provides event-type behaviour such that when operations take place that other applications may be dependant upon, any application may catch the event notification by simply defining a function with the corresponding name. The parameters are dictated by the application that instigates the delegation and carrying out the operation. The name of the delegate functions may be the same as the instigating function (e.g. DeleteUser), or different to better distinguish the purpose (e.g. DeleteUserDelegate).

The primary use of delegation is to maintain indexes.

It is essential to note that any indexed table (i.e. a 'record' table that is pointed to from other 'index' tables), must not have any new new table assignment made to it, e.g. don't use index[1] = users[1] and then save("users[1]",{…}) as the index will still point to the old table unless the delegate mechanisms update it accordingly (thus in this example one must also call teller.Delegate("IndexUser",id) to create pointers to this new table that replaced the old. This also applies to any subtable. To update values within such a table without replacing, you must use the teller.Update function.

Behaviours

Working with hierarchical data tables (nested hashmaps and arrays) contrasts with column and row-based data tables. Consider this when designing your data structures, and optimise for retrieving tables and subtables, instead of individual values.

Persisted (saved) values must be strings, numbers, booleans or tables. Cyclic references and pointers in persisted values are treated as unique key-value combinations and thus will be duplicated, loosing their prior associations.
Keys must be numbers or strings.

Provide original_value when saving a table or save changed or deleted values individually instead of entire parent tables; when deleting a table (new_value is nil) original_value may be specified as {} (e.g. an empty table).

When using persist for a new table or a changed one having no deleted keys, specify recurse as false.

Advanced Use

Database table keys are specified using native Lua table syntax, however this cannot be used to retrieve the entire database table or a root key (e.g. fetch(database)). However the key may be specified instead as a string, e.g. fetch "database", and for which hierarchical keys incorporating variables may use the following function.

Savvy (architecture)

This manual is a pre-release preview. Please follow Moonstalk on Twitter or Facebook to be notified of a public release.

Conception

Moonstalk is not ‘inspired’ by any other particular framework, instead its inception was driven by the desire for clean and flexible markup in views, with transparent database handling, and a preference to use RAM and avoid disk access as much as possible.

Its methodology does build upon the convention-over-configuration approach, extensively using tables from which values may be pulled (including the database, e.g. <title>?(db.record.title)</title>), and in which behaviours can be modified from their defaults (e.g. page.status=404). Further it avoids object-oriented programming (just as Lua does out-the-box), having no classes nor :method() calls (with some minor exceptions) instead favouring function calls that pass tables with typically optional keys (e.g. Function{arg1=value,arg2=value}).

Site and application bundles (folders with standardised file naming conventions) are used to simplify file management. Applications extend the core with functionality for specific usage scenarios, attempting to avoid including functionality by default that is not optimal for any scenario.

Components

Development Framework

Function

Enable the development and programming of sites and applications to serve dynamic web content, composed of both discrete and bundled layouts (design), logic (function) and assets, with an extensible set of functions.

Design

Easy to manage
Just drop all your HTML pages and templates into a directory.

Flexible templating
Drop a template file into a site, and dynamically defined templates.

Clean URLs
Addresses are easily mapped so as to correlate clearly to their intended content and functionality. (Optional REST.)

Understandable language
Lua is both simple for beginners and flexible enough for extensibility (integration with the shell, C and other environments).

Internationalisation
Integral and transparent user-localisation, using translated vocabularies, and utility functions for localised values such as dates, money and transliterated text.

Zero-downtime updates
Applications and sites should be updateable without downtime.

Convention over configuration
Akin to other frameworks, but promotes simple function-table relationships over more 'complex' objective style programming. (Classes, objects, methods and typing are not inherent in standard Lua, but may be employed using third-party modules through Lua's flexible metatable system.)

Model

Domain-specific (site) and cross-domain (application) bundles, comprising functions for acting upon and generating responses, invoked and configured through addresses and hook functions. Content–logic isolation is employed with a faceted view–controller model that supports templating, wherein responses are the merger of one or more sections.

Optimisation

All functions and content are loaded once and cached in RAM, but may be reloaded by sequentially restarting multiple backends for zero-downtime upgrades and updates. In developer mode, existing views and controllers are refreshed with each request to facilitate development.

Visitor analytics (logging) is considered best handled by third-party services using JavaScript, therefore Moonstalk is typically run with minimal logging for security auditing, performance optimisation and troubleshooting purposes only. If detailed logging is required, it may be enabled in the web server.

Performance

Static and dynamic pages without database queries are typically executed in about 1 millisecond. Pages with database queries typically execute in 1–5ms.

No schema
Tables may be structured in any way, with any value, without declaration.

Open storage
Data saved on disk is accessible with other environments.

Model

The database is a server daemon functioning as a shared Lua environment, operating for local clients (Scribe pages backends) over UNIX Domain Sockets. Persisted data is collated and written to an SQLite database on a dedicated thread. Additionally in a cluster configuration where replication is enabled, the database will maintain network connections with its peer nodes for synchronisation.

Optimisation

The database is intended for the retrieval of tables containing multiple name-value pairs in a single transaction, as opposed making multiple transactions for keys holding individual values, thus reducing overhead. Both as documents (tables) or aggregated results from Map-Reduce style function operations.

The database server keeps a dedicated connection open for each backend, executing queries sequentially on its main thread, whilst using separate threads for persistence and replication. Queries are thus not delayed by disk writes, and/or network issues resultant from the execution of preceding queries.

The persistence and replication functions batch changes together at a configurable interval to avoid resource (i.e. disk or network) saturation, and the overhead of many small transactions with commonly-changed values (e.g. counters). Typically the interval is low enough that in the case of a crash or unexpected power-off event, only a few seconds of non-critical data could ever be lost.

--TODO: Critical writes can be saved synchronously (immediately) and without delaying pending queries, through the use of a spawned thread that exits once either persistence or replication has completed. Likewise critical reads, except that they exit once a required number of nodes have responded (e.g. the first). Note however that the response to critical queries may be delayed by subsequent queries (synchronous responses are not supported).

Procedures are native Lua functions that execute on the main thread and thus block all subsequent queries. Procedures should therefore define structural actions that execute swiftly, unless they spawn a thread to complete their task (in which case no return value is possible), or utilise the functionality provided by the Tasks application to provide a result via polling.

Indexes allow for faster alternate database table lookups but introduce additional complexity in the maintenance of table structures and ownership by applications. Indexes are not saved to reduce persistence tasks, but [re]built on startup, and should be maintained on a per-query basis using delegated event handlers. See the Teller section for details.

Saved data accessibility is provided by using the open SQLite database file format. Individual values are saved in this under their unique combined table key namespace, and using flags to preserve data types. This data is read into the database's Lua environment upon starting.

Performance

Transaction times depend upon 3 factors—the size of the response, the complexity of the query (negligible for most fetch lookups), and the queue of pending queries.

This does not consider outstanding requests being queued for distribution amongst page backends by the web server, when under high traffic.

For a single fetch transaction, inclusive of overheads, a nil or simple chunk of text/HTML typically returns in 150 microseconds (0.15 milliseconds), whilst a table containing a few dozen nested keys returns in 200–300 microseconds. Lookups for non-numeric keys, within nested tables, and within very large tables introduce a little additional overhead.

This results in peak performance of 15,000 operations per second per Gigahertz (i.e per dedicated 'CPU' second on a single CPU core). Pages backends carrying out this many operations would probably be using at least the same amount of CPU time, therefore you would get half this performance per core, unless the database has a dedicated core. Realistically a page request would never make more than a few database requests, and currently 10,000+ requests per second is perfectly feasible. Performance for complex queries (functions) obviously varies according to the scope of the data. For higher traffic sites a multi-core 2Ghz+ processor is essential.

RAM requirements are approximately double that of disk storage requirements, i.e. for 5 MB of on-disk data (about 40,000 records of 200 characters, or 6,000 blog posts), the database will require 10 MB of RAM. Bear in mind that the non-persistent page cache functionality will require additional memory for every non-authenticated URL's HTML content.

Optimisation

Moonstalk's focus is on serving web applications. Static content is considered best served from a CDN, not from the same server as a webapp. Therefore static and dynamic content are considered separately. However Moonstalk configures the webserver to serve both, and thus can be used for low-use static assets and CDN origin-pull without unduly reducing capacity for serving web applications.

Additionally this approach enables the webserver to accept long-running uploads without blocking the FastCGI backends, only invoking a backend upon receiving the completed upload. It further allows for fine-grained security (i.e. rejecting requests before they reach the backend).

In using multiple separate FastCGI processes, the number of processes is tuned to make maximum use of the available CPU cores, or indeed to restrict use to a subset of cores such that capacity can be reserved for other processes such as a database.

The processes preload and cache the sites (settings, views, controllers and applications) compiling functions, keeping everything in RAM, thus vastly reducing processing times as compared loading from disk with every request.

Performance

With both Scribe backends and Teller database with a typical web application, 165 requests per Gigahertz can be handled (i.e per dedicated 'CPU' second on a single CPU core). On a quad core 2.5Ghz CPU, the engines can thus serve 4,500 requests per second.

It's not yet apparent if there are scheduling (inter-process) bottlenecks that might prevent increased capacity on hardware with better performance without further optimisation.

Credits

Moonstalk's development is supported by The Moon Mill, and is released as open-source under the terms of the Artistic Licence 2.0. (Note that the licence will be changed to GPL once a roadmap has been established.)

Origins

Moonstalk was conceived in mid-2010 during the development of a SaaS e-commerce platform, that had been designed to take advantage of the Lua environment to provide very low-latency database-driven page generation. It's architecture and (potentially dubious!) programming has been undertaken by Jacob Jay (@jacobjay) almost continuously since then, in both India and France.