The Oforth Programming Language

Oforth Tutorial 1 : Basics

I - Oforth interpreter

Oforth interpreter is launched using --i option. If installation is ok (otherwise check OFORTH_PATH variable), you should see something like that, which means that oforth interpreter is now waiting for input :

Interpreter is very simple : a word (separator is space) is read and performed before reading next word. For instance, if you write :

> 1 1.2 "aaaa"

Interpreter reads 1 and performs it, reads 1.2 and performs it, reads "aaaa" and performs it, and so on until there is no more word to read and perform.

Leaving the interpreter is done using #bye function.

II - Data stack

Unlike most languages, Oforth uses a stack, the data stack, to hold created objects, parameters and returned values. It is a Last In First Out stack.

To do so, Oforth uses a RPN notation : parameters are all pushed on the data stack before calling a function or method. The function uses thoses parameters to do its job, then, if necessary, pushes its return value on the data stack. This can be strange at the beginning, but this allow to keep interpreter as simple as possible : no need for instructions separator, no need to read an entire instruction to decide what to do, the interpreter just reads a word at a time and performs it.

For example, in order to perform "1 + 2", you write :

>1 2 +

- "1" is read and performed, and the result is to push Integer 1 on the data stack

- "2" is read and performed, and the result is to push Integer 2 on the stack : now 2 integers are on the stack.

- "+" is read and performed, and the result is to remove 2 objects from the stack, add them and push the result on the data stack.

After that, the data stack has only one object : integer 3. Function #.s can be used to print the data stack :

>.s
[1] (Integer) 3
ok

This means that the data stack has one element, that it is an integer and that its value is 3.

1) Integer 3 is now very far from the top of the stack : it is now at position 6

2) 12 and 13 have been added and 25 is on the top of the stack (position 1), without changing objects under them.

3) All objects on the stack have a class and a value.

4) Everything is an object. For instance, Integer is a class but it is also an object that can be pushed on the stack like any other object. Integer is an instance of class Class (at position 2 on this stack).

5) If we go a little deeper into the interpreter work : if a word is detected as an object, it is pushed on the stack and if the word is detected as a function or method, it is performed immediatly.

To finish with the stack, let's clear it, using #clr function

>clr .s
Empty
ok

Now the data stack is empty.

Using RPN is not natural and needs some practice. Let's calculate : sqrt(1 + ln(12.4))

>1 12.4 ln + sqrt .s
[1] (Float) 1.87555231135018
ok

1 has been pushed on the stack, then 12.4, then 12.4 has been replaced by ln(12.4) and 1 is still here, #+ added those two numbers, then #sqrt has replaced it with its square root, which, at the end, is the only object remaining on the stack. As you can see, with RPN, no parenthesis is needed : the stack holds all intermediary results.

If you want to print the stack after each command, you can use #.show function. Default is to not print the stack after each command.

You can also use #. to print and consume the top of the stack.

III - Hello, World

Now that we are a little more familiar with the data stack, we can write our first word.

>"Hello, World" .
Hello, World ok

String "Hello, World" is pushed on the stack and #. removes it and send it to the console.

Now the function : a function is just a named piece of code that will perform this code when called (ie when interpreter reads its name).

>: hello "Hello, World" . ;
ok
>hello
Hello, World ok

A function declaration begins with : followed by function name, and ends with ; . Between, are the function instructions that will be performed when this function is called.

Quite simple, but we must understand that #hello, like all functions or methods, has used the data stack : it pushed astring on it, then removed it.
The data stack is shared by all functions : you can imagine it as a pile of plates. When #hello runs and pushes a string on the stack, it adds a plate to the pile. And when #hello runs #., the string is removed from the stack and the pile is again as it was before #hello call.

To end with this first function, it is time to answer a question you might have in mind : from the beginning of this tutorial, why a '#' is used before functions or methods names ? We have to remember that, eveything is an object that can be pushed on the stack, used as parameter, ... It is also the case for functions and methods. Creating a function #hello is creating an instance of Function class, a function object with "hello" as its name. When "hello" string sent to the interpreter, it detects a function name and perform the function. Now if we send #hello to the interpreter, it detects an object and pushes it on the stack.

>#hello .s
[1] (Function) #hello
ok

Object #hello is now on the stack. This object can be used as parameter for others functions or methods as any other object. For instance, #perform is a method that performs the object on the top of the stack :

Various methods can be used on function or method objects. Let's finish this chapter with some examples :

>#hello name println
hello
ok
>10 #hello times
Hello, World Hello, World Hello, World Hello, World Hello, World Hello, World Hello, World Hello, World Hello, World Hello, World ok
>#hello bench
Hello, World 236
ok

For the last example, #bench executes the top of the stack and prints number of micro seconds elapsed. Here, it took 236 microseconds to perform #hello.

V - Data stack operations and conventions

The data stack is a great feature. Well, perhap's, right now, you are thinking that it is not really the case :), but you will see that, with the data stack, you will write very concise programs and factoring words is a key. If you are not confortable with the data stack and RPN, you will miss lots of Oforth strenghts.

Lots of words are defined to manipulate directly the data stack and will be presented here. But, the first thing to understand is that the data stack only holds references to objects. Objects themselves (with the big exception of integers) are stored elsewhere (on the heap or into the dictionary). So, when you manipulate the data stack elements, you manipulate only references to objects. When you remove an element from the DS, you are just removing a reference, you don't remove an object. All objects, whatever their size is (strings, list, ...), consume only one emplacement on the data stack, that holds reference to this object.

Comments allow to describe the effect of a word on the data stack. Comments are initiated using #// or #--. A stack effect is decribed by showing elements used by a word, then --, then elements returned by a word. The convention is that the top of the stack is always to the right part. For instance :

// a b c -- d e

decribes a word that takes 3 elements from the stack (the top of the stack is c) and that, when returning, leaves 2 elements on the stack (the top of the stack is e).

Convention for elements type is :

n : an integer
u : an unsigned integer
b : a boolean (false or true)
f : a float
c : a character
s : a string
x,y,z : an object, whatever its type is
[x] : a collection of objects of type x

Now, we can introduce words that manipulate the data stack :

dup // x -- x x : duplicates the top of the stack

Again, only the reference is duplicated, not the object itself. For instance :

Other words exist to manipulate the stack, but these words will be used during the tutorial. You can see "lang" references for a list of all words.

VI - Oforth words

Words are named objects defined into the dictionary :

They are available during all the current session and are not removed by garbage collector even if not used.

They are detected by the interpreter that will handle them according to the word class.

Two words can't have the same name : if a new word is created with the same name, an exception is thrown.

Words are defined into "lang/words.of" file.

Words define Oforth object metamodel. They can be :

a class : it is an instance of Class class and is used to instanciate objects of this class. When the interpreter performs a class, it pushes it on the stack.

>Integer .s
[1] (Class) Integer
ok

a property : it is an instance of Property class and is used to set properties to classes. Like classes, when the interpreter performs a property, it pushes it on the stack

>Comparable .s
[1] (Property) Comparable
ok

a function : it is an instance of Function class and is used to name a piece of code. When the interpreter performs a function, it performs function's code.

a method : it is an instance of Method class. Each class can have an specific implementation of a method (polymorphism). When the interpreted performs a method, it execute a specific implementation according to the class of the object on the top of the stack.

a constant : it is an instance of Constant class. When theinterpreter performs a constant, it pushes constant value on the stack.

>Pi .s
[1] (Float) 3.14159265358979
ok

Other words type exist but we will discuss about them later.

VII - Functions

Functions are named piece of code. A general function declaration is :

: function_name
| var1 var2 ... varn |
&nbsp instructions ;

A function declaration begin by #: and ends with #; . Into this declaration :

function_name is the name of the function, which must not be the name of an already defined word.

var1, ... varn are function local variables
If the function does not declare local variables, || are not required (see #hello)
They are initialized to null value.
They can be used to store intermediary results instead of keeping them on the data stack.

Using a variable name into instructions will push the variable value on the stack.

Using "->var" into instructions will remove the top of the stack and store it into variable var.

instructions are function code that will be performed when the function is called. A instruction can be :

Push a word on the stack (a constant, a class, ...)

Push a local variable on the stack : this is done by just using variable name

Remove the top of the stack and store it into a local variable : this is done using " ->var_name"

Call a function or a method : this is done by just using function or method name.

Local variables define function environment. This environnement is not on the stack, it it linked to function call and is removed when the function returns. So storing the top of stack into a variable will remove it from the stack and store it into the function environnement. It then can be pushed on stack, used to call other functions, ...

Example 1 : square

: square dup * ; // x -- y

#dup duplicates the top of the stack. So, to calculate the square of a number, we #dup it and call #*. The result is already on the stack, so there is nothing more to do (this is just an example, this word already exists as #sq into lang package).

Example 2 : fact (imperative)

: fact // n1 -- n2
&nbsp| i | 1 swap loop: i [ i * ] ;

The comment says that #fact takes an integer from the stack and leaves an integer on the stack when it returns.

Into the function body :

A local variable is declared (loops need a local variable).

1 is pushed on the stack.

#swap exchange the two objects on top of the stack, so the number we want to calculate the factorial is now on top of the stack and 1 is behind.

loop: i [ instructions ] is a built-in instruction : it removes an integer from the stack and will loop on instructions for each integer between 1 and this integer (included). Into thoses instructions, the local variable can be pushed on the stack and contains the current index.

So here, i is pushed on the stack, the two integers on top of the stack are removed, multiplied and the result is pushed on the stack.

Do you agree that this function will replace the top of the stack with its factorial ? If you are not familiar with RPN and data stack and don't see how this factorial is calculated, try to follow the data stack while the loop is performed. I suggest you to not continue this tutorial before you fully understand this example and how the data stack is used.

This example also introduces a new convention : words that require something after to work (which is not the natural way with RPN) are postfixed by ':'. For instance, #loop: needs a local variable to work, so it is postfixed.

Example 3 : fact (imperative) with a parameter

Oforth allows to declare parameters on functions. Parameters are local variables, but are initialized with values on the stack (instead of null value) and those values are removed from the stack. Like local variables, parameters are stored into the function environnement. Parameters are never mandatory, but they can sometimes simplify stack operations. Let's rewrite #fact with a parameter :

: fact(n) -- n1
&nbsp| i | 1 n loop: i [ i * ] ;

A parameter (n) has been declared for #fact. So, when #fact is performed, it removes the top of the stack and store it as parameter value. Here, this parameter is just pushed on the stack after pushing 1, and a #swap is not needed. There is no really advantage to use a parameter here.

Note for Forthers : Oforth does not provides direct access to return stack with words like >R and R> so, sometimes, declaring locals (parameters or variables) will be required.

Example 4 : fact (recursive)

: fact(n) -- n1
n ifZero: [ 1 ] else: [ n dup 1 - fact * ] ;

Now fact is defined as a recursive function. This function introduces tests. Oforth provides various tests and they all work the same way : they remove the top of the stack and test it. If the test is OK, instructions within brackets that follow are executed. Otherwise, the function jumps after those instructions. A test can always be followed by a #else: block. If present, this block is only performed if the test is KO.

Tests provided are :

ifTrue: [ ... ] : instructions are performed only if the top of the stack is not "false" (ie not 0)
ifFalse: [ ... ] : instructions are performed only if the top of the stack is "false" (ie 0)
ifZero: [ ... ] : instructions are performed only if the top of the stack is 0
ifNull: [ ... ] : instructions are performed only if the top of the stack is null
ifNotNull: [ ... ] : instructions are performed only if the top of the stack is not null

Now the function : if n is 0, #fact just returns 1 on the stack. Otherwise, it pushes n, pushes n-1, call #fact recursively and multiply the result. Because n is a parameter of #fact, its value is stored into environnement call and is different for each call to #fact.

Example 5 : fibonacci sequence, introduction to blocks

Fibonacci sequence is u(n+1) = u(n)+u(n-1). It can be written by many ways.

The first one is to use a reccurence :

: fib(n) -- n1
n 1 <= ifTrue: [ n ] else: [ n 2 - fib n 1 - fib + ] ;

Not much to say about this implementation, it is the same logic than #fact.

The second one is to use the data stack to hold intermediary results. If fib(n-1) and fib(n) are on the stack, what are the instructions that transform them into fib(n) and fib(n+1) ?
A way to do this is : "tuck +" :
- fib(n-1) fib(n) --> tuck --> fib(n) fib(n-1) fib(n) --> + --> fib(n) fib(n+1)

So performing n times "tuck +" on 1 1 will calculate fib(n) and fib(n+1) and, after removing fib(n+1), fib(n) will remains on the data stack.

: fib(n) -- n1
| i | 1 1 n loop: i [ tuck + ] drop ;

But Oforth does not like loops : loops are prone to bugs, they need a local variable to be declared and they are not very "functional". Here, we just want to perform "tuck +" n times, without using the index. So we can use the #time function we used on #hello earlier.

times // n aWord -- ... : perform the word n times

For this, we could create a word that perform "tuck +" and use this word into #fib. But what would be the name of this word ? Names created must be representative and reusable. Here, it seems better to use an anonymous function. Forthers call this a quotation, Oforth calls this a block. A block is a word with no name and is declared like this :

#[ tuck + ]

When a block is created, it is pushed on the stack like any other object. Then it can be used as parameter and can be performed using #perform. So the new fib version could be

: fib(n) 1 1 n #[ tuck + ] times drop ; // n1 -- n2

Blocks are the reason why syntax for tests is different from classical Forth. When you write :

ifTrue: [ ... ]

this simulates conditional block execution according to the top of the stack and is comparable to block syntax.

Blocks are a powerful feature. Functions can return blocks and you will see later that function can even return a block that uses function's variables. In this case, blocks are closures that will react according to parameters of the function that create them (even if the function that created them have returned). For instance, you can write :

>: f( a -- aBlock ) #[ a + ] ;
>12 13 f perform .
25 ok

Example 6 : Sum of digits of a number

This functions will introduce the #while loop. This loop allows to perform instructions while a test is OK :

while( instructions1 ) [ intructions2 ]

instructions1 are performed and must leave an object on the stack. If this object is not false, instruction2 are performed and the program flow jumps back to perform instruction1 again. This will continue until instruction1 returns false (or 0). In this case, the functions jumps after the instruction2 block

: sumDigits(n) -- n1
0 while(n) [ n 10 /mod ->n + ] ;

n is pushed on the stack. If n is 0, the function jumps after the block (and finish). If n is not 0 the block is performed using /mod, which is a word that returns 2 objects : the remained and the quotient of two numbers :

/mod // n d -- remainder quotient

If #fact is still declared into your system, you can try :

1000 fact sumDigits .
10539 ok

Example 7 : Parameter's order

When more than one parameter are declared for a function, the last declared parameter value is retrieved from the top of the stack, and so on until all paramters are removerd from the stack. So, declaring this function :

: f(a, b) a b - ;

will store (and remove) the top of the stack into parameter b, will store (and remove) the second element and store it into a. So this function will calculate the second element of the stack minus the top. Its stack effect will be : // n m -- (n-m). This code is equivalent to :

: f | a b | ->b ->a a b - ;

Note for Forthers : Since V0.9.23, parameter's order is now the same order than { } words.

IX - Basic types

Oforth defines some basic types that are detected by the interpreter. Interpreter loop will :

Retrieve the next word (until a space is detected).

Check if this word is defined into the dictionary.

If the word is a function or a method, perform it

Otherwise, if the word is a constant, push its value on the stack.

Otherwise, if the word is an integer, push that integer on the stack.

Otherwise, if the word is a float, push that float on the stack.

Otherwise, throw an exception : "Can't evaluate".

Integers
Integer is the most basic type. They have arbitrary length. If an integer is a small integer (-2^30 - 2^30 on 32bits), it is pushed on the stack. Otherwise, an object is created and its reference is pushed on the stack.
Integers respond to lots of words (see "lang" package reference for a complete list). Among those words, we find :

Booleans
Oforth does not have a boolean type. Boolean are integers : 0 is the boolean false and 1 is the bollean true. Everything that is not false is considered as true.
Integers respond to logical words :

Floats
Floats are also detected by the interpreter.
They respond to lots of words (see "lang" package reference for a complete list. Because Oforth uses polymorphism, the same words are used for integers and for floats. Among those words, we find :

Characters
Oforth does not have a character type. Characters are integers which value is character unicode value.

>'a' toUpper .
65 ok

Strings
Strings are an array of UTF8 characters. They are created using "".

>"abcd" "efgh" + .
abcdefgh ok
>"abcdef" toUpper .
ABCDE ok

IX - Constants and immutability

Immutability, even is not mandatory (a mutable class can be created, see Classes chapter), is the default behaviour. Objects have to be immutables for some particular uses and this immutability is checked at runtime.

Among immutability contraints, we find :

Set class immutables attributes (discussed into Classes chapter).

Pass objects between tasks (discussed into Parallelism chapter).

Set constant value.

A constant is a word that will be visible by all functions and methods. A strong constraint is to block mutable objects for value of constants.

A constant is created using the following syntax :

value const: constant_name

For instance (see Float.of file) :

-1.0 acos const: MyPi

Now Pi is defined as a word and each time the interpreter reads a word that is a constant, it pushes its value on the stack.

Classes does not have a syntax to declare internal constant. In order to link a constant to a class a naming convention is used : put the class name as the first part of the constant name. For instance (see TCPSocket.of) :

2 const: TCPSocket.IPV6

X - Methods and polymorphism

All the words we have created until now are functions. Functions are Forth words. In order to allow Object Oriented Programming, Oforth extends words to a new kind of words : classes and methods. Methods are words that allow to specify a particular implementation for each class.

First of all, methods don't belong to classes, a method object can be created independently of any class :

Method new: method_name

This creates a new method with name method_name and without specifying any implementation. If this method already exists into the dictionary, the call is OK but nothing is done.

However, this syntax is rarely used. It's main use is to be able to use this method into bodies even if no implementation have been declared.

This declaration defines an implementation of method_name for aClass. If the method object does not exist yet, it is created by the first implementation declaration for this method (that's why method creation is rarely used).

Unlike functions, when a method is performed, it need a receiver to decide which implementation to use. And this receiver is always the top of the stack. When a method is called :

The virtual machine checks the top of the stack object and retrieves its class.

Then the correct method implementation is choosen according to this class. If no method implementation can be found for this class, an exception ExDoesNotUnderstand is raised.

Then the top of the stack is removed from the stack and is stored into a special parameter : self, which is the method receiver. This parameter does not have to be declared : all methods have this parameter.

If a method has declared parameters, they are handled like functions (and they should be "under" the receiver).

Then the method implementation body begins, like functions.

The most important thing to understand with method implementations is point 3) : the top of the stack, like other parameters, will be removed from the stack by the virtual machine before method body is performed and stored into method environnement as "self" during body execution.

Example 1 : myDrop

>Object method: myDrop ;ok > 12 myDrop .s Empty ok

When myDrop is performed, the virtual machine check for the top of the stack. 12 is an Integer and Integer is a subclass of Object, so the virtual machine find an implementation of #myDrop compatible with Integer and perform it.

This example illustrate point 3) : the receiver (here 12) has been removed from the stack and, when the method ends, nothing is left on the stack.

Example 2 : myDup

>Object method: myDup self dup ; // x -- x x

Because the receiver has been removed from the stack, we have to push it, then dup it in order to implement a #dup method.

#minusOne pushes the receiver one the stack and removes one to it. Because implementation has been declared for Number, it works for integers, floats, ... but not for strings.

Most of built-in words are methods. This allows to have the same name for the same operation with different implementations. For instance #+ is a method and is declared for various classes: Integer, Float, String, ... You only have to remember one name : + and it will work for all objects.

Virtual methods implementation :

When #method: is used to implement a method for a particular class, this implementation will be used if the receiver is an instance of this class or its subclasses. But subclasses can't redefine this implementation. In order to define an implementation that can be redefined into suclasses, #virtual: should be used instead of #method:

#super allows to call the method implementation declared at the parent level (and, if it does not exist, at the first level the method is declared into the hierarchy).

Some advices for virtual methods :

By default, use #method: and not #virtual. It is not a performance advise. If a subclass should not redefine a method, using #method: prevents it to do so.

If subclasses have to redefine a method and #virtual is needed, we should think twice about it : is our class hierarchy truly good ? Why do subclasses really need to redefine a method if they really implement a "is a" relation.

If subclasses really need to redefine the method, then declare it as virtual into the parent.

XI - Another way to pass parameters to functions and methods

The classical way to pass parameters to function and methods is to push them on the stack before calling the function. For instance, with this function :

Here again, this is just "sugar" : interpreter will translate this version into the previous one before compiling this function.

To be more general, if a, b and c are objects and f is a function :

a b c f
f(a, b, c)

are the same call to function f (and the second one is converted into the first call by the interpreter).

Just like functions, the interpreter also allows this "sugar" syntax for method calls. But unlike functions, the receiver must always be on the top of the stack, so, if aObject is the receiver and m a method:

aObject m(a, b, c)is converted into : a b c aObject m

Since V0.9.23, parameter order is now conform to { } Forth words. This applies to declarations and to this "sugar" syntax for calling words.

XII - Collections and high order functions and methods

Even if Oforth is an imperative language, it can handle some functional programming features. But first, let's create some collections.

The main collection is an array (an instance of Array class). It is an immutable collection, which means that, once the array is created it cannot be updated anymore.

An array is either created explicitly, either as the return value of methods that manipulate collections.

Lists are also created as the result of high order functions. High order functions work on collections by applying functions, methods or blocks on each element of the collection. Some high order methods (see lang/Collection.of for more) :

#apply : this method applies a runnable on each element of a collection

#map : this method applies a runnable on each element of a collection and regroup results into a new list.

#filter : this method filters a collection according to a test and returns a new collection with elements that conform with this test.

...

First let's see how #apply works, and the best way to undertand it, and its interactions with the data stack, is to see #apply source code (into lang/Collection.of) :

#apply removes an object from the data stack (the collection) and enters into a loop : for each element of the receiver it pushes this element, then the runnable sent as parameter and calls #perform on this parameter. If something is created on the stack, it is not removed and stay on the stack.

#++ has been applied on each element of the collection and the results stay on the stack.

>0 #+ 100 seq apply .
5050 ok

Here, we push 0 on the stack, then call #apply on [1..100] with #+ as parameter : for each element, this element is pushed on the stack, then #+ is performed, which result is the sum of all integers from 1 to 100. This method is defined as #sum into "lang/Collection.of").

Here, it is interesting (but not mandatory at all) to use the "sugar" syntax presented before to clearly show that #+ is a parameter of #apply :

>0 100 seq apply(#+)

Of course, blocks can also be sent as parameter to #apply :

>0 1000 seq apply(#[ sqrt + ]) .
21097.4558874807 ok

Block sent as parameter is performed on each element of the collection : sqrt is calculated, then #+ accumulate the result, so this calculates sum of square roots of all integers from 1 to 1000

#map differs from #apply as all results are regrouped into a new list :

X - Blocks revisited

We have seen that blocks are unamed piece of code : #[ sqrt + ] creates an object on the stack and, when this object is performed, block code is performed.

A block body can also use variables and parameters (and receivers) of the function or method that creates it :

>: dynblock(n) #[ n + ] ;

This functions creates blocks on the stack and block body uses function parameter n. When the function finishes, function environnement is deleted but the block will keep the value this parameter has when the block was created :

Here, the created block will keep the value of this parameter until garbage collector deletes the block. The block can be performed long after the function call has been done, the parameter value is now included into block environnement. This behaviour is similar to a functionnal language returning a function.

This function returns a block that uses function parameter f. When this block will be performed, it will take the top of the stack (x) and calculate (f(x+e) - f(x)) / e.

Let's try with f(x) = x^2 + x

>2.0 #[ dup sq + ] deriv perform .s
[1] (Float) 5.00000009395762
ok

You cannot create locale variables dedicated to a block, but local variables of the function that created the block can be used into the block body. This is what has been done into #deriv : a local variable x has been created into function body and used into the block.