(Outdated) Idea for a token based WPS handling

Info

This is a design paper of an idea to make the wps-handling token-based.
Not all wps-tags are handled in this paper.
This is only an idea, maybe this is the wrong way to do it if the wps-code should be rewritten.

Paper

this file describes a possible new methode for the wps-handling (token-based)
currently this only describes the used data-struct and how the wps gets updated
the biggest part will be to generate the tokens from the wps-format-file.
All non-"static"-tags and text (static-tags = which needs only be parsed on wps-file-loading)
will be converted to tokens.
the text-token holds all non-tag text-string until the line ends or a tag was found
each found tag (also "normal" text) in the wps-format will be represented by an token
each token points to the next token. When an tag is used in an conditional then the token
(1)points to the next token after the conditional via "next",
(2) points to the first token in the conditional itself via "next_conditional"
known problems in this design:
- how to handle the creation of the tokens and how to manage then needed memory-space to hold all tokens
(because we can't know how many tokens will be needed upon loading)
- how to handle when between two tags are only a few spaces(<5) to avoid to use an extra token(if possible).
pseudo-code:
var strings // buffer which holds all "normal" text-string each string is null-terminated
define wps_token:
var type // which tag it is e.g. image-loading-tag either the tag-string(without the %/%?) or a number representation
union
pvar value // the value(as an pointer to the string) of the tag if type = text otherwise null
var id // image id if type = image otherwise -1
var line // line-number
var conditional // if found as an conditional
pvar next // points to the next token if conditional = 1 it points
// to the token after the conditional
pvar next_conditional // points to first token in the conditional if conditional = 1
struct-size:
4 Bytes (type)
+ 4 Bytes (value/id)
+ 4 Bytes (line)
+ 1 Byte (conditional)
+ 4 Bytes (next)
+ 4 Bytes (next_conditional)
------------------------------
19 Bytes
function update_wps:
var token = start_token
var oldcharpos = 0
var after_con = null
var oldline = 0
do
if token.conditional // is token an conditional
value = get_value(token) // get conditional value from the tag represented by the token
if value != -1 // check if the tag was valid to be used in an conditional
if(!after_con) // is this token the first conditional (when an conditional is in an conditional)
after_con = token.next // save the token after the conditional
// go to the result-token of the conditional
i = 0
while i < value
token = token.next_conditional
i++
// check if the token is an empty one (the result isn't used)
// when empty then jump to the token after the (first) conditional
if(token.type == empty)
token = after_con
// check if the new token is not conditional-one
else if(!token.conditional)
handle_token(token) // handle the token
token = after_con // jump after the (first) conditional
after_con = null
else
token = token.next // go to the next token
else
handle_token(token) // handle the token
token = token.next // go to the next token
if token.line != oldline
oldcharpos = 0
while token.next != null // loop until all tokens are handled
update_lcd() // update the lcd with the new frame-buffer
function handle_token(token): // handles non conditional tokens
if token.type != text and token.type != image // if the token is not a simple-text-string or an image
value = get_value(token) // get the value of the token as an string
print(oldcharpos,token.line,value) // draw the value-string on the given position
oldcharpos = string_length(value) // save the new string position for new to display values/text
else if token.type == text // if token is an simple-text-string
print(oldcharpos,token.line,token.value) // draw the text
oldcharpos = string_length(token.value) // save the new string position for new to display values
else if token.type == image // if token is an image
draw_image(token.id) // draw the image
function get_value(token) //returns the value for the wps-tag
// if token.conditional is set then the "normal" value is mapped
// to a conditional-value if the tag can be used in an conditional
// otherwise -1 (when token.conditional = 1)

Comments

-- MatthiasM - 25 Dec 2005: nice concept. But why don't make everything a token, why only dynamic elements?
If you also make "normal" text parts a token it would be much easier to add "attributes" to screen elements.
e.g. to give elements (even text parts) foreground and background color, to set fonts, to add icons, absolute or relative
screen positions, ...
The hardest and biggest part would still be parsing the WPS file and generate appropriate tokens

-- NicolasPennequin - 15 Jan 2007
There are two flyspray entries about this : FS#2898 and FS#4826.
I've started work on trying to implement this, though not exactly in the way described above (I posted some preliminary code on FS#4826). IMHO it can be much simpler than what's suggested, i.e. no need to keep track of the next conditional and the next token.

-- NicolasPennequin - 20 Mar 2007
I have made good progress on this. I have created a new tracker entry for my work : FS#6862. I will try to find time to edit this page to explain the concepts of the patch, and maybe add information and ideas about a possible parser plugin and bytecode WPS format.