LiveCode Forums

A Little Message To The Developers RE: LiveCode 7.0.3

A Little Message To The Developers RE: LiveCode 7.0.3

Posted: Thu Feb 26, 2015 7:03 pm

by mickpitkin92

Greetings folks,

So I got to try out LiveCode 7.0.3 today and I have to say I am impressed, the engine is the fastest I've used since the open source revamp began, I am looking at my Binary File Copying Stack v2.0 project (I might have a gift for others if this goes to plan, would be nice to have somewhere central to host it by the way) in the debugger and as I press F10 and F11 to go through the code, the engine and IDE are not skipping a beat, during version 6 I noticed the engine was quite laggy in updating the UI between pressing the F11, executing the line of code and then updating the debugger UI.

But 7.0.3 is glorious, startup and shutdown is instant, you'd honestly think I had an SSD and not mechanical drives, I've noticed a particular increase in the speed at which the engine can process a repeat loop working on the files in a folder like System32 on Windows (A folder with 3,730 files as of Windows 8.1 x64), what would previously take a second or two on version 6 is now instantaneous, Process Hacker 2 reports a CPU usage of < 0.01 executing the 30 tick idle loop in the BCFSv2 and only increases to about 1.06 when performing an idle task such as copying a file.

Honestly, whatever you have done with the engine, keep doing it, this is beautiful and I am definitely looking forward to seeing the upcoming enhancements to the engine.

With love, Michykins

PS. I am not being sarcastic neither.

Re: A Little Message To The Developers RE: LiveCode 7.0.3

Posted: Thu Feb 26, 2015 9:17 pm

by FourthWorld

Very glad to read that - thanks for posting it.

I think we owe a debt of gratitude there for the dev team-community partnership between long-time user Jacque Gay and RunRev engineer Ali Lloyd.

To help identify any performance differences between 7.0 and earlier versions, many of us participated in a project led by Malte to craft specific isolated benchmarks. Interesting as that project was, the results weren't actionable since many of the underlying methods for so many aspects of engine behavior operate very differently in v7 than in earlier versions.

So instead RunRev's Peter Brett put out a call for any projects where speed is a noticeable issue under v7, and Jacque submitted one of hers. Apparently in the course of reviewing her stack Ali discovered a significant opportunity for boosting throughput, and jumped on it.

Another earlier change may also play a role here. It's less significant in terms of per-iteration benefit, but at least as pervasive: copy-on-write-only.

In earlier versions, when passing values to handlers the value was copied unless you'd specified that it should be handled by reference using "@", e.g.:

This meant a relatively expensive allocation and copy for each value passed to every handler.

In v7 values are always passed by reference initially, which means copying just the 4-byte address rather than the entire value itself.

But passing by reference is very different from passing by value: by value means you're working on a copy of the data, so the original is unaltered, but by reference the original data is used.

When you want to operate on the original data that's a good thing, but if you want the handler to alter the copy and leave the original alone, how does this new scheme handle that?

Smartly.

It detects when an operation will alter the value passed in, and unless it was explicitly declared as "by reference" with "@" in the argument list, at that time it will copy the data so the handler leaves the original alone.

If you think about the number of times we have handler that read data passed to them without altering it, you can begin to appreciate the value this change has for overall performance.

Coupled with the great work Ali's done with the issue Jaqcue found, it looks like v7 is shaping up very nicely, well on its way to being not only the most feature-rich version of LC ever, but the most bug-free and in many cases the most performant.

Re: A Little Message To The Developers RE: LiveCode 7.0.3

Posted: Thu Feb 26, 2015 11:26 pm

by jacque

I would so love to take credit for that stack, but it was someone else. Hopefully they'll step up and take credit. (Was it Malte? Maybe.)

But I did have a drink with Ali at the last conference, does that count?

Re: A Little Message To The Developers RE: LiveCode 7.0.3

Posted: Sat Feb 28, 2015 10:50 am

by malte

I think it started on the lists. Need to check out 7.0.3 urgently tonight.

Re: A Little Message To The Developers RE: LiveCode 7.0.3

Posted: Sat Feb 28, 2015 12:13 pm

by Dixie

A little premature with the applause, you chaps !... Stacks won't run in the iOS simulator here if they are running on 7.0.3 (Stable)... go back to 7.0.2 (rc2) and there is no problem (sigh)

Re: A Little Message To The Developers RE: LiveCode 7.0.3

Posted: Sat Feb 28, 2015 12:45 pm

by LCMark

@Dixie: Sorry that this isn't working for you - which simulator versions have you tried? Is it 32-bit or 64-bit you are simulating?

Re: A Little Message To The Developers RE: LiveCode 7.0.3

Posted: Sat Feb 28, 2015 3:28 pm

by malte

Just checked 7.0.3

What keeps me from migrating to 7.x still is present though. Most controls I use in my projects make massive use of arrays. Handling those in 7 did not improve (yet). The team is aware though and I still have hope that it can be optimized. (Penalty is about 10 to 12 times the slower than in previous versions of the engine).

Cheers,

Malte

Re: A Little Message To The Developers RE: LiveCode 7.0.3

Posted: Sat Feb 28, 2015 4:41 pm

by FourthWorld

Malte, much of my work is also heavily dependent on arrays, so I'd be keen to see this addressed. Do you have a link to the test stack that measures the difference?

Re: A Little Message To The Developers RE: LiveCode 7.0.3

Re: A Little Message To The Developers RE: LiveCode 7.0.3

Posted: Sun Mar 01, 2015 5:38 pm

by FourthWorld

Thanks for that test stack, Malte. Given the many deep changes between the core of the v6 engine and v7, many isolated tests can be less helpful than we might think, because the methods used to achieve those results are now often handled through very different means, often eluding any true apples-to-apples comparisons in profiling.

That said, array operations are key to so many aspects of our work with LiveCode that I agree they merit attention. And more interesting was the sort command, the one that did indeed take up to 10 times as long to complete compared to earlier versions when I ran your test here.

I've forwarded a link to your post to the core team at RunRev, and encouraged them to review sort and some aspects of array handling to see what can be done to boost performance.

I'll keep you posted with the outcomes of that discussion.

Re: A Little Message To The Developers RE: LiveCode 7.0.3

Posted: Wed Mar 04, 2015 5:43 am

by mwieder

Malte-

Your sample stack shows up some *dramatic* slowdowns in all 7.x versions.

6.6.2:
Time taken to build data: 638 ms
Time taken to sort keys: 127 ms
Time taken to rebuild data: 155 ms

6.7.0:
Time taken to build data: 564 ms
Time taken to sort keys: 115 ms
Time taken to rebuild data: 116 ms

6.7.3:
Time taken to build data: 617 ms
Time taken to sort keys: 110 ms
Time taken to rebuild data: 116 ms

7.0.3:
Time taken to build data: 2472 ms
Time taken to sort keys: 1094 ms
Time taken to rebuild data: 1036 ms

Re: A Little Message To The Developers RE: LiveCode 7.0.3

Posted: Wed Mar 04, 2015 11:46 am

by malte

Hi Mark,

that's what has me worried.

Re: A Little Message To The Developers RE: LiveCode 7.0.3

Posted: Fri Mar 06, 2015 8:56 pm

by LCMark

I spent a good deal of time analyzing Malte's test stack over the weekend and a number of things come up as a result.

The first is that the 'simpleArrayTest' script is really highly synthetic - it is constructing a numerically keyed array, sorting the numeric keys, and then rebuilding a list in order. The problem here is that it has a whole step in the middle which you wouldn't see in practice (because it is a 'no-op' abstractly) and it is operating on integers. Thus whilst indicative of a slow-down it is perhaps not a very good example to use as a source of optimization suggestions for general code (indeed, this is one area where with the LC7 architecture it would be possible to - in this explicit case - make the keys() and sort() functions no-ops with a little bit of behind-the-scenes-cleverness in the underlying types which would have no effect at all in any area other but instances of this case).

So @malte, I realize this stack is probably modelling code you have in your larger system in a simple way, however it would be incredibly useful if you could provide a version which does a similar operation but with a 'real' dataset. I'm imagining here you are building large arrays with text keys (non-numeric), then having to extract, filter, sort etc. them in various ways. Real-world data here is incredibly important as it would be a real test of the speed of the primitives in 'useful' cases, rather than just number processing the engine (theoretically) doesn't need to do anyway.

(By the way, I'm not saying that if you put 'real' data in things will be more performant - I'm fully prepared for the worse here - real world data just would help focus on which specific aspects need work first).

Now, having said that there is a fair bit of information that I have gleaned from this specific case... Here are a few together with potential improvements:

The keys() function is a lot slower and seems quite immune to optimization. This is (as far as @livecodefraser and I could tell) entirely down to cache-locality. This has been caused by the change in how arrays are represented in 7.0 compared to 6.7. Even though the old array data-structure was implemented before cache behavior was a concern in programming it turns out it was hugely cache-friendly when an array was built all in one go. This could be tackled by a couple of internal changes (merging the data-type which is used to represent the key in an array (a name) with strings; and allowing small strings to be stored inline in the strings memory block, rather than in a separately allocated array).

The sort() function is doing way too much work at the moment - it is copying strings rather than referencing ranges. This is certainly something which we can (quite easily) do something about.

Indexing arrays is now substantially faster for specific cases, but it does not cover the cases highlighted in this example. Indeed, we need to ensure that needless mapping between names and strings is eliminated. One way to achieve this is to make the result of the 'keys()' function construct a string lazily - really keys() returns a vector of names which when viewed as a string should be separated by a return character.

Eliminating the conversion from integer to string that occurs when making numeric keys in an array (this is something which 6.7 does too - but as integers are quite small byte-wise when made into strings its not a huge issue there).

Eliminating the need for the copy which repeat for each <stringchunk> currently does. (6.7 didn't copy, but then 6.7 was hugely unsafe here - if you mutated the container variable used in the repeat for each line, you'd cause unintended and sometimes catastrophic side-effects!)

Overall there are lots of other small improvements which we know we could make - particularly with classification of types of string (which is a function of the unicode codepoints they contain, and their order) and the codepaths then used to process them.

So, anyway, we know we've got some work to do here - and I can assure you we aren't ignoring the issue.

(I should perhaps end by saying that 7.0 is not all bad news at all - code which manipulates large heavily nested arrays, or very large strings both as atomic entities will run substantially faster because the 7.0 architecture eliminates structural copies for assignment)

Re: A Little Message To The Developers RE: LiveCode 7.0.3

Posted: Fri Mar 06, 2015 10:21 pm

by malte

Hi Mark,

first of all: I really really appreciate that you are looking into the performance issues I see...

what this test tries to mimic is a) a live search on a datagrid (search while you type style of thing), b) the building of data for an rTree, where data is fetched from a DB, temporariely written into an array in a set of function calls. Keys are integers, as nodes and leaves need to be built in order, and this in the end requires the sorting, as the order of the keys in the array is not stable. Therefore simply using repeat for each key is impossible, or at least not desirable.

For a) most of the time I can clearly get along with getting the dgText and filter on that, however, the teststack also somewhat reflects what happens internally when the array for a dg is being built from setting the dgText Also if you manually build dgData dg requires numeric keys for its array and we face a similar scenario. Most of the time my data set are rather large, thus the huge number of lines in my test. That said, a more realistic script for building data / getting the subset would be:

on mouseUp
lock screen
local tData,tTest,tLog,tSort,tOutput
put "Testing filter performance with engine"&&the version into tLog
put the millisecs into tTest
repeat with i=1 to 500000
put i &","& any item of "true,false" & cr after tData
end repeat
put cr & "Time taken to build data:"&&the millisecs - tTest&&"ms" after tLog
put the millisecs into tTest
-- set a cprop to be able to restore original data in live search
set the cData of me to tData
-- live search part begins here. This will happen on any keystroke
put cr & "Time taken to set cProp:"&&the millisecs - tTest&&"ms" after tLog
put the millisecs into tTest
put the cData of me into tData
filter tData without "*"&",true"
put cr & "Time taken to retrieve and filter data:"&&the millisecs - tTest&&"ms" after tLog
put tLog into fld "log"
unlock screen
end mouseUp

Testing filter performance with engine 6.7.3
Time taken to build data: 489 ms
Time taken to set cProp: 8 ms
Time taken to retrieve and filter data: 47 ms

Testing filter performance with engine 7.0.3
Time taken to build data: 2699 ms
Time taken to set cProp: 0 ms
Time taken to retrieve and filter data: 173 ms

For b) the simple array test is rather realistic. Of course in both cases building the data would try to use repeat for each wherever possible, but still the sorting problem would remain.

All the best,

Malte

Re: A Little Message To The Developers RE: LiveCode 7.0.3

Posted: Sat Mar 07, 2015 12:13 pm

by LCMark

@malte: Thanks for taking the time to respond - knowing the context of what simpleArrayTest is trying to model helps an inordinate amount

I take it that in the real script, the 'sort' in simpleArrayTest has a 'by' clause? i.e. you'd actually be sorting the list of integer keys based on data in tData? If so it would be really helpful to have an example clause which is being used there for further analysis. (This is why I mis-inferred the use-case - as the sort was just re-ordering the integer keys to be in order, I presumed it was modelling what you need to do to get a list of text keys in alphabetical order).

Okay, so the problem here is that the script is currently having to do far far too much work (even in 6.7) in order to achieve the effect it has - its essentially taking a small trip abroad and coming back home to basically generate a list of integers which reflect the order of the original elements relative to some sort function. Indeed, here are the steps actually being taken by the engine:

Produce a numerically-keyed array with 500000 elements

Iterate through the keys of the array in hash-order and produce a string of about 500K in length

Split the key by lines (internal to sort) (**)

Sort the lines by the sorting function (in this case, just numeric comparison)

Combine the lines with return back into a string (**)

Iterate through each line and use them to reconstruct a string with the original data in the new order

Now, I'm pretty confident we can make 7 get similar speeds to 6.7 for these precise sequence of steps - it requires a fair bit of internal re-plumbing (as outlined above) that we have already started to plan. However, this is a reasonable size job and whilst thoroughly worth doing is not going to change overnight. However, the more near-term approach is to make sure that script has the primitives it needs to do what it is doing above without taking a small trip via Venus! After all, not doing work is always faster than doing work.

Indeed, unless I mis-read the script and its intent, there are only two operations which are needed to eliminate the majority of the string construction / deconstruction overhead. The engine already has the idea of 'sequence' - a numerically keyed array with all keys 1..the number of elements - so you really want to be able to write:

Basically, 'sort' needs to operate on 'sequences', and 'combine' needs to recombine such sequences in numeric order (this would cut out the steps marked (**) in the list of operations above).

Exactly the same can be said about 'filter' as 'sort' - if we make it act on sequences, then you can essentially cut out a string build and split operation.

Obviously this approach would require a small update to script to take advantage of. However, over time, if we make 7.0 keys() function lazily convert to a string (keys() is really a sequence, with some considerations about elements containing the list delimiter) then you get much the same effect.