DOL is now successfully loading an object from an SD card and running it!!! So I thought that I would start a thread for it.

I will keep this post updated with the latest version.

New beta version of DOL2, haven't updated version number since basically nothing in DOL itself has changed, just the demo that comes with it. Much improved but its not finished yet. Dump all the *.dol files onto your sd card. TV needs to be on pins 12+, keyboard on 26+ and sd on 0-3. I'll update it to include the spec soon http://forums.parallax.com/images/smilies/smile.gif. Then load SDM_DOL2_Core_V001. You should get a Hello world message. Then type in "upper" and after a short period of time the screen will stop displaying new stuff. This loads the file upper.dol and gives it access to standard input, output and error streams. If you type on the keyboard you will see the stuff converted to uppercase (but don't try anything other than lowercase letters, it just adds on "A"-"a"). You will also find a file in the zip that has a basic object tree that I've been working on. All the stuff below is old but I'll leave it here at least until I get this version finished.

Current version V_014
-Should be able to be used.
-Requires an SD. You need a TV and mouse for the demo.
-See DOL readme in zip file for instructions.
-Loads a file from the SD card and displays "Hello World!" on the TV demonstrating loading a TV driver and running it.
-Converted the mouse driver. Not all of the methods are callable but there should be enough to be useful.

Issues
-Setting freed memory to zero was causing an error so it is now commented out. I'll try to fix it for the next version.

Next version
-More testing.
-Maybe a keyboard and mouse driver.

Thanks to
-hippy for all his work on spin op-code and object format
-Mike Green for modified version of fsrw
-Peter for his heap object
-Chip for making such a great chip

Steven

Post Edited (stevenmess2004) : 7/11/2008 1:21:57 PM GMT

Peter Verkaik

01-14-2008, 07:15 PM

Steven,
Is it possible to use a hash function for the object/method names instead of constants?
We could then use
callobject(hash(string("objectname")),hash(string("methodname")),arg1,arg2)
Here are some·hashfunctions
http://www.cse.yorku.ca/~oz/hash.html

For objectname you can of cource use the filename.
That really should improve readability (and solve maintenance issues).

regards peter

hippy

01-14-2008, 08:06 PM

I think some hash encoding is a good idea. Just packing the first four characters into a long would be a start, 5-bit characters would allow six character names, proper hashing even better.

The only downside is that it adds a lot of run-time overhead doing the hashing on every call and throughout that case statement every time. Ideally a built-in string hashing function for the PropTool is needed here, or macro support, so it can be done at compile time.

It might be possible to do a JIT hashing to speed things up. The string pointed to has the first msb clear if it is character data ( assuming ASCII characters =< $7F ), and the bytes in bytecode could be turned into a hash and written back with msb set. Care would have to be taken with strings less than three characters and the hash wouldn't be long aligned, so there may still need to be a fair amount of bit shifting.

Added : Or change PropTool so every string is only ever generated once. Then the address of the string is its hash. I can think of a few possible cases where that would be undesirable, and in those it can be worked round. - Won't work because the loadable object has its own strings and will not be the same as the loader does. Oh well.

Post Edited (hippy) : 1/14/2008 1:17:04 PM GMT

Peter Verkaik

01-14-2008, 08:38 PM

A lot of the overhead can be moved to loadObject, which already takes
quite some time to load an object. The objectname can be converted
to a hashnumber, that hashnumber loaded into a free entry of objectNameList[],
and if object present on SD and enough·memory and a free entry available,
loadObject can return the entry number 1-16 (handle) or null (object not loaded).
The handle can be passed to callObject, making the search for the entry
obsolete which saves time. That handle would also be passed to unloadObject,
again eliminating·a search.
Edit: in fact, objectNameList[] can be eliminated alltogether, as objectStartPtr[]
holds a nonzero value for a loaded object, or null for a free entry.

regards peter

Post Edited (Peter Verkaik) : 1/14/2008 1:52:25 PM GMT

stevenmess2004

01-15-2008, 01:03 PM

There is probably quite a few places where I could speed things up and save memory. I probably don't need the objectAllocated variable either. Just check the objectStartPtr[] for 0 will tell us if the object is allocated. As for the objectName[] we could probably just return the startPtr (or the index) like Peter said and then you don't have to remember what it is in your code. As for method names it may be possible to do the hash when loading the function. But why not just use an enumeration starting at different values? Have certain areas reserved for common drivers (TV, VGA, Graphics, serial, etc) and then start everything else at some other value? I know its not ideal but I can't think of a better way unless we get a pre-processor for Prop-Tool (weren't there a couple of pre-processors written?)

Good to have some feedback from some others http://forums.parallax.com/images/smilies/smile.gif Time to start some more fixing.

Steven

Ariba

01-15-2008, 01:57 PM

stevenmess2004 said...
(weren't there a couple of pre-processors written?)

Thanks for the links Ariba, I may be able to use PreSpin to make using DOL easier. I'll have a better look at it next week sometime.

stevenmess2004

01-15-2008, 05:39 PM

Another day and another version of DOL. See the top post. This one is now loading two objects from the SD card and demonstrating calls between the objects. You should see stuff on the screen on the VGA and TV output. There is still a fudge factor of -8 in the length of the VAR section. I think I just figured out what it is. Hippy is the -8 because we are reading the offset from the 8th memory position? Next update will focus on getting the documentation up to date and improved.

Steven

Post Edited (stevenmess2004) : 1/15/2008 10:47:27 AM GMT

hippy

01-15-2008, 11:33 PM

I'm afraid I haven't been closely following the development of your code, so I'm afraid I cannot relate that -8 to my simpler version and I'm not sure of its purpose.

stevenmess2004

01-16-2008, 02:55 AM

Hippy, to get the amount of VAR space that I need to allocate, I read the initial stack pointer from the object header. When I subtract away the program length and the program start position I am left with 8 more bytes than I should have. It has happened on two different objects so it seems to be consistent. The -8 simple corrects for the extra bytes.

Steven

stevenmess2004

01-16-2008, 05:15 PM

More documentation as promised. If you have a look at dummyObject2 and DYNDemo1 they should tell you almost everything you need to know. Also changed to not having the TV driver in the DOL object. This saves space and allows more memory for objects. See first post for new version.

stevenmess2004

01-19-2008, 04:42 PM

There is another new version of DOL. This version is now useable. There is a bug with the aborts and I haven't tested the call to DOL for the SD card yet but everything else is working. See the first post.

stevenmess2004

01-19-2008, 05:13 PM

I'm getting this error message when I try to upload the zip file. Any hints?
* You cannot upload files that use MIME type : application/x-zip.

Harrison.

01-20-2008, 12:23 AM

I get the same error when I try to use firefox to upload files to this forum. I have to switch to IE to get around that error. You could always edit the browser MIME type on your computer, but that's just a pain and kind of a hack.

Ym2413a

01-20-2008, 08:21 AM

I can't wait to give this a try. :)
I remember me and a few other guys from around here talking about how something like this would help development of large programs.

Zip file is now working using Safari on a Mac http://forums.parallax.com/images/smilies/smile.gif. If anyone uses this please let me know what you think. Any comments will be greatly appreciated (positive or negative). I was going to try to make it so that multiple objects of the same kind used the same code but I don't think I can do it (I just thought how I could but it will take some work to get it working and will be slower again. Who uses spin when they want speed anyway?)

Thanks. Hopefully I'll have a new version up tonight. Just a couple of small things and a keyboard/mouse driver. After that its time to try and get some event driven (from mouse and keyboard) code working.

Can anyone see why this produces garbage on the screen. It is just an object that runs a modified version of Graphics_Demo. The changes I have made to Graphics_Demo are to make it single buffered and to store the bitmap in an array rather than at some arbitrary point in memory. Compile testKern and save it as kernal.bin onto your memory card. If you put a call to start the Graphics_Demo at the start of the methodRunner method in testKern and load it into the propeller (eeprom or ram) it works. Is Graphics_Demo or Graphics using some set address that I am not seeing?

Some help would be great.

Thanks

Steven

I am about half way through the next version of DOL which will allow for an inheritance and only loading VAR sections if the PUB and PRI sections are already loaded.

stevenmess2004

06-22-2008, 07:22 PM

New version. See first post.

stevenmess2004

07-11-2008, 08:26 PM

Another new version. I just busted my sd card interface and can't be bothered fixing it tonight so I wasn't able to test the last change that was supposed to make things uppercase. If its not making things uppercase, change the stdout.stdout(stdin.stdin+"A"-"a") to just stdout.stdout(stdin.stdin). This will just mirror what you type to the screen. It also spits out a bit of data onto the serial port so that I could see what was happening. The reason it took so long was because it wasn't working when I had the stdIn and stdOut streams the wrong way around. (seems really strange, wonder why http://forums.parallax.com/images/smilies/smile.gif probably has something to do with doing this too late at night). There is also a object tree file that gives an idea of what I am working towards.

stevenmess2004

07-12-2008, 05:46 AM

sdenson, I assume that since you deleted (you can't hide, it emails me all the postshttp://forums.parallax.com/images/smilies/smile.gif ) your post you figured it out. The place to change the pin assignments for the tv and keyboard is in the terminal file where they are started. Just change these two lines to whatever you are using

When it starts, DOL loads terminal. Terminal then loads tv_text as the standard output and standard error output streams and comboKey as the standard input stream. Then it waits for you to type something and loads that. If you type in "upper" it loads upper.dol and calls the start method with the standard input, output and error streams in the argument list. Upper then just echo's what you type to the screen using the standard input and output streams that it has been given.

Plans for the future are to
1. Finish DOL2 so that it unloads objects and a couple of other things
2. Reorder/rename methods in the rest of the objects in the object tree file so that we can use more things as stdioe
3. Write a pre-processor that fills in all these numbers and generates headers automatically.
4. Write a post-processor/fix dol that gets rid of the header objects in the binary since they are just taking up space.
5. Fix inheritance either by playing with the method table (hard but faster) or calling the parent object with code (easy but slower)

jazzed

07-12-2008, 09:51 AM

Ya, I had a hard time waking up today:)

Been looking at a way to do a download/save to disk via serial port. This would be to make updates more convenient than they are now (thank Tom for not caching the SD card driver ... no choice on "safely removing" on windows though). I have working download "client/server" code from another project. Do you have a serial DOL?

The basic methods you have for console services are interesting, but not having a basic driver suite is limiting. Being able to "extend" objects is interesting and a desirable alternative to the simple unixy file handle way, and I can see how it can be done. You know it will be difficult to design as well as use. Having the template headers is desirable for detecting buggy calls at compile time rather than crash time ... considing only one long per method, I don't see how getting rid of them is a win unless it is an option for deployment -vs- development.

You should seriously consider the one line printf ideas: http://forums.parallax.com/forums/default.aspx?f=25&m=277986 Modules snprintf+itoatoi adds 301 longs for first use, but having many tv.dec, tv.out, etc... is fatter after a point and quite inconvenient. Eliminating blah.dec(n), blah.hex(n,d), etc from the stdout/stderr requirement is a win.

You have a reasonable and expandable piece of work here. I think good communication of your design and usage could make it more embraceable by a larger piece of the community. Being able to easily replace components is a great advantage and will be very welcomed when the mythical PropII comes (which because of many items, makes a larger general computing platform possible).

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔

stevenmess2004

07-12-2008, 04:24 PM

That's weird, in the email your name is sdenson but online its jazzed. Maybe something for the tech support to look into?

jazzed said...
Ya, I had a hard time waking up today:)

Been looking at a way to do a download/save to disk via serial port. This would be to make updates more convenient than they are now (thank Tom for not caching the SD card driver ... no choice on "safely removing" on windows though). I have working download "client/server" code from another project. Do you have a serial DOL?

Not yet, after I finish the unload code fixing more drivers is the next thing on the todo list.

said...
The basic methods you have for console services are interesting, but not having a basic driver suite is limiting. Being able to "extend" objects is interesting and a desirable alternative to the simple unixy file handle way, and I can see how it can be done. You know it will be difficult to design as well as use. Having the template headers is desirable for detecting buggy calls at compile time rather than crash time ... considing only one long per method, I don't see how getting rid of them is a win unless it is an option for deployment -vs- development.

Getting rid of them gets rid of the need to maintain two copies of the same file for custom objects (its too easy to change one and forget the other). You would still need them for the standard objects.

[qoute=""]You should seriously consider the one line printf ideas: http://forums.parallax.com/forums/default.aspx?f=25&m=277986 Modules snprintf+itoatoi adds 301 longs for first use, but having many tv.dec, tv.out, etc... is fatter after a point and quite inconvenient. Eliminating blah.dec(n), blah.hex(n,d), etc from the stdout/stderr requirement is a win.
I could make it so that the snprintf object is passed in the parameter list like the stdin, stdout and stderr objects. Could add a str output to the standard output and get rid of the advanced input/output routines which would save space.

said...
You have a reasonable and expandable piece of work here. I think good communication of your design and usage could make it more embraceable by a larger piece of the community. Being able to easily replace components is a great advantage and will be very welcomed when the mythical PropII comes (which because of many items, makes a larger general computing platform possible).

Yeah, once I get the terminal working better than I'll start writing some documentation and some better demo's. Maybe I could write the pre-processor for it http://forums.parallax.com/images/smilies/smile.gif Only problem is that we can only have one file open at a time on one sd card. It should be possible though to load another complete driver for an sd card on different pins.

jazzed

07-14-2008, 12:10 PM

Hi Steven,

Tried a few things yesterday. The upperCase.spin file has a reference to Simple_Serial in it; I could not get that to work. I tried creating a serial_DOL as attached, but that doesn't work either. In american football terms: it's 4th and 10, all I can do is punt. Have a look please.

Thanks.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔

stevenmess2004

07-14-2008, 07:17 PM

I'll check into it tomorrow.

stevenmess2004

07-15-2008, 05:28 PM

The problem is that I haven't finished the terminal yet so its not actually passing the arguments yet http://forums.parallax.com/images/smilies/smile.gif. Also, I am using the first 3 longs for references to the standard input, output and error streams. In fact, I'll even say that I'll start the actual arguments at fifth long so that there is room for the format object if I want to use it. I'll have to fix up the terminal before its going to work properly. Does it work if you hardcode the numbers in? (I haven't fixed my sd card holder yet...)

stevenmess2004

01-07-2010, 06:06 PM

Almost time for another update. I've got sphinx modified so that it now spits out the index of an object in the object table :). Now just to test my other modification to sphinx... Should be new code posted this weekend.

trodoss

01-08-2010, 01:54 AM

Very nice! Looking forward to the update.

--trodoss

KPR

04-15-2010, 11:48 AM

Any further development on this??

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
New ICON coming, gotta wait for the INK to heal, now we have colour!

stevenmess2004

04-15-2010, 05:11 PM

Some, but nothing ready to release yet. I made one (to get the index in the object table) of the mods to sphinx and then couldn't figure out how to make the second (remove objects that are meant to be loaded dynamically).

I've started on another new version that cuts back on some of it's functionality to try and make it easier to use and take up less space.

I've also run into a problem of sorts, the heap object I'm using is byte aligned rather than long which is needed.

I also need to start using the latest version of fsrw.

And the biggest problem was that I had an sd card failure and haven't got around to fixing it yet...

I'm going to be away from my computer next week but after that I'll put some time into it, especially if someone here sends me a friendly reminder in my pms...

In fact, I'm looking at the code right now :)

stevenmess2004

04-15-2010, 05:48 PM

Here is my latest version. Not test at all and this isn't a working demo yet, just thought some people may be interested in what I was working on.

I want to make a linked list or something for it that will take advantage of the dynamic nature of it to help test with...

Dave Hein

04-16-2010, 06:26 AM

Steven, do you have a description of how you modify the offsets?· I'm having trouble understanding what your code is doing.

stevenmess2004

04-16-2010, 03:46 PM

Dave Hein said...
Steven, do you have a description of how you modify the offsets? I'm having trouble understanding what your code is doing.

That's probably not surprising, sometimes I have problems with it http://forums.parallax.com/images/smilies/smile.gif

Okay, first reading the header, this should be simple and the code can be improved now that fsrw has a seek command

'read ProgramStart, VARStart and StackStart from header
'subtracting these will give us program length and var length
'set objectSize - from header
'read all the data in the header and use whatever we need
repeat 8
sd.pgetc
objectLength:=sd.pgetc '8 'byte of varBASE
objectLength.byte[&#173;1&#093;:=sd.pgetc '9 'byte of varBase
objectLength:=objectLength-$10
varSize:=sd.pgetc 'A 'byte of stack start
varSize.byte[&#173;1&#093;:=sd.pgetc 'B 'byte of stack start
varSize:=VARSize-$10-objectLength-8 '-$8 fudge factor to be figured out
repeat 4 'get to end of header
sd.pgetc

This gives us two important pieces of information, the object length (DAT+PUB+PRI+Object/method table) and the size of the VAR area. A little math is needed to figure it all out and I wouldn't have a clue why there is an extra -8 in the varSize calculation but it needed it to work.

Next is allocating memory and putting the addresses together in a useful format. I stripped it of a lot of extra stuff to get only the essentials.

'allocate space for methods
objectStartBase:=mem.allocate(objectLength+3) 'need the extra 3 bytes so we can make sure that the object is long aligned
'load methods
sd.pread(objectStartBase,objectLength)

'allocate space for VAR
objectVarBase:=mem.allocate(varSize) 'need the extra 3 to ensure vars are long aligned
'zero the var area
bytefill(objectVarBase,0,varSize)

The arguments to this are the objectAddr we got previously from DOL and the position. position is the position of the address in the object table. To get that you start at 0 and add 1 for every PUB, PRI and Object (including each instance of an array object) to the object you want to change. This method when called will fix up the offsets so that then you can call the object with a normal call. Note that I had to split the calculation into two parts so that overflows during the calculation didn't become a problem (maybe the problem you were having with C-DOS?).

Now, there are still a couple of problems.
1. How do we get the position? I sorted that with a modification to sphinx. Pretty simple but I have to find what I did because I think I lost the modified code on a bad sd card :(
To use the mod you just did this

CON
objectIndex=@object

worked nicely though only in a con block. Was an easy mod.
2. We still need a space in the object table. I started modifying sphinx so that you could use a notation like
object:@"object"
which would compile everything as normal and then just not include object in the final linking process. There was a free byte I used in the sop file to mark the unused objects but I couldn't figure out an easy way to not include the file.

Hopefully that helps. If anythings not clear than ask away.

stevenmess2004

04-16-2010, 06:51 PM

Something that may be helpful is this file from hippy.

Dave Hein

04-16-2010, 07:37 PM

Steven,

Thanks for the explanation.· It's much clearer now.· I am still a bit confused by the VAR section.· I see that you allocate memory for it, and set the VAR offset in the table to point to it.· I thought the VAR entry in the table was an offset from the previous object's VAR space.· It appears that your using it as an absolute address.· I'll need to study that some more to understand it.

In this bit of code the second line fixes up the offset to the object and the third line fixes up the offset to the var section. "this" is a variable that was figured out earlier that conatains the absolute addresses of the object and var sections.

stevenmess2004

01-30-2011, 11:36 AM

Here is a new version of DOL. This one will copy create a new copy of the VAR section of an object in the hub allowing you to add and remove objects in the hub. Should be good for things like dynamic linked lists or binary trees.

The demo is pretty basic and shows creating five new objects and assigning a different VAR variable to each, than calling each and seeing all the separate values.

There is code in DOL for loading objects from an SD card but it hasn't been tested yet.

Dr_Acula

03-29-2011, 11:50 PM

Thanks for the link to this thread from the cogject thread. This is a fascinating piece of work. I'm coming up to speed with how this works - in post #1 is the source code and in that is a mouse driver code and I can see there is a 'methodrunner' which replaces calls to each method.

The cogjects that are now working are keyboard, serial and the two cog VGA driver.

TV, Mouse and other VGA drivers are next on the list. The underlying SD card driver is Kye's code which happens to fill a cog, so this hub space can be recycled for other cogjects. 8k of ram already saved.

Combine that with the code here and it ought to be possible to move both cog code and the supporting spin code in and out of memory, either from an SD card, or from external memory. That means one can have arbitrarily large programs on a standard propeller.

How many DOL objects have been created so far, and how 'hard' are they to create?

stevenmess2004

03-30-2011, 01:35 AM

Sorry, should have said to look at the version in the last post. There are some significant differences in how things work that should simplify a lot of things.

I've been thinking about DOL and cogject a bit and if you only want to create an object that load an existing object into a cog and then get unloaded it should be really easy. I'm currently at work but will post some ideas tonight.

Writing objects that fully take advantage of DOL is a bit of a pain because it really needs two features that none of the compilers support. These are 1. Getting the index of an item in the method table and 2. Not including a referenced object in the binary output file. There are workarounds for both but they are messy and prone to errors.

If the sole purpose is only to load cogs and then free memory we can get around both problems easily by putting all the hard bits in a single object that doesn't need changing.

Dr_Acula

03-30-2011, 02:10 AM

Re the compilers not supporting these two things, I wonder if the authors of the BST/homespun might be able to help?

Is it simplistic to think of something that the compiler produces, a huge detailed text file for instance, with links to all the data and where it ended up etc, and you could then use this information? "Not including a referenced object"... there must be something one could do there. I'm thinking back to the CP/M project where we needed #ifdefs and then like and Brad was able to help out there with custom commands.

What would the syntax look like? Something that starts with # and doesn't include a block of code?

At the moment with cogjects, the aim is to be able to load and reload code as needed. For instance, consider a program that loads up a text graphic driver and asks the user some questions, then unloads the graphics driver and keyboard and loads up a graphics driver and mouse to play a game. At the moment you can recycle the cog space, but you end up with all the spin code from previous objects still in hub memory.

It would be great to be able to unload the spin code too. Treat an object as something that can be loaded and unloaded as often as you like. I think you already have this working but more objects are needed. I guess the essential ones are sd card, keyboard, mouse, TV, VGA (text and graphics).

With cogjects, I am trying to change the standard objects as little as possible. The pasm code stays exactly the same. Generally, all the 'start' methods are having one extra variable being passed which is the location of the cog loader space in hub ram. Most of the methods remain unchanged. Generally the start methods are requiring a lot of rewriting - eg any values that are implanted into pasm by way of variables now have to be done with a longmove to a pre-calculated area.

If we spliced that in with your code, I think you add a method driver which references all the methods.

Of all the existing objects I have coded, the keyboard one was probably the simplest because it has hardly any methods and all the values are passed to pasm with a PAR. I wonder what the standard keyboard object would look like in DOL?

stevenmess2004

03-30-2011, 11:42 AM

Hi Drac

First, attached is a new version of DOL that implements loading objects from sd cards and also a loadcogject method that loads an object from an sd card, gets it started in a cog and then exits. To use it you will first have to compile FullDuplexSerialCogjectCog.spin and save it to the root directry of your sd card with the filename "fdscc.bin". Obviously all the pin settings will need setting up for your board. Please note that this isn't tested at all because I don't currently have a spare sd card. Don't try it if there is stuff on your sd card you haven't backed up. If you have any problems with it post the stuff that gets dumped to the serial terminal and I'll have a look.

To get back your questions.

Brad indicated a year or two ago that it should be pretty easy to add the support needed but other stuff was understandably higher on his priority list at the time. We need two things indicated in the code box below.

OBJ
obj1 : #"dummyObject" 'This leaves a spot in the object table for the object and lets us use the object but doesn't actually include the object in the binary

CON
objIndex:=@obj1 'or
objIndex:=#obj1 'This puts the index of obj1 in the object/method table into objIndex
'I got this working in sphinx but I would have to find it again and I can't remember which syntax I used...

If you have a look at the attached FullDuplexSerial changes, all I had to do was separate the bit that gets loaded into the cog from the spin parts of the object. In this case it was easy. Some of the other drivers may be harder. Once that is done we simply change the spin part of the driver to call DOL to load the cog instead of doing it all in the one file.

stevenmess2004

03-30-2011, 11:44 AM

Should have said, don't turn on the remove unused code/methods option in bst. It will muck-up DOL.

Cluso99

03-30-2011, 01:13 PM

Drac: as a temporary solution, could you use a special string in the line above to denote an address that is required. Then perhaps, using the listing you could scan and find the relevant addresses. ???