Earlier today, I realized that I was keeping too much of the design of Thelema, Kether, and other things related to them in my head without really working out the details. I hadn't written any notes on the matter in over a year, during which time I have changed a lot of my ideas around, and clarified a lot more, and I noticed just how jumbled my thoughts were getting. I decided to sketch out a few new design notes, and update my old ones, and while I am still finding some inconsistencies and outdated ideas, I am working on getting my current thoughts down in writing for future reference.

I have put the current notes on my GDrive, and I will hopefully be working on updating them periodically. Any constructive comments or questions would be appreciated.

_________________Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTFμή εἶναι βασιλικήν ἀτραπόν ἐπί γεωμετρίανLisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.

I don't have anything specific to say about your language, but i've also been working on a language for about 3 years now and wanted to share some opinion and hindsight that gave me the most help in the design process and might be useful for you.

1) Understanding the Continuum,This was a huge win for me personally, on understanding what I was actually doing, and what i was designing. It was a bit like a separation of concerns.

I have come to settle on the following areas,that all influence each other, and unfortunately create a loop. Making a change in one place can ripple around to the other areas. And highlights to me why you can not have "one true" language, each "language" is just the settled down positions of these areas. Adding say a Concurrency Model to a language may cause problems in the syntax for e.g. if statements (which has happened to Smalltalk).

TL;DR; Get a good understanding of what bits you're designing. I kept too much of a mishmash of ideas without understanding how they relate.

2) Ditch BNF (Your milage may vary, my language is object-oriented and has statements as well as expressions.)I have come to the conclusion that BNF is a waste of time. My reasoning is that what it is trying to describe is functional composition, but it omits details and makes this more confusing that it really needs to me. I imagine 90% of people will want to right a Recursive-Descent parser anyway so map your syntax design to what you'll actually be writing. I decided to go with XML/XSLT to describe my syntax and transform that into my code required to parse.

I required additional information, most notably naming each matched statement. If that statement was optional or not, and control-flow constructs (one-or-more, one-of which map to symbols used in BNF)

3) Get to code alreadyI totally regret spending so much time in the design phase and working on paper. I also regret trying to code in C++ from the get go. I ditched both of these things for off the shelf tools and tools that were very forgiving, Current compiler is written in Javascript, XML and XSLT. - I am much more interested in the flexibility to try new things quickly than anything right now. And you can start talking about what it DOES instead of what it will do. You really can't tell from paper alone how well something goes.

I have given a lot of thought to all of these issues, but I hadn't really tried to codify them this way. That's something I ought to try, thank you.

cxzuk wrote:

TL;DR; Get a good understanding of what bits you're designing. I kept too much of a mishmash of ideas without understanding how they relate.

This is very relevant. I have spent too much time thinking about things without nailing down the details and getting them down in writing.

cxzuk wrote:

2) Ditch BNF (Your milage may vary, my language is object-oriented and has statements as well as expressions.)I have come to the conclusion that BNF is a waste of time. My reasoning is that what it is trying to describe is functional composition, but it omits details and makes this more confusing that it really needs to me. I imagine 90% of people will want to right a Recursive-Descent parser anyway so map your syntax design to what you'll actually be writing. I decided to go with XML/XSLT to describe my syntax and transform that into my code required to parse.

I required additional information, most notably naming each matched statement. If that statement was optional or not, and control-flow constructs (one-or-more, one-of which map to symbols used in BNF).

I have to disagree with you here, but I suspect that it also reflects a misunderstanding of BNF itself. Backus-Naur Form is for defining lexemes and syntax, and nothing more; It serves mainly to formalize the design of the lexical analyzer and the parser. It sounds as if you were trying to apply it to the semantics, which is simply a bad idea all around. Abstractly at least, the parser shouldn't even know things like element matching, or which construct is a conditional and which aren't. While those all get knotted up together in a most direct-emit recursive-descent parsers,

In any case, I was only using EBNF to define the lexemes, as parsing an s-expression language is pretty much trivial, even where read macros are used in the program. Most of the things that would have a special syntax for in conventional languages, such as assignment, variable definition, arithmetic expressions, conditional constructs, and so forth are all done with s-expressions instead, and if a syntax is really needed, it can be defined programmatically with read macros. For example, rather than the infix expression

Code:

int a = 3 * 4;

a Lisp would use

Code:

(define a (* 3 4))

where 'define' and '*' are standard forms (or in some cases, library procedures). The general syntax of '(<function> <arg-1> <arg-2> ... <arg-n>)' is pretty much the basis of all s-expression languages, Lisp or not (most s-expr languages are List processing languages, but some, like TRAC or my own language, Thelema, are not).

This is especially true in this case, since the core language has no literal forms at all; numeric literals, string literals, character literals, and anything else I choose to have a literal syntax for will be defined in a combination of macros and specializers.

cxzuk wrote:

3) Get to code alreadyI totally regret spending so much time in the design phase and working on paper. I also regret trying to code in C++ from the get go. I ditched both of these things for off the shelf tools and tools that were very forgiving, Current compiler is written in Javascript, XML and XSLT. - I am much more interested in the flexibility to try new things quickly than anything right now. And you can start talking about what it DOES instead of what it will do. You really can't tell from paper alone how well something goes.

Getting to coding already is good advice, bit not always easy for me. I tend to get nervous, and put things off in hopes of finding a better solution. As for my toolchain, I intend to write those myself; am currently bootstrapping my assembler, Assiah, in R6RS Scheme, and will do the same for the bootstrap version of Thelema, but as soon as I get a working version I intend to re-write it to be self-hosting.

_________________Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTFμή εἶναι βασιλικήν ἀτραπόν ἐπί γεωμετρίανLisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.

Oh, and I've reconsidered the pattern-matching syntax, realizing that I could eliminate the need for a special Pattern type if I make two extensions to the Parameter-List type literal (yes, parameter lists as a first-class domain in Thelema). By adding a pseudo-parameter keyword, ":article", and an ellipsis syntax to the parameter fields, and combining that with the existing plan to use signature matching in dispatch (i.e., all procedures in Thelema are the equivalent of a generic in Common Lisp), I can add pattern matching to any group of homonym procedures, not just macros. I had a eureka moment about this yesterday, and while I am still considering the implications of this, it seems an more elegant solution to this issue.

_________________Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTFμή εἶναι βασιλικήν ἀτραπόν ἐπί γεωμετρίανLisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.

Earlier today, I realized that I was keeping too much of the design of Thelema, Kether, and other things related to them in my head without really working out the details. I hadn't written any notes on the matter in over a year, during which time I have changed a lot of my ideas around, and clarified a lot more, and I noticed just how jumbled my thoughts were getting.

It seems you haven't defined the GOAL.

Why do you need a new language? What it should do for you? Why existing languages are bad for you? Do you know the way your language will employ to do things for you? Will the way reward you for the efforts required? Is the way realistic? Really?

The goal of "educating myself" doesn't go here. The important part of the education process is the learning how to define a good goal.

_________________My previous account (embryo) was accidentally deleted, so I have no chance but to use something new. But may be it was a good lesson about software reliability

I gather that liberal education, basic research, and the thrill of discovering something new is of no interest to you, then? I am an academic, at heart if not (yet) by profession; I don't really care for the applications as long as I learn something no one else has known before, or proven that something works when few people think it will.

Goals, you say? Fine:

Demonstrate that an s-expression language can be compiled efficiently and produce code that does not require elaborate runtime support, making it is suitable for system-level development on stock hardware.

Develop a workable means to extend the compiler to allow special-case constructs and fine-grained CPU-specific code generation to be smoothly integrated into the compiler as language libraries, without needing to change the compiler itself and in such a way that the specializations can go into and out of scope during the course of the compilation.

To integrate code meta-data such as version identifiers, release stages, and hypertextual code documentation into the compilation process, so that it can manage things such as optimizations, warnings and documentation consistency in a structured fashion.

Develop a language with minimal intrinsic data structures and control structures that can nonetheless permit higher-level abstractions.

Develop a lisp-like language with a modern module system that is scalable without being cumbersome.

Develop a library for said language that covers most of the common use cases, or at least set in place a framework comparable to CPAN or PyPi to allow for collaborative library development.

To provide an efficient combination of Ahead-Of-Time and Just-In-Time compilation for automatically fetched networked code, such that only the parts of the code expected to be used get transferred over the network.

Its a lot, and that's just some of my goals for just the compiler. Now do you see why I don't want to commit to a set of specific goals or let my ambition wander any further?

_________________Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTFμή εἶναι βασιλικήν ἀτραπόν ἐπί γεωμετρίανLisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.

I gather that liberal education, basic research, and the thrill of discovering something new is of no interest to you

It is of interest, but the time is missing. In an ideal world I wouldn't be bothered with the time, but the world is just a bit too far from the ideal.

Schol-R-LEA wrote:

I am an academic, at heart if not (yet) by profession; I don't really care for the applications as long as I learn something no one else has known before, or proven that something works when few people think it will.

Academicians are welcome

Schol-R-LEA wrote:

Demonstrate that an s-expression language can be compiled efficiently and produce code that does not require elaborate runtime support, making it is suitable for system-level development on stock hardware.

Develop a language with minimal intrinsic data structures and control structures that can nonetheless permit higher-level abstractions.

Develop a lisp-like language with a modern module system that is scalable without being cumbersome.

It's about Lisp. As I understand it's just the language of your choice.

I see a language as a subordinate entity in the pair language-compiler. It is possible to express anything using the majority of modern languages and the difference is often about some personal preferences. Some parts of a language can be convenient for a particular task but no language is convenient for all tasks. It leads me to the conclusion about the leading role of the compiler. I need some means to communicate with the compiler and the means are called "language". Some "means" are better for some tasks but the time required to study all the means available doesn't pays for the advantages gained. So, I see the compiler as a main advantage producer. I can express my thoughts in Java (just because I used to it a bit more than to other languages) and next let the compiler to optimize my thoughts. You can express the same in the Lisp, but next step is always about compiler. In some sense the compiler is a magic wand that can help us be creative and productive. And there's no such wand in the language field.

So, for me the language is not very important and some personal preferences can play here at full speed.

Schol-R-LEA wrote:

Develop a workable means to extend the compiler to allow special-case constructs and fine-grained CPU-specific code generation to be smoothly integrated into the compiler as language libraries, without needing to change the compiler itself and in such a way that the specializations can go into and out of scope during the course of the compilation.

To integrate code meta-data such as version identifiers, release stages, and hypertextual code documentation into the compilation process, so that it can manage things such as optimizations, warnings and documentation consistency in a structured fashion.

To provide an efficient combination of Ahead-Of-Time and Just-In-Time compilation for automatically fetched networked code, such that only the parts of the code expected to be used get transferred over the network.

This part is about compiler.

Easily extensible compiler is a good goal. But it also is the hard goal. I'm trying to grasp it while time allows, but still not very close to the good outcome.

And for the metadata issue I can see no problem in existing languages. The way of annotating your code is straightforward and clear. The only problem here is the enforcement of some rules for the compiler to prevent my laziness from overcoming my productivity. Here is the place for language constructs that allow the compiler to enforce the rules. But even in existing languages there's no much trouble about the enforcement because I can define a framework using inheritance or visibility means that are built in many languages. So, the metadata issue looks as a minor problem for me.

The JIT and AOT issue is mostly about the first goal of the flexible and extensible compiler. If you have such compiler then it's just a matter of some performance measuring to decide what part can be run when. Generally the three stages of the compilation exist - after code is written (development phase), after code is at the destination machine (installation phase) and after the code is running (execution phase). All you need is to measure the time required for the compilation stages to be achieved. Next you can divide the stages across the phases of the code distribution.

So, the most important goal is the compiler itself. Flexible, extensible and easy to understand. Even if it is too slow or the generated code is not very performance worthy - it's just a matter of evolution to improve this part.

Schol-R-LEA wrote:

Develop a library for said language that covers most of the common use cases, or at least set in place a framework comparable to CPAN or PyPi to allow for collaborative library development.

Collaborative software (like git or SVN or CVS) is usually not part of a language. And why should it be? If it wasn't about attracting developers then it looks a bit redundant for academic study.

Schol-R-LEA wrote:

Now do you see why I don't want to commit to a set of specific goals or let my ambition wander any further?

Now I see that your real goal is the compiler development

In fact with a good compiler development of an OS is very easy. So, I join your way of OS development

_________________My previous account (embryo) was accidentally deleted, so I have no chance but to use something new. But may be it was a good lesson about software reliability

The thing is, those are just some of the goals related to the compiler and the language; I am not sure how to express all of the others clearly. I have equally ambitious ideas regarding the OS kernel, the hypervisor system, the development environment, the document system, and the user environment, some of which are equally vague.

I am trying, with little success so far, to focus on the immediate task at hand (which right now is writing a simple XML parser so I can read the XML reference sheets at http://ref.x86asm.net/ into my assembler; while I could probably use an existing library like SSAX, I will need to eventually write one for Thelema anyway, so this is good practice), but my lack of focus has been getting the better of me. Since I am currently unemployed at rather at loose ends, I suffer from a very odd problem: too much time on my hands with too little external structure, and not enough discipline to force my own structure on things. I seem to be improving, but at a glacial pace.

When my mother, brother, and sister-in-law insisted that I apply for Social Security Disability for my psychological issues, and Jim offered to support me while the application was being processed, I agreed in part because I knew that the appeal process can take years, and I convinced myself that there was a good chance that the time away from working would give me the chance to heal, transition, and get better control over my depression and anxiety. With a naiveté someone in their late forties should not fall victim to, I assumed I would be better before the application was actually approved. At worst, I thought I would be on disability for a year or two, and would eventually get my life back on track.

Now I am not so sure if that's ever going to happen.

I hate the idea of being supported by either family or the government - as if I hadn't been often enough already - but the truth is, I am a mess, and I am fooling myself if I think I can actually accomplish anything. I keep telling people that they shouldn't support me, that it would be better for everyone if I were out of their lives permanently, but I can't bring myself to act on that obvious fact or push them away hard enough that they can protect themselves from the disasters I inevitably cause. It makes me feel weak, and fills me with guilt for the trouble I am putting Jim and Elise to over my worthless carcass. I appreciate the effort, but I can't help feeling I am not worthy of it.

_________________Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTFμή εἶναι βασιλικήν ἀτραπόν ἐπί γεωμετρίανLisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.

Note to self: look into incorporating capabilities into qajects, or even implementing capability sharing as qaject passing. By doing this it may be possible to make capabilities an active rather than passive system, while simultaneously providing a simple mechanism for limiting the capabilities themselves (because the capability itself would be the code for accessing the shared resource, and no general access code would exist).

Need to understand both quajects and capabilities better before I can be sure, though.

_________________Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTFμή εἶναι βασιλικήν ἀτραπόν ἐπί γεωμετρίανLisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.

I decided to go with XML/XSLT to describe my syntax and transform that into my code required to parse.

I totally regret spending so much time in the design phase and working on paper. I also regret trying to code in C++ from the get go. I ditched both of these things for off the shelf tools and tools that were very forgiving, Current compiler is written in Javascript, XML and XSLT. - I am much more interested in the flexibility to try new things quickly than anything right now. And you can start talking about what it DOES instead of what it will do.

I went with C#, XML and XSLT, myself. Although the C# part is pretty trivial -- it pretty much just runs the XSLT transforms, calculates memory locations, and formats the binaries correctly.

I do want to, at some point, come up with a "higher level" language that can be converted to XML and then transformed with XSLT, but I haven't gotten that far, yet. Right now, I'm just typing in XML in the Visual Studio editor.

I would be interested in seeing what your "language" looks like, though.

_________________Project: OZoneSource: GitHubCurrent Task: DOSBox Compatibility"The more they overthink the plumbing, the easier it is to stop up the drain." - Montgomery Scott

I have equally ambitious ideas regarding the OS kernel, the hypervisor system, the development environment, the document system, and the user environment, some of which are equally vague.

But first you need to concentrate on something single and manageable. Else no idea will be close to it's implementation in your life.

Schol-R-LEA wrote:

I am trying, with little success so far, to focus on the immediate task at hand (which right now is writing a simple XML parser so I can read the XML reference sheets at http://ref.x86asm.net/ into my assembler; while I could probably use an existing library like SSAX, I will need to eventually write one for Thelema anyway, so this is good practice), but my lack of focus has been getting the better of me.

Ok, it's the compiler's project part and it means you are trying to concentrate, but tedious work is bothering you. May be it is possible to invent a way that doesn't bother you so much? Try something like parser generator which is given a EBNF and outputs a parser ready to use. It means you can replace the tedious part with some higher level thinking that has more fun in it.

Schol-R-LEA wrote:

Since I am currently unemployed at rather at loose ends, I suffer from a very odd problem: too much time on my hands with too little external structure, and not enough discipline to force my own structure on things.

You can report your progress here and we will laugh when the progress is too small (it's, of course, to motivate you)

Schol-R-LEA wrote:

I hate the idea of being supported by either family or the government

If you drain your family then it's really bad. But if you use the government for your best then it's just very good. The government is a bunch of corrupt self serving morons and using their money is not really worse than taking a cup or two from a river - there's plenty of water and no fish will ever notice the missed cup of water.

But there can be another problem - you just do not want to perform the steps required to get the money from government. Then you can remember that the alternative is to drain your family and it is very, very, very bad. Yes, it's a kind of negative motivation.

Schol-R-LEA wrote:

but the truth is, I am a mess, and I am fooling myself if I think I can actually accomplish anything.

At least you are capable of reading, understanding and writing interesting posts here. So, you're not a complete mess

Schol-R-LEA wrote:

but I can't help feeling I am not worthy of it.

The government waits for you. They just buy the votes with their wellfair money and it's absolutely normal to take some bucks from those bastards.

_________________My previous account (embryo) was accidentally deleted, so I have no chance but to use something new. But may be it was a good lesson about software reliability

Reading deeper into the Synthesis papers, I see that Dr. Massalin seems to have considered to use of quaject-based capabilities, and even mentions C-lists - a technique used in many implementations of capabilities - as an analogy to one of the security mechanisms used in Synthesis. From this, and some other things, I suspect that Synthesis did indeed implement capabilities, and did so by way of quajects. I intend to keep reading to see if I am correct in this conclusion.

Also, the system of hierarchical Synthetic machines has me reconsidering the use of a separate hypervisor, or rather, I can now see that the hypervisor should also be the root s-machine, and paravirtualize the subordinate s-machines. If I can find a suitable way to have subordinate s-machines run client operating systems paravirtualized, and allow for folding or snapping of paravirtualized services, this would allow for a hierarchy in which individual synthesis machines can run entire client OSes without involving the root s-machine, which would improve isolation of client operating systems with minimal loss of efficiency. Whether this is feasible will have to be determined later.

_________________Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTFμή εἶναι βασιλικήν ἀτραπόν ἐπί γεωμετρίανLisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.

More OS related stuff, though relevant to the compiler design: in the main paper, Massalin states that since asynchronous context switches (e.g., interrupt handlers) cannot capture the pre-empted process's state, they must save all state, while yields (sleep(), IPC, I/O waits) are within the context and so can capture it. But what if there was a way of maintaining a state record on a per-block level that the interrupt handler or scheduler can trap when the asynchronous pre-emption occurs? If the data structure is incrementally updated with the entry into any lexical block which uses additional context (say, as part of process of generating a lambda with a non-empty parameter list, which would propagate through any type of lexical block macro that generates a lambda), in a way that the interrupt handler can always find and apply it, and if the processing of this structure is faster than the equivalent mass context save, it might be worth doing. I'll need to remember this and perform that experiment some day, or at least see if there are any compelling reasons it would not be effective.

EDIT: This idea is probably moot, as further reading correctly points out that the relevant saving optimizations is not in knowing the state needed by the interrupted process, but that used by the interrupt itself. In retrospect, this should have been obvious, but it seems like the sort of misunderstanding that is pervasive in OS design, and programming in general.

_________________Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTFμή εἶναι βασιλικήν ἀτραπόν ἐπί γεωμετρίανLisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.

Massalin states that since asynchronous context switches (e.g., interrupt handlers) cannot capture the pre-empted process's state, they must save all state, while yields (sleep(), IPC, I/O waits) are within the context and so can capture it.

I understand the words but miss the meaning. Why interrupt handler can't capture anything about the process? Just read registers and follow the pointers from the process's root structure.

_________________My previous account (embryo) was accidentally deleted, so I have no chance but to use something new. But may be it was a good lesson about software reliability

Important files* Makefile - I use gedit, and use the build shortcut to transform my XML into Javascript parser. (F8)* js-transform.xsl - converts the XML nodes into a parser (constructs a parse tree, a concrete syntax tree). * design.xhtml - The syntax definition of the language. What statements are valid.* test.html - a bodged editor to test the parser.

My language "word/token types" are numbers, sign's, identifier's and selector's. If thats what your language needs too, you can just copy the above files, replace "design.xhtml" with your own definition and press F8.

Who is online

Users browsing this forum: No registered users and 1 guest

You cannot post new topics in this forumYou cannot reply to topics in this forumYou cannot edit your posts in this forumYou cannot delete your posts in this forumYou cannot post attachments in this forum