Ack... I obviously am just not understanding packages. I look at howthe maths lib is written and it "exports" most of the things (liststhem in L). But that means (sign) is defined. So if I also use someother library that defines a function (sign)... there's a nameconflict which I thought package was supposed to help avoid. So... I'mconfused.

Here's some more evidence of my confusedness. I'm using the clojureport. I put the following in a file:

It mentions that internal symbols are "hidden", but I can still accessthem by including the package prefix on them, so how are they"hidden"?

I look at how the maths lib is written and it "exports" most of thethings (lists them in L). But that means (sign) is defined. So if Ialso use some other library that defines a function (sign)... there'sa name conflict which I thought package was supposed to help avoid.So... I'm confused.

Regarding the exported function symbols, you can, if you wish,
guarantee that they will not be overwritten by giving them the status
of system functions which will cause Shen to raise an error if any
future definition overwrites them.

How can I access both versions of commonName? REPL session belowshowing how loading the second overwrites the first.

Perhaps I need to take a step back and ask: if I'm writing a packagein a file and it needs to depend on various libraries, what is theproper way of importing them?

I could see how one could possibly call (load filename), magicallyfigure out what package it just loaded, use (external pkgname) on thatand prepend a qualifier to all those symbols selectively maybe, butthat would need to run at compile time...

As said, if you want to guarantee your external symbols cannot be
overwritten, you need to systemf them.

In Shendoc it says

The philosophy of Shen is that once a programmer has decided a symbol
is internal to a package and is hidden from view that decision cannot
be overridden except by changing the definition of the package. Hence
the complexities of IMPORT and EXPORT found in Common Lisp are not
reproduced in Shen. You cannot be 'in' a package in Shen within the
REPL. It is possible to declare a package in a package.

Symbols that are declared external are supposed to be freely usable by
all packages.

Mark

On Apr 5, 3:50 pm, Joel Shellman <j...@mentics.com> wrote:
> Here's an example. Let's say I have two libraries that I want to use.
> So, I load them:
>
> Test1.shen:
> (package mypackage1- [commonName]
> (define commonName -> (output "1"))
> )
> Test2.shen:
> (package mypackage2- [commonName]
> (define commonName -> (output "2"))
> )
>
> How can I access both versions of commonName? REPL session below
> showing how loading the second overwrites the first.
>
> Perhaps I need to take a step back and ask: if I'm writing a package
> in a file and it needs to depend on various libraries, what is the
> proper way of importing them?
>
> I could see how one could possibly call (load filename), magically
> figure out what package it just loaded, use (external pkgname) on that
> and prepend a qualifier to all those symbols selectively maybe, but
> that would need to run at compile time...
>

> Symbols that are declared external are supposed to be freely usable by
> all packages.

I should have said 'Symbols that are declared external are supposed to
be freely usable by the REPL or in any unpackaged program'. If you
use external symbols S of package A inside package B you have to
specifically say in package B that you want these symbols exported in
the package B. The 'external' function is there to help you recover
those symbols i.e. S = (external A).

> As said, if you want to guarantee your external symbols cannot be> overwritten, you need to systemf them.

But that's not what I asked. I want to know how I can access both atthe same time.

> Symbols that are declared external are supposed to be freely usable by> all packages.

But what if they conflict? Thousands of people making variouslibraries are going to export the same symbol name because they areintended to be used. And when I need to use both libraries at the sametime, how do I do that?

Well, if you make a function external to a package, meaning that you
are allowing people to refer to it without qualification; and somebody
else does the same then either you will get an overwrite or a systemf
error if you try to load both. There is no possibility of being able
to access two different definitions of the same symbol at the same
time. I'm not sure what this could mean actually.

On Thu, Apr 5, 2012 at 12:29 PM, Mark Tarver <dr.mt...@gmail.com> wrote:> Well, if you make a function external to a package, meaning that you> are allowing people to refer to it without qualification; and somebody> else does the same then either you will get an overwrite or a systemf> error if you try to load both. There is no possibility of being able> to access two different definitions of the same symbol at the same> time. I'm not sure what this could mean actually.

maths already exports all its functions. So, if that convention isfollowed, there are going to be a lot of conflicts.

Is this only a problem in the REPL or unpackaged program? Youmentioned something about how a package could access other packages ina qualified way and so a packaged program could access both at thesame time?

How does one package state that it depends on another package so thatother package will be loaded and it's symbols become available (whenwe're NOT working at the REPL)?

In short, I guess my question is: how does one write and organize anapplication in Shen, like when I'm writing an application that mayhave 10's or 100's of packages that have various dependencies witheach other and other libraries.

For that matter, is there even a way to run Shen outside the REPL yet.Sorry... I read every page under Learning Shen and at least most ofthe Shen Standard doc, if there is something else I should readthrough that answers these questions, let me know.

If your package P requires other packages to function then you need to
place in your code and before the package itself; a list of files that
you expect the system to load. If you want to refer to the external
symbols of these packages in P, then you need to have something like

You can't access two different symbol definitions of one symbol if
they have both been made external. If they are internal to each
package, no problem.

You cannot as yet run Shen outside the REPL though you can use the
dump function to dump native code to a file which can be run outside
the REPL if the native code allows it. Ramil's JS port actually does
not allow this because there is a JS restriction about writing to
files based (I guess) on client security. Hakan's Clojure port does
allow this - at least I seem to remember that his later version did,
which will be be only days old.

> If your package P requires other packages to function then you need to> place in your code and before the package itself; a list of files that> you expect the system to load. If you want to refer to the external> symbols of these packages in P, then you need to have something like>> (load "P1.shen")

Let's say P1 and P2 both have external symbol S. At this point, couldwe prepend all P1's external symbols with "p1-" so when we run:

(load "P2.shen")

it doesn't redefine the prepended symbols and then we prepend all P2'ssymbols with "p2-" and then...

I think the easiest way to understand the Shen package system is to
understand how it works. The approach is very simple. Shendoc says

The polyadic function package has the form (package S L E1 ... En)
where

1. S is a symbol beginning in lowercase which is the name of a
package; (e.g mypackage).
2. A list L (possibly empty) of non-variable symbols.
3. E1 ... En are a series of Shen expressions.

The Shen reader prepends the package symbol before all the symbols
when evaluating E1 ... En apart from those symbols which are

(a) symbols listed as belonging to the system (such as ->, define,
cons, append etc) or ...
(b) symbols which are variables or ...
(c) ... symbols which are listed in L or .....
(d) ... symbols internal to the Shen package or ....
(e) ... symbols consisting entirely of underscores or entirely of
equals signs.

So ....

> Let's say P1 and P2 both have external symbol S. At this point, could
> we prepend all P1's external symbols with "p1-" so when we run:

The answer is 'no' because the external symbols of P are fixed by the
package P and not by anything outside of P. But you could use an
external symbol of P1 internally in P2 with a totally different
meaning without any clash. The idea is that when a programmer makes
a symbol external he intends to make that function globally accessible
*under the meaning he has given*, and to override his decision is
really to change the package and this should and does require a code
change in the package itself.

>> Let's say P1 and P2 both have external symbol S. At this point, could>> we prepend all P1's external symbols with "p1-" so when we run:>> The answer is 'no' because the external symbols of P are fixed by the> package P and not by anything outside of P. But you could use an> external symbol of P1 internally in P2 with a totally different> meaning without any clash

I was suggesting that as a proposal of how we might be able to solvethe problem. I need to access a function with the same name in P1 andP2 from within my package P3. If you're saying that's impossible... itmakes no sense.

For example, right now I have a workspace open in which there are 1977methods named "read" across all libraries. The likelihood that at somepoint I'll need to access two of those from the same package is veryhigh. You might suggest they should not be declared external? But thenit's the case of whatever library author "gets there first"? Forexample, maths exports the sign symbol. So that means that all futureusers of Shen should never declare sign to be external or else otherswon't be able to use their package at the same time as the mathspackage?

I'm sorry to keep pursuing this, but this is really not making senseto me. All other languages I have used provide a way to import thingsfrom other packages and deal with conflicting names. So, I'm reallyconfused by the suggestion that that would not be possible.

I see your thinking. You say 'I have a workspace open in which there
are 1977 methods named "read" across all libraries.' You want to have
a tool that enables you to deal with a library like this.

The basic answer is that Shen does not have it because I would regard
it as extremely undesirable to have 1977 functions called 'read'
external across a library. So I guess for me the question is a bit
like 'How does your plane fly when the tanks are empty and a wing has
fallen off?' . The answer is it does not, because you are not
supposed to get into that awful state. Providing support tools for
people to do bad things was not my goal. Library management means
avoiding that mess.

> I was suggesting that as a proposal of how we might be able to solve
> the problem. I need to access a function with the same name in P1 and
> P2 from within my package P3. If you're saying that's impossible... it
> makes no sense.

What I'm saying is that it makes no sense to maintain a library where
this happens because the point of making a symbol external is that you
are saying 'This is unique and can be used freely in the REPL'. I
actually would incline to 'systemf' all external symbols of a package
so that any programmer who did overwrite one definition with another
in loading packages would raise an error and be required to actually
change the code he is loading.

> The basic answer is that Shen does not have it because I would regard> it as extremely undesirable to have 1977 functions called 'read'> external across a library.

It's many libraries--hundreds. And since they are written by differentauthors, there's no way to coordinate who is "allowed" to externalize"read" and who is not. As soon as one person has done so in Shen, hemakes it impossible for anyone else to do it.

> supposed to get into that awful state. Providing support tools for> people to do bad things was not my goal. Library management means> avoiding that mess.

So avoiding externalizing at all? Then why have that feature?

> What I'm saying is that it makes no sense to maintain a library where> this happens because the point of making a symbol external is that you> are saying 'This is unique and can be used freely in the REPL'.

But that is just one library author. There will be thousands oflibrary authors. And as soon as one of them externalizes a name, itmeans that noone else can.

And I'm not talking about the REPL. The REPL is not really interestingto me. It's useful during development, but you don't writeapplications that way, do you? I understand Shen is kind of limited tothe REPL right now, but I'm looking forward to how I can write bigcomplex applications with it. And package/namespace management is acrucial part of any such undertaking.

> I actually would incline to 'systemf' all external symbols of a package> so that any programmer who did overwrite one definition with another> in loading packages would raise an error and be required to actually> change the code he is loading.

In what I'm discussing, you can't change the code you're loading. It'swritten by someoneelse, it's binary, it's not your code. That's like telling me I needto go change the maths library every time I run into a conflict,except I might not even have the source code to the library so itwould be impossible for me to change it even if I wanted to.

On Apr 6, 4:59 pm, Joel Shellman <j...@mentics.com> wrote:
> > The basic answer is that Shen does not have it because I would regard
> > it as extremely undesirable to have 1977 functions called 'read'
> > external across a library.
>
> It's many libraries--hundreds. And since they are written by different
> authors, there's no way to coordinate who is "allowed" to externalize
> "read" and who is not. As soon as one person has done so in Shen, he
> makes it impossible for anyone else to do it.

What you need here is a library management tool, not a package
management tool, to ensure that externalised symbols remain unique
within the library. Not an impossible task but not one that we need be
too unduly worried about at this stage. Yes; if someone takes your
favourite word and externalises it, it can be mildly annoying, but not
generally incapacitating. Surely better than having 1977 meanings of
one symbol pushed to the front and people never being certain in any
application, exactly what one they are talking to.

> So avoiding externalizing at all? Then why have that feature?

Because basic bread-and-butter functions like 'tan' are more usefully
invoked in that form rather than 'maths-lib-tan'. We try to avoid
cluttering up the namespace with auxiliary functions like 'tan-help'
etc.

> In what I'm discussing, you can't change the code you're loading. It's
> written by someone
> else, it's binary, it's not your code. That's like telling me I need
> to go change the maths library every time I run into a conflict,
> except I might not even have the source code to the library so it
> would be impossible for me to change it even if I wanted to.

If the thing that is bothering you is that you want to use 'sign' but
the maths library has used it; then you can get round this by using
macros. You simply define a macro that maps 'sign' to your internal
symbol. If you load the maths library first and then your macro, you
will have all the benefit of the maths library and control over the
'sign' symbol. However anybody using your work and expecting to find
'sign' with the original maths meaning will be surprised no doubt.

On Apr 6, 4:59 pm, Joel Shellman <j...@mentics.com> wrote: > > The basic answer is that Shen does not have it because I would regard > > it as extremely undesirable to have 1977 functions called 'read' > > external across a library. > > It's many libraries--hundreds. And since they are written by different > authors, there's no way to coordinate who is "allowed" to externalize > "read" and who is not. As soon as one person has done so in Shen, he > makes it impossible for anyone else to do it.

What you need here is a library management tool, not a package management tool, to ensure that externalised symbols remain unique within the library.

Why unique "in the library"? It's not about a symbol being unique in one library but across several (if I understood Joel Shellman right).

> So avoiding externalizing at all? Then why have that feature?

Because basic bread-and-butter functions like 'tan' are more usefully invoked in that form rather than 'maths-lib-tan'. We try to avoid cluttering up the namespace with auxiliary functions like 'tan-help' etc.

But one writing on a sunbathing library might need the math library while using his tan function more often.

> In what I'm discussing, you can't change the code you're loading. It's > written by someone > else, it's binary, it's not your code. That's like telling me I need > to go change the maths library every time I run into a conflict, > except I might not even have the source code to the library so it > would be impossible for me to change it even if I wanted to.

The library is open source BSD.

The sunbathing library is proprietary and you cannot change the code.

I thought packaging was there to avoid naming conflicts and exporting was there for convenience. Where is the point when exporting leads to naming conflicts despite packaging and packaging only has any meaning when you don't export symbols?

> Why unique "in the library"? It's not about a symbol being unique in one
> library but across several (if I understood Joel Shellman right).

There's several things being mixed together here. The number of
libraries is not important. And its not true that 'As soon as one
person has [made a symbol external] in Shen, he makes it impossible
for anyone else to do it.' You can, at the moment, overwrite external
symbols. What is important is that in a well organised library, which
is what a standard library is, you do not want this to happen. This
IMO is a question of civilised library management. What happens in
the Wild West is another matter altogether.

> > > So avoiding externalizing at all? Then why have that feature?

As explained; externalising makes sense when you have a symbol like
'tan' with a recognised meaning. So you make it external for ease of
use.

> But one writing on a sunbathing library might need the math library while
> using his tan function more often.

I don't understand this statement.

> The sunbathing library is proprietary and you cannot change the code.

Well arguably if somebody has made their code proprietary and
externalised a function with a recognised meaning perhaps one should
respect their choice and, if using their work, accept their decision.

>
> I thought packaging was there to avoid naming conflicts and exporting was
> there for convenience. Where is the point when exporting leads to naming
> conflicts despite packaging and packaging only has any meaning when you

> don't export symbols?-

Packaging does not eliminate naming conflicts; what it does do is
reduce their probability by hiding auxiliary functions and
(potentially) providing a structured way of dealing with them when
they do arise. I would, for example, favour 'systemf'ing external
symbols in the standard library to make it impossible to overwrite
external functions thus avoiding overwrites.