I didn't wanted to reinvent the wheel, that's why I studied already
available package managers. Well, after trying out several of these, I
decided to give NPM.js, the NODE.js package manager a more decent try. The
advantage of reusing an already existing package manager is obvious, they
are well established, work flawlessly on a couple of platforms, have a good
documentation, broad acceptance and large communities behind.

Also, with NPM.js, it's also possible to specify dependencies to a registry
and at the same time to private git repositories. Each dependency (including
the project itself) has a semantic version, which can be used to reason
about compatible dependency versions. This all is amazing when building up
complex projects.

Okay, right now, I work on some examples and already released some modules
in the registry itself. For those of you, already familiar with publishing
own modules in NPM.js can do this already. This means the registry is free
for all and without any restrictive control. Others should wait for the
documentations... Anyway, just feel free to try it out, give feedback,
comments, drawbacks, improvements... For me, its just an idea I wanted to
try out and share with you.

Re: A Package Manager for OpenSCAD

It looks interesting, and I think that a decent package manager would be a good addition to the openscad community. As you note, there have been a couple of attempts in the past, but they've not made it past the proof of concept stage.

I'm not really familiar with npm. How do libraries make it into the registry? If my openscad project makes use of a library that is currently hosted on Thingiverse, is there a way to fetch that as a dependency?

For an openscad package manager to take off, either there needs to be a fair bit of initial work getting a bunch of libraries into the registry to kick things off (and probably on an ongoing basis), or it needs to have the ability to fetch semi-random dependencies where people haven't taken the time to tidy things up for the registry, e.g. references to libraries residing in thingiverse and random openscad libraries stored in github projects (presumably the latter would associated with particular tags/commits, not sure if you could do anything about versioning those on thingiverse).

I didn't wanted to reinvent the wheel, that's why I studied already
available package managers. Well, after trying out several of these, I
decided to give NPM.js, the NODE.js package manager a more decent try. The
advantage of reusing an already existing package manager is obvious, they
are well established, work flawlessly on a couple of platforms, have a good
documentation, broad acceptance and large communities behind.

Also, with NPM.js, it's also possible to specify dependencies to a registry
and at the same time to private git repositories. Each dependency (including
the project itself) has a semantic version, which can be used to reason
about compatible dependency versions. This all is amazing when building up
complex projects.

Okay, right now, I work on some examples and already released some modules
in the registry itself. For those of you, already familiar with publishing
own modules in NPM.js can do this already. This means the registry is free
for all and without any restrictive control. Others should wait for the
documentations... Anyway, just feel free to try it out, give feedback,
comments, drawbacks, improvements... For me, its just an idea I wanted to
try out and share with you.

Re: A Package Manager for OpenSCAD

What would be nice to avoid, is where one person decides they
don't want to have their package published any more and it breaks
all its descendants... (something similar happened in the
javascript world not so long past I believe) Its all right
drinking from an open source, as long as you are sure what people
are putting into the river up stream...

This would tend to suggest some kind of archived versioning, so
people use dependencies of a specific version and upgrade their
dependency versions as they maintain their package.

I'm not sure if there is something "off the shelf" that could do
this, but it would make the whole ecosystem much more robust...

On 04/03/18 01:40, Len Trigg wrote:

It looks interesting, and I think that a decent package manager would be a
good addition to the openscad community. As you note, there have been a
couple of attempts in the past, but they've not made it past the proof of
concept stage.
I'm not really familiar with npm. How do libraries make it into the
registry? If my openscad project makes use of a library that is currently
hosted on Thingiverse, is there a way to fetch that as a dependency?
For an openscad package manager to take off, either there needs to be a
fair bit of initial work getting a bunch of libraries into the registry to
kick things off (and probably on an ongoing basis), or it needs to have the
ability to fetch semi-random dependencies where people haven't taken the
time to tidy things up for the registry, e.g. references to libraries
residing in thingiverse and random openscad libraries stored in github
projects (presumably the latter would associated with particular
tags/commits, not sure if you could do anything about versioning those on
thingiverse).
Cheers,
Len.
On 28 February 2018 at 20:31, Mindfab [hidden email] wrote:

Hey guys,
I know, I'm not the first and maybe also not the last one... but here it
is,
a package manager for OpenSCAD. A Quickstart is described under the
following link... https://github.com/RobertFach/Openscad-Modules<https://github.com/RobertFach/Openscad-Modules>
I didn't wanted to reinvent the wheel, that's why I studied already
available package managers. Well, after trying out several of these, I
decided to give NPM.js, the NODE.js package manager a more decent try. The
advantage of reusing an already existing package manager is obvious, they
are well established, work flawlessly on a couple of platforms, have a
good
documentation, broad acceptance and large communities behind.
Also, with NPM.js, it's also possible to specify dependencies to a registry
and at the same time to private git repositories. Each dependency
(including
the project itself) has a semantic version, which can be used to reason
about compatible dependency versions. This all is amazing when building up
complex projects.
Okay, right now, I work on some examples and already released some modules
in the registry itself. For those of you, already familiar with publishing
own modules in NPM.js can do this already. This means the registry is free
for all and without any restrictive control. Others should wait for the
documentations... Anyway, just feel free to try it out, give feedback,
comments, drawbacks, improvements... For me, its just an idea I wanted to
try out and share with you.
Cheers, Mindfab aka Robert
--
Sent from: http://forum.openscad.org/
_______________________________________________
OpenSCAD mailing list
[hidden email]http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

This simulates a namespace and works perfect with npm. It also has another
advantage which is nested dependencies. That means your modules can require
dependencies in different versions. NPM takes care of all of this.

Re: A Package Manager for OpenSCAD

You are right, this is a serious problem, but it can be dealt with. Right
now, I don't see it to serious, but the registry can be tuned in the way
that published versions cannot be deleted. This will be anyway a good
choice, because in semantic versioning (I think this is what you are
refering to by archived versioning) a published version should never be
changed. I'm not 100% sure how its implemented in the registry that I is
currently used.

Re: A Package Manager for OpenSCAD

Publishing is pretty straight forward. If you are already familiar with NPM
you can do this already. Otherwise, just give me some time to complete the
more complex examples. In principle it works like that. You register or
login to the registry by choosing a username/password combination. Once you
are logged in, you can publish packages if you are the owner of a package or
if its a new package. Packages are identified by the "name" and "version"
fields.

Right now its not possible to use libraries from thingiverse. I also think
that it would be better to motivate the thingiverse library authors to move
to the package manager or at least use github as a repository. With NPM it's
already possible to use github as a package source. Using Thingiverse as a
source would require to modify NPM itself, this I would like to avoid. (NPM
supports tarballs over http - but this would only help with a small
percentage of the libraries on thingiverse).

My roadline was as follows:
1) Completing the documention and more examples
2) Asking thingiverse and github OpenSCAD library authors to add their
libraries to the registry
3) Move some libraries, where the authors did not respond, to the registry
by cloning them on github and adding a package.json file.
4) Iterating through 1,2,3,4

What do you mean by fetching semi-random dependencies?

My personal workflow already changed a lot. Beforehand I few repositories
with a couple of modules in different versions. Now, I have a lot of
repositories where my modules are stored in different versions. A new
project is always started as a new repository and with a package.json file
refering to the required dependencies. This way, my projects can always
reuse artifacts/modules from other projects and its pretty easy to increase
complexity by requiring already existing modules/projects.

I think things will be more clear with more examples. Meanwhile, just have a
look at NPM.js to get a feeling of what is possible.

Re: A Package Manager for OpenSCAD

On 03/04/2018 07:04 PM, Mindfab wrote:
> Thx for your feedback.
>
> Publishing is pretty straight forward. If you are already familiar with NPM
> you can do this already. Otherwise, just give me some time to complete the
> more complex examples. In principle it works like that. You register or
> login to the registry by choosing a username/password combination. Once you
> are logged in, you can publish packages if you are the owner of a package or
> if its a new package. Packages are identified by the "name" and "version"
> fields.
While pipenv sounds like a parallel tool compared to the one you are
proposing, it might be useful to see what the developers of pipenv are
doing.
Descriptive article:
https://opensource.com/article/18/2/why-python-devs-should-use-pipenv

Right now this level of thinking is above my head, I am glad to see
people working on ideas like this.

Re: A Package Manager for OpenSCAD

I have come across this thread after having an idea to use npm to manage
packages for OpenSCAD :) I already use NPM to store some NodeJS modules and
feel that OpenSCAD is missing such central modules repo. I am going to give
it a try and publish some modules I use in my projects. Can I somehow
register?

Re: A Package Manager for OpenSCAD

With NodeJS when you have project dir project1 and run
npm install some_module

inside, the module copy comes from central repository to
project1/node_modules/some_module

So whe I import some_module from JavaScript source file by name like:
require ('some_module');

NodeJS runtime would search ./node_modules/some_module to resolve this
dependency (even more, if it doest not find requested module in project dir
./node_modules, it would go one level up to ../node_modules, then one more
level up to ../../node_modules etc - so with this mecanics some module deps
can be shared among multiple projects).

This logic should be implemented on the level of build/runtime tool (NodeJS
or OpenSCAD), not on the level of package manager.

Of course one cane import module like
require ('./node_modules/some_module');

Re: A Package Manager for OpenSCAD

With npm, it’s possible to do ‘npm install --prefix <someprefix>’ to install into any location
For global installs, setting the prefix to OpenSCAD’s global library folder would work automatically.
For local installs, we could combine this with setting the OPENSCADPATH environment variable.

Re: A Package Manager for OpenSCAD

1i7 wrote
> I have come across this thread after having an idea to use npm to manage
> packages for OpenSCAD :) I already use NPM to store some NodeJS modules
> and
> feel that OpenSCAD is missing such central modules repo. I am going to
> give
> it a try and publish some modules I use in my projects. Can I somehow
> register?

That would be great!
You can register yourself by using npms adduser command against the
openscad-modules registry. More details can be found in the github
repository.

Re: A Package Manager for OpenSCAD

> Also, good module repository support might require better build logic
> integration in OpenSCAD itself.
>
> With NodeJS when you have project dir project1 and run
> npm install some_module
>
> inside, the module copy comes from central repository to
> project1/node_modules/some_module
>
> So whe I import some_module from JavaScript source file by name like:
> require ('some_module');
>
> NodeJS runtime would search ./node_modules/some_module to resolve this
> dependency (even more, if it doest not find requested module in project
> dir
> ./node_modules, it would go one level up to ../node_modules, then one more
> level up to ../../node_modules etc - so with this mecanics some module
> deps
> can be shared among multiple projects).
>
> This logic should be implemented on the level of build/runtime tool
> (NodeJS
> or OpenSCAD), not on the level of package manager.
>
> Of course one cane import module like
> require ('./node_modules/some_module');
>
> But this would not be so elegant.

Well, one point we should consider when thinking about this optimization is
the problem of name clashing and versioning conflicts. If you install all
modules in a common location the user would need to handle conflicting
module versions and also naming issues. In my opinion, the advantage of
having a modular systems, where I (as a module author and consumer) only
have to take care about my own dependencies (and not the one somebody else
is using in combination with my module) outperforms the advantage of saving
space. So, using "include <node_modules/fractals/fractals.scad>" inside a
module, as its described in the simple example, does this trick. Maybe it's
not a real feature of OpenSCAD, but it works and gives us the isolation for
naming and versioning issues :)

Beside that, I agree, that the look and feel is not as good as it could be.
The 'node_modules' part is just in the path because I didn't want to add
another command line parameter to the npm call. Maybe there is a better
solution that has a better presentation/usability but preserves the above
mentioned property?

Re: A Package Manager for OpenSCAD

>>
>> Of course one cane import module like
>> require ('./node_modules/some_module’)
>
> With npm, it’s possible to do ‘npm install --prefix
> <someprefix>
> ’ to install into any location
> For global installs, setting the prefix to OpenSCAD’s global library
> folder would work automatically.
> For local installs, we could combine this with setting the OPENSCADPATH
> environment variable.
>
> For more into, see
> https://en.wikibooks.org/wiki/OpenSCAD_User_Manual/Libraries>
> -Marius

Hi Marius, thx for your feedback. I just answered 1i7 regarding the
suggestion to install modules globally... Right now, I think there is one
big advantage in installing the modules in the NPMs preferred hierarchy
under a local folder, whićh is, it helps us to avoid naming and versioning
issues. A more detailed explanation can be found in the reply to 1i7
question.

In my opinion having this ability can speed up the development of modules,
as module authors itself doesn't need to be concerned about the context of
their module usage, they just need to be concerned about their module and
their dependencies.

Re: A Package Manager for OpenSCAD

I believe the existing of a standard package manager will prompt owners to
write libs with specific names.

Gadgetmind wrote

> On 04/03/18 01:40, Len Trigg wrote:
>> I think that a decent package manager would be a good addition to the
>> openscad community.
>
> While I don't disagree, I've been bitten more than once by clashing
> names within libraries. These cause mayhem that's *very* hard to debug.
>
> Namespaces, or at least a convention on naming within libraries to avoid
> these issues, would soon turn out to be pretty important IMO.
>
>
>
> _______________________________________________
> OpenSCAD mailing list

Re: A Package Manager for OpenSCAD

I worried about this quite a bit while working on scad_bundler (a
different package manager).

This is actually less of an issue than it appears if libraries are
built well. See the Conventions listed on the github page for
scad_bundler, the 3rd bullet and later is meant to address this and
it seems to work pretty well (so far) in practice.

I believe the existing of a standard package manager will prompt owners to
write libs with specific names.
Gadgetmind wrote

On 04/03/18 01:40, Len Trigg wrote:

I think that a decent package manager would be a good addition to the
openscad community.

While I don't disagree, I've been bitten more than once by clashing
names within libraries. These cause mayhem that's *very* hard to debug.
Namespaces, or at least a convention on naming within libraries to avoid
these issues, would soon turn out to be pretty important IMO.

Re: A Package Manager for OpenSCAD

I was trying to illustrate that user.scad doesn't know about
priv_mod() at all - it's scope ends in prublic.scad, if you will.
But what that means is we can have more than one priv_mod() in
different files, so long as the same file doesn't use/include them
both directly.

I added a public2.scad and private2.scad. Both private.scad and
private2.scad have a module named priv_mod(), but each can only be
called from the file that use'd them. And neither leaks into
user.scad's namespace.