Unity Technologies

I'm trying to use git: path to my repository but my format is slightly different because we're using our corporate gitlab server:

Code (CSharp):

"git@gitlab.company.com:group-name/package-name.git#branch-name"

differences are that our git prefix for the path includes "@gitlab.company.com" and our repository is not owned by a user its under a gitlab group. We have our SSH keys set up as we normally would have for source tree but despite that, we get an issue saying that Unity doesn't understand the format of our path.

Click to expand...

Package Manager does not SCP pattern for the moment. It is in our plan though. For now, use the SSH scheme.

Thanks for the suggestion guys. This seems to have progressed me but now I'm getting errors saying that permission has been denied. This is usually the same error I get if I ever forget to add my ssh key to the putty agent when using source tree. But my ssh key definitely is assigned to my pageant key list, the repo url is definitely correct, the repo exists and I'm the owner of both the group and the project.

In short, I'm kind of at a loss at what to try next. Any help appreciated!

They use different authentication methods but as far as I know all git servers support both forms so try converting your url to the other form.

Click to expand...

Actually the second form is a variant form of SSH URLs. SSH is a different mechanism to establish the connection with the remote Git server and perform authentication. HTTPS and SSH URLs are not strictly interchangeable, since some configuration changes may be needed.

A note about SSH URLs: the Package Manager supports them, but not in that variant form. It needs to be in the following form:

Code (CSharp):

ssh://git@github.com/Unity-Technologies/ScriptableRenderPipeline.git

Notice that the colon (':' symbol) needs to be changed to a forward slash ('/' symbol).

Thanks for the suggestion guys. This seems to have progressed me but now I'm getting errors saying that permission has been denied. This is usually the same error I get if I ever forget to add my ssh key to the putty agent when using source tree. But my ssh key definitely is assigned to my pageant key list, the repo url is definitely correct, the repo exists and I'm the owner of both the group and the project.

In short, I'm kind of at a loss at what to try next. Any help appreciated!

Click to expand...

I know for us, we used SourceTree which loaded SSH keys from Pageant. SourceTree uses it's own internal git.

The git installed to the system, aka the git used from PowerShell or the Terminal, uses a different SSH key. I had to convert my old key or create a new one and put it at C:\users\UserName\.ssh\id_rsa . This allowed Unity to connect to our git repos, since it didn't have access to SourceTree's internal git.

Thanks @GarthSmith! I did exactly as you said and made sure to add the new public key I made using git gui to my gitlab profile and that the key itself had an empty password. But even when I try to clone the repository from the command line I'm still getting the same issue. It asks for a password and denies me permission even though my repo is public, I'm the owner and I've got my non source tree git all set up.

It likely isn't a Unity issue since I can't clone form the terminal but now I'm even more confused as to what's wrong.

One other thing I noticed as well is that the unity package manager recursively checks out submodules, but that breaks things for me cause I've added other packages I'm currently working on into my main project to easily update them.

Unity Technologies

One other thing I noticed as well is that the unity package manager recursively checks out submodules, but that breaks things for me cause I've added other packages I'm currently working on into my main project to easily update them.

Is there a way to prevent this submodule checkout?

Click to expand...

I'm not sure what you mean about this. The Package Manager only clones the repository, there are no submodule operations performed.

Could you please provide a bit more detail on your setup? Ideally, I suggest Reporting a Bug (in Unity: Help > Report a Bug) with a small repro project where this issue happens, and post the case number here. Thanks!

I'm not sure what you mean about this. The Package Manager only clones the repository, there are no submodule operations performed.

Click to expand...

With Git, cloning does do submodule operations if given the --recursive argument is provided.

I have a main project that has submodules and I've symbolically linked the folders into my project as a sibling of my proper upm package. Next, I perform a split tree git operation on the main package I want downloaded into user's machines which then gets pushed to a upm specific branch where I put my release tags.

But it seems that the Unity Package Manager is cloning the branch recursively which is also pulling in the submodules in projects that are using the main module.

I'm not sure it warrants a bug report as it may be by design, but I'd at least like to be able to have the option of adding a package without it recursively cloning the repository.

argument is passed to the git clone operation. And yes, it turns out that this is by design: the reason for that is that our implementation was initially derived from npm, and keeping the same behaviour would avoid unexpected differences between the Unity Package Manager and npm.

We don't have any plans (yet) to change it at this time since it would be a breaking change, and this limitation can be worked around fairly easily by using submodules outside of packages only, or under folders that are ignored by Unity ('.'-prefixed or '~'-suffixed, for instance). Your workflow seems non-trivial, so maybe the following idea won't help, but you could consider using .gitignore/.npmignore as well to prevent the submodule files from being committed to the split tree repository, or included in the final package.

argument is passed to the git clone operation. And yes, it turns out that this is by design: the reason for that is that our implementation was initially derived from npm, and keeping the same behaviour would avoid unexpected differences between the Unity Package Manager and npm.

We don't have any plans (yet) to change it at this time since it would be a breaking change, and this limitation can be worked around fairly easily by using submodules outside of packages only, or under folders that are ignored by Unity ('.'-prefixed or '~'-suffixed, for instance). Your workflow seems non-trivial, so maybe the following idea won't help, but you could consider using .gitignore/.npmignore as well to prevent the submodule files from being committed to the split tree repository, or included in the final package.

But I did stumble upon an interesting fact. While observing the package download I noticed that it wasn't pulling the specific branch, but instead the master (default) branch first, then switching to the tag/branch specified by the # suffix.

This could be part of the problem we're seeing. Instead if the upm was checking out that specific tag/branch directly, then the submodules won't show up (they're left overs from the branch switch).

This explains why there's no submodule folder in my upm branch, but one appears in the package cache.

I tested this out with a local clone as well just to make sure, and sure enough, the submodlues folder gets left behind if you check out the master (defualt) branch first vs checking out the tag/branch directly.

One other option is to do a hard reset when checking out the specific branch/tag to ensure there's not any left over files, but ideally we should be checking out only the branch/tag specified.

Since it doesn't support subfolders, yet, I tried to use a simple Unity project as a package (just a single script). However, when UPM tries to resolve the package, I'm getting these errors:

Read only asset Packages/com.company.common/Packages has no meta file.
Read only asset Packages/com.company.common/Assets has no meta file.
Read only asset Packages/com.company.common/ProjectSettings has no meta file.
Read only asset Packages/com.company.common/package.json has no meta file.

For anyone trying to setup your library with UPM and struggles with including both the library source and a Unity project to test and add examples to. I struggled for a long time trying to add a package.json to a subfolder in git, but didn't get it work (and I also finally read comments in this thread explaining that it's not implemented / supported yet). I ended up with the following structure in my project:

Repo:

Source

package.json - for Source called "com.mambojambostudios.unity-atoms-src"

UnityProject (containing examples and tests)

Packages/manifest.json - Points to "file:../../Source" as a dependency

I was able to make it work. The workflow that worked for me is to use git submodule and store in a separate folder outside of Assets. I then add as local package the Assets folder of the repository that was added as a submodule. This way, I could still maintain the Unity project of that external common code repository.

Since it doesn't support subfolders, yet, I tried to use a simple Unity project as a package (just a single script). However, when UPM tries to resolve the package, I'm getting these errors:

Read only asset Packages/com.company.common/Packages has no meta file.
Read only asset Packages/com.company.common/Assets has no meta file.
Read only asset Packages/com.company.common/ProjectSettings has no meta file.
Read only asset Packages/com.company.common/package.json has no meta file.

I just hit the same problem... Turns out you can't sync a a unity project as a package (nor can you sync subfolders of a repo), which kind of s*cks. So you need a local project and only sync your Assets or a subfolder of that on Git.

I just hit the same problem... Turns out you can't sync a a unity project as a package (nor can you sync subfolders of a repo), which kind of s*cks. So you need a local project and only sync your Assets or a subfolder of that on Git.

This is not super practical .

Click to expand...

Actually it doesn't really makes sense to sync an entire Unity project, there would be a lot of useless files included on the package.

A good option is to have the desired package content on one repository and have another repository to hold your package development project (if really needed, as developing a package normally means something usable between projects, so any local project should be enough to develop the package), which can then have the package content repository as a submodule (it could be the entire Assets folder if you wanted to, but it doesn't seems a good practice.

Actually it doesn't really makes sense to sync an entire Unity project, there would be a lot of useless files included on the package.

Click to expand...

It does make sense to sync entire Unity projects on Git. I never ever worked on a project where files were not synched on a version control system.

For development purposes, the base project should be shared on the version control system as it is the only way to ensure everyone uses the correct version of Unity and packages.

Anyway, I ended up synching only what's found in the Assets folder, but it would be great to someday be able to do that with full projects as I wouldn't need to setup my packages manifest each time I sync my files on a new machine. Or have a way to "upload" part of a project as a package, the same way the Asset Store works.

Your final example isn't ideal for me as the "package dev project repo" only has one manifest file, and I may be developing modules with each their own set of dependencies.

For instance, I may create a day & night cycle system that works on HDRP, but has a branch for LWRP, which causes that repo to have at least two branches. Once you have branches and an array of disctinct dev environment, your central project repo makes no longer sense.

Oh, and my entire game project is not on Git, but on Plastic SCM. There's 40gb+ of binaries there.

Unity Technologies

But I did stumble upon an interesting fact. While observing the package download I noticed that it wasn't pulling the specific branch, but instead the master (default) branch first, then switching to the tag/branch specified by the # suffix.

This could be part of the problem we're seeing. Instead if the upm was checking out that specific tag/branch directly, then the submodules won't show up (they're left overs from the branch switch).

This explains why there's no submodule folder in my upm branch, but one appears in the package cache.

I tested this out with a local clone as well just to make sure, and sure enough, the submodlues folder gets left behind if you check out the master (defualt) branch first vs checking out the tag/branch directly.

One other option is to do a hard reset when checking out the specific branch/tag to ensure there's not any left over files, but ideally we should be checking out only the branch/tag specified.

Click to expand...

Thanks a lot for your perseverance and for reporting your findings. That indeed seems like a bug, because that's definitely unexpected! I'll add the notes to the bug report.

For instance, I may create a day & night cycle system that works on HDRP, but has a branch for LWRP, which causes that repo to have at least two branches. Once you have branches and an array of disctinct dev environment, your central project repo makes no longer sense.

Click to expand...

Similar here, you could have a core package with shared stuff and have both the HDRP and LWRP packages have dependency on that.

If you need a specific project setup then probably your system is not ideal to be used as a package, it seems more like a base project instead, like UFPS.

I am developing a few packages myself and I didn't found any issues which the workflow I described, I've created only one development project to hold temporary files that doesn't fit into any package and just use others non-versioned projects to actually develop the packages, the only "issue" is that when I go to a new computer I need to create an empty project to start working on some package, but it could be the equivalent to cloning a repo, but much faster.

Actually it doesn't really makes sense to sync an entire Unity project, there would be a lot of useless files included on the package.

A good option is to have the desired package content on one repository and have another repository to hold your package development project (if really needed, as developing a package normally means something usable between projects, so any local project should be enough to develop the package), which can then have the package content repository as a submodule (it could be the entire Assets folder if you wanted to, but it doesn't seems a good practice.

I am working like that with one personal package I am developing and the workflow is pretty natural.

Click to expand...

Agreed, I've actually been using a git subtree split operation that just culls what's in the upm package in a branch dedicated solely for that, so people can still download and open the project, but the releases are much smaller on the upm branch itself.

Sorry for digging this but, I still don't get what's the point to don't just support Git dependencies inside a package manifest. What are the benefit to restrict us not to have this easy automatic deep dependency?

When we tell someone to install our package. If that package also have dependency on another custom package outside of unity. Are there anyway we could resolve deep dependency chain automatically. If it not then will it be?

Unity Technologies

@Thaina , it could be hard to choose the correct Git package dependencies. Remember that only one version of a package can be installed in your project at any moment. If we enable Git package dependency inside a package manifest, it complicates the package resolution. It is, in theory, possible for two packages from a different Git repository to have the same name and version but with different content. There is also the possibility for a package to depends on the registry version of the package and another one on a Git package. There would not be any guarantee of which package will be selected and we will also have to define a way to render the behaviour deterministic.

That said, we are considering letting users *take* this risk if they want to.

@Thaina , it could be hard to choose the correct Git package dependencies. Remember that only one version of a package can be installed in your project at any moment. If we enable Git package dependency inside a package manifest, it complicates the package resolution. It is, in theory, possible for two packages from a different Git repository to have the same name and version but with different content. There is also the possibility for a package to depends on the registry version of the package and another one on a Git package. There would not be any guarantee of which package will be selected and we will also have to define a way to render the behaviour deterministic.

That said, we are considering letting users *take* this risk if they want to.

Regards,

Pascal

Click to expand...

Thank you very much for your clarification. But I was still stand for having custom dependency anywhere

However I also agree that there would be a drawback we need to choose to accept by doing so

If we consider that collision of package name is more important. There was one way I think we could solve it, by having git url as a key and git commit hash as a version instead. This will guarantee that difference content will have no name collision. And I think it would be better, an individual package would have only one way an owner want to distributed anyway. If an owner prefer git or their own server then they would more likely to put it that way

Code (CSharp):

{

"com.unity.xxx":"1.0.0"// normal

"https//github.com/owner/project.git":"abcdefhash"// git

}

This way might have a package dependency resolution a little more complicate that it must test that the key is not url. And then it must check that, which hash have higher version on the server, before resolving

But even without this format. Having custom git dependency like npm would still be better than nothing

@M_R If we need to have editing the any file other than dependencies object in the manifest then I think the custom scope registry already possible

The point to have git dependency is that, we could have anyone only need to add one line of dependency, and package manager will do all the work on resolving dependency. No one need to care for each specific package when they just want to include some feature into the project

The point of having package manager is to have all people conveniently include package of anyone, as easy as possible, if anyone need to go tweak things here and there to make a package can resolve dependency then it would be a failed manager

@okcompute_unity Thanks for reply. I tested this cool feature with 2018.3 and works well.

I have additional two question about this:

1. How to customize "View documentation", "View changelog", "View license" link in Package Maganger window. Currently, it opens the link start with "https://docs.unity3d.com/Packages/" and return 404 error. I want to change this link to my own site.

2. Can I depend other custom package for custom package? (can I use git protocol for that? or should I use private npm repository?)

Unity Technologies

1. How to customize "View documentation", "View changelog", "View license" link in Package Maganger window. Currently, it opens the link start with "https://docs.unity3d.com/Packages/" and return 404 error. I want to change this link to my own site.

Click to expand...

The URL is hardcoded and you cannot change it. The UI team has a story in its backlog to make support alternate doc link in the package manifest. That should cover your use case. Finally, the buttons should now show up when a 404 is returned. This is an issue that is also already in the UI team backlog to be addressed.

Unity Technologies

This is an interesting package (and clever! ). Please note the risk you are taking when using this package:

The package implementation is based on internal Package Manager implementation details (or non-public API). Meaning this implementation can change at any moment to accommodate a new feature or bug fix. Unity *strongly* recommend users doesn't refer to files or data from the

Library

folder.

As far as I can tell, the implementation is non-deterministic. Meaning that if you share your project with someone else, this other user risk to not have the same list of packages installed. (This could probably be fixed by the author by simply sorting the input array).

I would say this is fine to use this package in _hobbyist_ or solo project.

My team is actually studying a way to support this feature and keeping a deterministic dependencies resolution and also selecting the proper Git package conflict resolution algorithm (i.e. When two packages request the same package name from distinct repositories, which one should be used? When two packages rquest the same package version from distinct repository branches, which one should be used?).

Hope it gives you some idea of the risk you take when using such package

But a red button appear and say :
package.json and package.json.meta do not exist on this branch/tag
So what I'm doing wrong please ?

Click to expand...

You can't just drop any repository as a package, the developer needs to configure it to be compatible with Unity Packages (this repo seems to be a complete project, it probably won't work as intended if just dropped inside another project).

If anybody else here is having trouble understanding how to properly create a git hosted package I've created a sample repo that might be of use to you. I landed hear on Google hoping for an answer on how to make a git package but never really got a straight answer anywhere on how to do so.

Oops...

"Unity", Unity logos, and other Unity trademarks are trademarks or registered trademarks of Unity Technologies or its affiliates in the U.S. and elsewhere (more info here). Other names or brands are trademarks of their respective owners.