Unity Technologies

Glad you have found the pattern . As stated in the release notes, this is an experimental feature. We are targeting production ready quality for 2019.1. Super glad that you are trying this out though. Please, let us know if you find any issues.

And here is a minimalist documentation if you would like to expand your testing further and maybe for others who are reading this thread.

Requirements

To use Git packages in a project, Git must be installed on the user machine. The Git executable path should be listed in the PATH system environment variable.

Project manifest lock attribute

This feature introduces a new lock attribute in the project manifest. This attribute is populated by the Package Manager to render the package configuration resolution deterministic.

Supported URL schemes/protocols

The Package Manager supports the same URL schemes as Git does. In fact, the URL is passed to the Git executable (minus the revision part). If Git ever adds support for more protocol/schemes, they will automatically be supported by the Package Manager.

List of supported schemes

git:

ssh:

https:

http:

file: (only URL-like path are supported for now. i.e. file://absolute/path/repo.git. System path will be supported later. This could allow using relative path for example file:../../repo.git)

We also support SCP style pattern:

git@github.com:user/repo.git

Revisions

You can select a specific revision to be cloned by the Package Manager. The revision is prefixed by the # character. The revision should be added after .git extension:

https://repository/path.git#revision

The revision can be any commit-ish - an expression that eventually resolves to a commit hash. The most common types of commit-ish are commit hashes, tags, and branches.

Limitations and good to know:

The Package Manager UI was not updated to support this new package source (ETA 2019.1). I haven't seen any more issue though.

Big repositories can take a long time to clone. We don't cache the repository so if you change revision, the repository will be cloned again.

The only way to update a Git package to the latest version is to remove the lock attribute from the project manifest or manually update the revision suffix.

Since the Package Manager uses the Git executable installed on your machine. Any global or system configuration or environment variable will be picked up. These settings may affect the behaviour of the system.

Some package managers offer a shorthand version of their supported URL schemes. For example, npm support short URLs like these: <github user>/<repository>, gitlab:<gitlab user>/<repository>. We don't support these syntaxes for the moment.

Unity Technologies

@okcompute_unity thanks for that information! That is really useful!!
Is there any chance that the package manager will support finding packages in subdirectories of a git repository? Let's say I have one repository that contains multiple packages in various subfolders of that repo. I want to target a specific one, would that be possible?

Unity Technologies

@falkj17 Yes, this is something on our roadmap. But we suggest putting your package at the root of the repository if possible. This will be easier for your users to find the package. For instance, users will have to know and specify the name of your package if it is deep in the repo. While if the package is at the root, only the URL will be required, the name will be optional.

Great, thanks! Any idea on how long it will take to add that functionality?

I completely agree that for visibility it would be better to have it in the root, but we would like to group them by different themes and have all relevant packages in one repo, if possible. Having every single package in a different repo also makes it more difficult to develop multiple packages in parallel

IMO for development alone it's better to keep the repos separated, but I could see use for being able to keep different packages in single repo as well, like if you keep the git as custom repository that just holds various open source assets as packages (I already have two separate repos for this purpose atm).

Also if you have like say, custom SRP fork right now, the repo contains many packages already and it would be handy if people could just ref to them directly.

Right now you can workaround this by just putting different packages into different branches/releases but it's kinda hacky IMO.

The only way to update a Git package to the latest version is to remove the lock attribute from the project manifest or manually update the revision suffix.

Click to expand...

If there isn't any revision specified, is the package manager able to detect new version of the git package ?

In other dependency manager similar to this, you can for example specify an exact version match and it will go fetch that one that's what specifying the revision looks like at the moment. But is there a way with git repos (I know I can do it with the scoped registries) to fetch and check if there is an update of the repo (based on the package.json) ?

The only way to update a Git package to the latest version is to remove the lock attribute from the project manifest or manually update the revision suffix.

Click to expand...

Package specific lock attribute is generated at the end of manifest once the package manager processes the git clone/update, it basically stores the hash from git repo and revision. If you clear it, it should update to latest.

if you don't specify the revision, I think it will just use the default branch configured for the git repo, and to update it later on refer to:

package specific lock attribute is generated at the end of manifest once the package manager processes the git clone/update, it basically stores the hash from git repo and revision, if you clear it, it should update to latest.

Click to expand...

That's what I was asking, dependency managers like Cocoapods will check if it's the latest commit if you did not specify a revision.

So what I'm actually asking is : Is there any plan to handle this usecase where the package manager will check if he has the latest commit or not on a specific branch?

I tried this and it didn't work.
I managed to make it work by specifying the url with the ssh scheme:

ssh://git@bitbucket.org/myorg/my-repo.git#1.0.0

git: scheme didn't work. also note the slash "/" between host and org instead of colon ":"
http(s) scheme is not working as it complains about missing credentials (and I don't want to specify my password in the url (that will get committed)) (note: my repos are all private)

@falkj17 Yes, this is something on our roadmap. But we suggest putting your package at the root of the repository if possible.

Click to expand...

I think you're missing a major use case for this. There is a LOT of great stuff already on Github and it usually ends up cluttering up Asset folders because there's no other way to bring it in to a project.

If you supported packages in subdirectories we could:

1. Fork the repo we want to use
2. Add a package.json to the correct directory.

Most Unity repos are entire projects. Forking and adding a package.json to the root dir won't work in most cases. Forking and rearranging the directory structure makes it impossible to track upstream changes.

I've experimented with submodules, sparse checkouts and similar to get this working but they are extremely fragile.

it usually ends up cluttering up Asset folders because there's no other way to bring it in to a project.

Click to expand...

If you share whole project IMO best place for custom packages is just to place them into Packages folder for two reasons: you don't have to modify manifest at all, Unity picks them from there as packages automatically and it also keeps your Assets folder clean as they are not there bloating it.

I'd love to be able to point into that git repos Packages/<packagename> path directly from the manifest as this would let people share whole projects and also at the same time let others pick the package only from the project.

If you share whole project IMO best place for custom packages is just to place them into Packages folder for two reasons: you don't have to modify manifest at all, Unity picks them from there as packages automatically and it also keeps your Assets folder clean as they are not there bloating it.

I'd love to be able to point into that git repos Packages/<packagename> path directly from the manifest as this would let people share whole projects and also at the same time let others pick the package only from the project.

Click to expand...

Unless I'm misreading you - you're making a slightly orthogonal point (which I agree with)?

I also understand that you are talking 2 different thing (I agree on both):
- @rizu says, if you own a repo with a Unity project, you should put your stuff inside Packages/<your.package.name>/ and leave Assets/ empty (or with samples only)
- @andybak says, existing github repos in the wild currently have stuff in Assets/, and he needs to fork them, add a package.json in their "root" (and asmdefs probably) then point to it.

given that (especially wrt. existing stuff), I vote for being able to point UPM to a subfolder of a git repo.

ps: question for unity devs: what would be the best practice for developing a git-hosted package from now on (until we get the ability to push them privately to your registry/asset store)?
(1) repo should contain everything needed to develop the package, without being a submodule
(2) I want to be able to pull it from package manager, either by using a "git:" dependency pointing to a tag, or by releasing it to a registry (main one or scoped) and requesting with the version

): you don't want everything on the root of a public repository. That'd make things a bit messy. For public repositories, it's better to just have a

README.md

, a

LICENSE.md

, a

CONTRIBUTE.md

, and any other documentation on the root, and the actual package on a subfolder so people don't need to download useless files into a project.

The Unity WakaTime plugin at https://github.com/vladfaust/unity-wakatime is currently following this syntax. But it means it can't be included in the manifest as a Git URL because the actual package is inside a subfolder.

Hi, will the new feature be able to add only specific subfolder. We have spine runtime that is containing code for other engines and we just need to include folder that is intended to be used with unity. Zeh mentioned it with https://YourGitHost.com/UserName/RepoName.git/folderName

Unity Technologies

Git repository sub-folder is not supported for the moment. Your package has to be located at the root of the Git repository otherwise the Package Manager won't find it. We are considering to implement sub-folder support in the future. No ETA though.

Out of curiosity, anyone knows of a package manager supporting sub-folders? I can't find one. I would be really interested to see the chosen syntax. I know the popular ones don't support it (e.g. npm or pip).

Unity Technologies

@Foriero, Interesting. Did not know it was possible to partially download a Git repository through SVN!. Sadly, this does not fit well with our implementation and future features that we have planned. Thank you for sharing though

Not sure if this exactly matches what you're looking for, but in Paket (a .NET dependency manager) you can specify a directory in the repository which contains NuGet packages. It uses the following syntax:-

Hey, I'm having trouble with git trying to clone a repo using my password-protected SSH-key. I am running an ssh-agent in the background though, so I don't know what I might need to do so unity picks it up.... Many thanks!

Unity Technologies

Package Manager is basically passing along GIT error messages through UPM when something goes amiss. If you're getting an error message that says "authentication error", it might be something to do with the SSH key location/availability/agent or credentials not being properly entered.

While your script may successfully start the ssh-agent in the background, the exported environment variables are only available to the bash shell running the script and to any child process it launches from that point on; environment variables from parent processes, unrelated processes, or previously created child processes cannot be changed by your script. Further, even if the script is located in .bashrc, it only gets executed in bash shells. Launching programs (both in Windows and macOS) does not use a shell, so the script is not executed. On the other hand, if you open a bash shell first, then "manually" launch the Unity Editor from that shell, it should work.

In other words, if you use the script by running it once, then you launch Unity by double-clicking its icon or using the Hub, the Unity Editor process won't have these variables set so ultimately the Git process invoked by Unity won't have them either.

That said, I see a few possible workarounds.
- You can create a bash script to launch the Unity executable or the Hub (caveat: the Hub must not already be running, otherwise that will just activate the existing process and it won't have the newly exported environment variables)
- Or since you're on Windows, you might try to invoke SETX to permanently set the environment variables in the Windows registry. However you need to ensure the Hub process is not already running, or that it gets restarted if it's already running. This solution would work best if you can ensure that the script runs once on startup and that the Hub is not started with Windows.

The last suggestion would have the benefit of seamlessly integrating the ssh-agent so it works as you expect it to.

@maximeb_unity hey thanks for the suggestion. The second one might just what i needed to also fix other ssh-agent related issues I've had with git clients (Tower, specifically). Will try this out asap! Cheers!

Unity Technologies

Yeah, Gitlab provides SSH "URLs" in the SCP-like format, which is not quite a URL. We don't support that format (yet) but we intend to because it will be easier - that format is commonly used in the Git world.

Oops...

"Unity", Unity logos, and other Unity trademarks are trademarks or registered trademarks of Unity Technologies or its affiliates in the U.S. and elsewhere (more info here). Other names or brands are trademarks of their respective owners.