Launchpad itself

~ubuntu-branches and other locations for git importer trees

Asked by Robie Basak

We'd like to fix a locations for our git importer trees. Should we take over ~ubuntu-branches? If so, we'll need some ACL changes presumably?

I'd like to conclude with the Launchpad team where we should put these, both for the short term and the long term (if different).

The importer needs two locations (possibly the same, but then per-ref ACLs would be needed):

1) A place that the importer can push to, but nobody else can. This would be the "official" reference location that developers will clone from.

2) A place where uploaders can supply git commits to the importer's next run for adoption into the "official" trees as rich history ("what changes I made to get to my upload of version X of package Y"). Right now, the importer looks for what we're calling an "upload tag" in the form of "refs/tags/upload/<version>". But we could use any other mechanism; all we need is a function that, given a package name and packaging version number, fetches a commit object whose tree object matches the upload. dgit's use of a field in the changes file would work, for example, provided that we also know which repository to fetch that commit from.

In theory both could be in ~ubuntu-branches, but this would only be safe if we could restrict uploaders to pushing tags of the form "refs/tags/upload/*" only, with only the importer being able to push any other tag. Or we could ask (and arrange tooling) for uploaders to push to refs/tags/upload/<version> in lp:~<uploader>/ubuntu/+source/<package>, for example, assuming that the importer can identify the same <uploader> from source package publishing history, but that may have practical performance implications (excessive storage).

Question information

Language:
English Edit question
Status:
Solved
For:
Launchpad itself Edit question
Assignee:
No assignee Edit question
Solved by:
Robie Basak
Solved:
Last query:
Last reply:
Revision history for this message
Launchpad Janitor (janitor) said : 		#1

This question was expired because it remained in the 'Open' state without activity for the last 15 days.

Revision history for this message
Robie Basak (racb) said : 		#2

Reopening

Revision history for this message
Launchpad Janitor (janitor) said : 		#3

This question was expired because it remained in the 'Open' state without activity for the last 15 days.

Revision history for this message
Robie Basak (racb) said : 		#4

Reopening

Revision history for this message
Colin Watson (cjwatson) said : 		#5

I wouldn't bother reusing ~ubuntu-branches for this, since it isn't a hardcoded "celebrity" user or anything. Instead, I'd set the imported repositories as the default repositories for the package target; this will mean that lp:ubuntu/+source/package is aliased to the imported repositories, and it will keep storage costs reasonable since pushes to lp:~UPLOADER/ubuntu/+source/package will share objects with the imported repositories.

However, I think we need some Launchpad changes before this can work.

Firstly, we definitely need a way to set the default repository for a package target (currently XXXed out until we figure out a sensible policy), otherwise none of the plans can work sensibly. For the Bazaar importer, I believe this worked by means of the importer user being a member of ~ubuntu-core-dev, but that's hideous and we should have a better way that doesn't grant upload access to the importer user. I'm not sure exactly what this would look like; perhaps a separate permission slot? It needs some thought.

After that, in the short term, it's probably sensible to rely on uploaders pushing to their own repository. In the longer term, I'd like it to be possible for uploaders to push directly to the imported repository as well, but that requires per-ref-pattern ACLs and a policy for pushes to lp:ubuntu/+source/package that knows about upload permissions. Both of those would require a moderate amount of work.

On a practical note, at present, we're extremely short-staffed - it's going to be mainly just me working on Launchpad for a while, and the chances of me having time to implement any of this are negligible. You should probably be working on the assumption that you're going to need to figure out and contribute the Launchpad changes for default repository handling, although I can certainly advise and review.

Revision history for this message
Colin Watson (cjwatson) said : 		#6

The bit about setting the default repository for a package target being XXXed out in the code was a memory/drafting error; there's no code limitation on setting the default repository for a package target, except that it relies on being able to edit the DistributionSourcePackage which is tied to upload permissions, and that's hideous as I described above.

Revision history for this message
Robie Basak (racb) said : 		#7

Thanks! That gives us a good understanding of what we need to do.

For reference, there was some further discussion at: https://irclogs.ubuntu.com/2017/02/03/%23launchpad.html#t12:02