Bazaar

Impossible to commit because of long commit times

Asked by Kirill Suzev

Hi!
I have a problem using bazaar - in out company we have a very large svn-repo and many developers, committing into in. I tried to develop via bzr, so I checked out trunk, made a branch of it and started to work.
Some time passed, my work in branch was finished, and I merged it back to trunk. I tried to commit, but bzr didn't let me do this, because my trunk repo was out of date. I ran bzr up and it took not less then 20 minutes.
After that I tried to commit, but funny story - repo was out of date again (that is no error, while I was updating there were new commits from co-workers).
So looks like I just can't commit in anyway.
Maybe it would be nice to have an option for bzr commit command to commit when repo is out of date, but non of modified by me files were changed in repo?

Question information

Language:
English Edit question
Status:
Solved
For:
Bazaar Edit question
Assignee:
No assignee Edit question
Solved by:
John A Meinel
Solved:
Last query:
Last reply:
Revision history for this message
Martin Pool (mbp) said : 		#1

Hi,

We could add a feature that would automatically rebase your changes onto the current tip, if there are no conflicts.

However, if commit is taking such a long time, there is no guarantee that this rebasing would complete within a reasonable amount of time. What I suggest you do is get a profile of the commit and file that as a bug against bzr-svn.

Run 'bzr --lsprof-file update.callgrind update' and paste that into the bug.

Revision history for this message
Kirill Suzev (suzev-kirill) said : 		#2

Well, yes, speeding up the commit and update commands will slightly help. But in case, when there are many developers, that commit very often, that hardly solve the problem.
So I would greatly appreciate, if you, guys, implement this feaure with commit.
Thanks!

Revision history for this message
Martin Pool (mbp) said : 		#3

On 10 November 2010 20:26, Kirill Suzev
<email address hidden> wrote:
> Question #133300 on Bazaar changed:
> https://answers.launchpad.net/bzr/+question/133300
>
>    Status: Answered => Open
>
> Kirill Suzev is still having a problem:
> Well, yes, speeding up the commit and update commands will slightly help. But in case, when there are many developers, that commit very often, that hardly solve the problem.
> So I would greatly appreciate, if you, guys, implement this feaure with commit.

One of the ideas behind bzr making it easier to branch, and easier to
have local branches, is that there will then be less contention on
centralized branches, both with regard to IO and also people
conflicting with each other or destabilizing the branch. Perhaps you
could look at having teams use their own individual integration
branches, and then merging from that into a central branch? Just an
idea.

--
Martin

Revision history for this message
Kirill Suzev (suzev-kirill) said : 		#4

The development process (that includes branch and commit rules) is kind of complex in our company, but anyway, feature implementation almost always goes in a feature branch, not in trunk. We commit to trunk only after the implementation is finished, except very small features, that are fast to develop. But anyway, it's just too many developers and teams, so commits happen very often. There is a possibility to commit at night via bazaar, but it's not comfy.
So still, I think it is a good idea to implement feature "that would automatically rebase your changes onto the current tip, if there are no conflicts". It will make work with bazaar faster, and it will help solve my problem, and it will make bazaar a better tool - everybody wins :)

Revision history for this message
Kirill Suzev (suzev-kirill) said : 		#5

I don't wont to be annoying, but what is the final answer? Will this feature be scheduled to implement?
I hope so, because in the way I see it, this feature is crucial for bazaar.

Revision history for this message
Vincent Ladeuil (vila) said : 		#6

No one is working on it nor is it planned AFAIK, patches will be welcome.

As for whether it is crucial for bazaar, well, the distributed model is a better answer to the centralized workflow when too many developers are trapped into the problem you're describing.

It's arguably something that should be addressed to help svn users though.

Note that the workflow you're describing (feature branches merged on trunk) is exactly the one we're using for bazaar itself.

A noticeable difference though, is that we use pqm (a bot acting as a gatekeeper) which is the only one allowed to commit on trunk after merging a submission. The submissions are queued and *this* scales (arguably we don't have the same number of committers than you).

Revision history for this message
Best John A Meinel (jameinel) said : 		#7

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/15/2010 8:40 AM, Vincent Ladeuil wrote:
> Question #133300 on Bazaar changed:
> https://answers.launchpad.net/bzr/+question/133300
>
> Status: Open => Answered
>
> Vincent Ladeuil proposed the following answer:
> No one is working on it nor is it planned AFAIK, patches will be
> welcome.
>
> As for whether it is crucial for bazaar, well, the distributed model is
> a better answer to the centralized workflow when too many developers are
> trapped into the problem you're describing.

I think the other problem is that bzr-svn is probably globally locked.
Meaning that if you have a bzr branch of "svn/project/branches/foo", and
somebody commits to "svn/project/branches/bar", bzr-svn will probably
complain.

We *do* want to block if somebody commits a change to
  svn/project/branches/foo/myfile.txt
and you are trying to commit a change to
  svn/project/branches/foo/otherfile.txt

I don't know how easy it is to tease apart, though. SVN doesn't do
tree-shaped atomicity. (it will let 2 people concurrently commit changes
to both of those files.)

I also don't know about performance reasons. How easy is it for bzr to
determine that the intermediate commits did or didn't touch files that
you care about. You said it took 10min+ to update from svn, which sounds
rather long.

I would agree that decentralizing it, and only having local branches,
that then occasionally get pushed back into svn. If you have your own
svn branches, I think 'push' can always succeed, and then you can merge
things back.

For example:

  svn://project/trunk
  svn://project/branches/svn-branch
  svn://project/branches/my-branch

/bzr-repo/trunk
 a mirror of 'svn://project/trunk'
 you only ever go into this to pull changes from trunk (either via
 update or pull)

/bzr-repo/svn-branch
 also only a mirror of svn-branch.

/bzr-repo/my-branch
 This is the primary branch that you do work on. When you are happy,
 you 'bzr push' it back to the svn repository.
 If people have been doing things you are interested in, they would
 have done them on "svn-branch". And you can always do:

   cd /bzr-repo/svn-branch
   bzr pull
   cd ../my-branch
   bzr merge ../svn-branch
   bzr commit -m "bring in the latest from 'svn-branch'"
   bzr push

I may not be correct about all of bzr-svn. And if you actually want to
be able to commit changes to "otherfile.txt" when "myfile.txt" (in the
same branch) has been modified in svn, that is actually something that
we *won't* support. (tree-level atomicity is important to us for
handling both distributed nature, and to actually make sure that you
commit a snapshot that can be properly restored later.)

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkzhS4gACgkQJdeBCYSNAAN0jACfQ9hPAJVRjIMas7r9S/T88lNq
+okAnipTn0EHJ30ckPVu5PH1MqfYGOIK
=8xY/
-----END PGP SIGNATURE-----

Revision history for this message
Kirill Suzev (suzev-kirill) said : 		#8

Good day, John.
First of all, I would like to thank you for your detailed answer.
I think, I understand your point. The problem is that subversion differs from bzr in a way they both handle history of changes.
So I think, I'll investigate perfomance issue a little, that should help.

Revision history for this message
Kirill Suzev (suzev-kirill) said : 		#9

Thanks John A Meinel, that solved my question.