bzr speed and frequent crashes from Sri Lanka (.lk)

Asked by chamindra

We have been using bzr for the sahana project (http://www.sahana.lk) and have to report that we are experiencing frequent crashes on bzr pull / merge. The speed being reported on the pull is also much less than my throughput to the internet. I do not experience this issue when using the launchpad website, other websites or when using ssh on other sites.

Below is an example where it has freezed on a bzr pull

Using saved parent location: http://bazaar.launchpad.net/~sahanapy-haiti/sahana/haiti-or/
[/ ] 976kB @ 1kB/s | Pull phase:Copying inventory texts

My throughput is 52kB/s typically compared to the 1-10kBs I am getting from the bzr launchpad server.

PING bazaar.launchpad.net (91.189.90.11): give icmp_seq=2 ttl=47 time=214 ms

This also has a tendency to unrecoverable corrupt my bzr instance. However asking other members of our global community they do not seems to have issues with bzr, but other members in the project from Sri Lanka have complained about similar issues.

Question information

Language:
English Edit question
Status:
Answered
For:
Bazaar Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Martin Pool (mbp) said :
#1

Hi Chamindra,

That's great to see free software being used for such a cause and I want to make sure bzr is helping you well.

I'm going to convert this into a support question at least until we determine that there is a specific bug here.

Revision history for this message
Martin Pool (mbp) said :
#2

Let's work out what's going wrong here.

Revision history for this message
Martin Pool (mbp) said :
#3

So, I just tried branching from that URL, and I downloaded 68M in 45s, about 1511kb/s. But this was over quite a different network connection.

When you say it crashes, do you mean it's actually crashing with an error message, or it's just staying stuck at this point for a long time?

To get some more data to track down the problem I'd like your help in trying a few things.

One is that if it crashes, please paste the error message from ~/.bzr.log here.

To look at network problems please run

bzr -Dtransport -Dhttp pull log+http://bazaar.launchpad.net/~sahanapy-haiti/sahana/haiti-or/

this will record some information into ~/.bzr.log; please attach that here too.

And there is another suggestion, which is that if you use ssh to get to Launchpad you may get better performance and you may have better protection from some types of problem on the network between your machine and Launchpad, such as buggy proxy servers. To do that you just need to say

bzr -Dhpss pull bzr+ssh://bazaar.launchpad.net/~sahanapy-haiti/sahana/haiti-or/

and if that hangs or errors, please let me know.

If you are running bzr before 2.0 please upgrade; if you're having trouble with that then ask.

Revision history for this message
chamindra (chamindra) said :
#4

It does not crash with an error message, but just gets stuck (pauses
on the commandline)

On of the problems is that my throughput from Sri Lanka is quite slow.
I typically get 52kBs down and about 1/3rd of that up. However the
link to the launchpad server was much slower than that. Before this I
was using CVS on sourceforge and that was relatively fast.

Attached below is a trace. note:I have obfuscated the name of my home directory

I am using ubuntu 9.04 on this machine and the bzr version reported is
1.13.1 (so yes it is before 2.0). I will eventually upgrade to 9.10,
but do not have time right now. What might be the easiest way for me
to upgrade.

Mon 2010-01-25 02:14:38 +0530
0.031 bzr arguments: [u'repositories']
0.055 looking for plugins in /home/person/.bazaar/plugins
0.055 looking for plugins in /usr/lib/python2.6/dist-packages/bzrlib/plugins
0.129 Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line
716, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line
889, in run_bzr
    cmd_obj = get_cmd_object(cmd, plugins_override=not opt_builtin)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line
180, in get_cmd_object
    raise errors.BzrCommandError('unknown command "%s"' % cmd_name)
BzrCommandError: unknown command "repositories"

0.129 return code 3

Mon 2010-01-25 02:14:43 +0530
0.031 bzr arguments: [u'help', u'repositories']
0.055 looking for plugins in /home/person/.bazaar/plugins
0.055 looking for plugins in /usr/lib/python2.6/dist-packages/bzrlib/plugins
0.125 encoding stdout as sys.stdout encoding 'UTF-8'
0.127 return code 0

Mon 2010-01-25 02:19:30 +0530
0.031 bzr arguments: [u'launchpad-login', u'chamindra']
0.055 looking for plugins in /home/person/.bazaar/plugins
0.055 looking for plugins in /usr/lib/python2.6/dist-packages/bzrlib/plugins
0.125 encoding stdout as sys.stdout encoding 'UTF-8'
0.193 failed to import pycurl: No module named pycurl
0.194 failed to instantiate transport
<bzrlib.registry._LazyObjectGetter object at a312e8c,
module='bzrlib.transport.http._pycurl' attribute='PyCurlTransport'>
for 'https://launchpad.net/': DependencyNotPresent(Unable to import
library "pycurl": No module named pycurl)
17.096 return code 0

Mon 2010-01-25 02:19:47 +0530
0.029 bzr arguments: [u'checkout', u'--lightweight', u'lp:sahana']
0.053 looking for plugins in /home/person/.bazaar/plugins
0.053 looking for plugins in /usr/lib/python2.6/dist-packages/bzrlib/plugins
0.124 encoding stdout as sys.stdout encoding 'UTF-8'
13.597 ssh implementation is OpenSSH
26.455 creating branch reference in file:///home/person/repo/vcs/sahana/.bzr/
28.281 trying to create missing lock
'/home/person/repo/vcs/sahana/.bzr/checkout/dirstate'
28.282 opening working tree '/home/person/repo/vcs/sahana'
139.013 Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line
716, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line
911, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line
547, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.6/dist-packages/bzrlib/builtins.py", line 1187, in run
    accelerator_tree, hardlink)
  File "/usr/lib/python2.6/dist-packages/bzrlib/branch.py", line 1105,
in create_checkout
    hardlink=hardlink)
  File "/usr/lib/python2.6/dist-packages/bzrlib/bzrdir.py", line 1481,
in create_workingtree
    accelerator_tree=accelerator_tree, hardlink=hardlink)
  File "/usr/lib/python2.6/dist-packages/bzrlib/workingtree_4.py",
line 1394, in initialize
    hardlink=hardlink, delta_from_tree=True)
  File "/usr/lib/python2.6/dist-packages/bzrlib/transform.py", line
2034, in build_tree
    delta_from_tree)
  File "/usr/lib/python2.6/dist-packages/bzrlib/transform.py", line
2126, in _build_tree
    accelerator_tree, hardlink)
  File "/usr/lib/python2.6/dist-packages/bzrlib/transform.py", line
2179, in _create_files
    new_desired_files)):
  File "/usr/lib/python2.6/dist-packages/bzrlib/revisiontree.py", line
79, in iter_files_bytes
    for result in self._repository.iter_files_bytes(repo_desired_files):
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line
1550, in iter_files_bytes
    for record in self.texts.get_record_stream(text_keys, 'unordered', True):
  File "/usr/lib/python2.6/dist-packages/bzrlib/knit.py", line 1327,
in get_record_stream
    ordering, include_delta_closure):
  File "/usr/lib/python2.6/dist-packages/bzrlib/knit.py", line 1338,
in _get_remaining_record_stream
    positions = self._get_components_positions(keys, allow_missing=True)
  File "/usr/lib/python2.6/dist-packages/bzrlib/knit.py", line 1101,
in _get_components_positions
    build_details = self._index.get_build_details(pending_components)
  File "/usr/lib/python2.6/dist-packages/bzrlib/knit.py", line 2850,
in get_build_details
    for entry in entries:
  File "/usr/lib/python2.6/dist-packages/bzrlib/knit.py", line 2878,
in _get_entries
    for node in self._graph_index.iter_entries(keys):
  File "/usr/lib/python2.6/dist-packages/bzrlib/index.py", line 1267,
in iter_entries
    for node in index.iter_entries(keys):
  File "/usr/lib/python2.6/dist-packages/bzrlib/index.py", line 613,
in iter_entries
    self._buffer_all()
  File "/usr/lib/python2.6/dist-packages/bzrlib/index.py", line 409,
in _buffer_all
    stream = self._transport.get(self._name)
  File "/usr/lib/python2.6/dist-packages/bzrlib/transport/remote.py",
line 223, in get
    return StringIO(self.get_bytes(relpath))
  File "/usr/lib/python2.6/dist-packages/bzrlib/transport/remote.py",
line 234, in get_bytes
    return response_handler.read_body_bytes()
  File "/usr/lib/python2.6/dist-packages/bzrlib/smart/message.py",
line 319, in read_body_bytes
    self._wait_for_response_end()
  File "/usr/lib/python2.6/dist-packages/bzrlib/smart/message.py",
line 268, in _wait_for_response_end
    self._read_more()
  File "/usr/lib/python2.6/dist-packages/bzrlib/smart/message.py",
line 277, in _read_more
    bytes = self._medium_request.read_bytes(next_read_size)
  File "/usr/lib/python2.6/dist-packages/bzrlib/smart/medium.py", line
459, in read_bytes
    return self._read_bytes(count)
  File "/usr/lib/python2.6/dist-packages/bzrlib/smart/medium.py", line
471, in _read_bytes
    return self._medium.read_bytes(count)
  File "/usr/lib/python2.6/dist-packages/bzrlib/smart/medium.py", line
149, in read_bytes
    return self._read_bytes(bytes_to_read)
  File "/usr/lib/python2.6/dist-packages/bzrlib/smart/medium.py", line
787, in _read_bytes
    bytes = self._read_from.read(bytes_to_read)
KeyboardInterrupt

139.013 return code 3

On Mon, Jan 25, 2010 at 10:44 PM, Martin Pool
<email address hidden> wrote:
> Your question #98657 on Bazaar changed:
> https://answers.edge.launchpad.net/bzr/+question/98657
>
>    Status: Open => Needs information
>
> Martin Pool requested for more information:
> So, I just tried branching from that URL, and I downloaded 68M in 45s,
> about 1511kb/s.  But this was over quite a different network connection.
>
> When you say it crashes, do you mean it's actually crashing with an
> error message, or it's just staying stuck at this point for a long time?
>
> To get some more data to track down the problem I'd like your help in
> trying a few things.
>
> One is that if it crashes, please paste the error message from
> ~/.bzr.log here.
>
> To look at network problems please run
>
> bzr -Dtransport -Dhttp pull log+http://bazaar.launchpad.net/~sahanapy-
> haiti/sahana/haiti-or/
>
> this will record some information into ~/.bzr.log; please attach that
> here too.
>
> And there is another suggestion, which is that if you use ssh to get to
> Launchpad you may get better performance and you may have better
> protection from some types of problem on the network between your
> machine and Launchpad, such as buggy proxy servers.  To do that you just
> need to say
>
> bzr -Dhpss pull bzr+ssh://bazaar.launchpad.net/~sahanapy-haiti/sahana
> /haiti-or/
>
> and if that hangs or errors, please let me know.
>
> If you are running bzr before 2.0 please upgrade; if you're having
> trouble with that then ask.
>
> --
> To answer this request for more information, you can either reply to
> this email or enter your reply at the following page:
> https://answers.edge.launchpad.net/bzr/+question/98657
>
> You received this question notification because you are a direct
> subscriber of the question.
>

--
Chamindra de Silva
Home: http://chamindra-de-silva.blogspot.com
Twitter: http://twitter.com/ChamindraS (note the last S!)

Revision history for this message
Martin Pool (mbp) said :
#5

Hi,

You can get the current bzr stable release through
https://launchpad.net/~bzr/+archive/ppa?field.series_filter=jaunty

If you're on a slow network I would really recommend you use a real
branch not a lightweight checkout, which will connect to the server
for many operations. Run

bzr init-repo --0.92 ~/sahana
bzr branch lp:sahaha ~/sahana/trunk

then within that directory say 'bzr pull' to bring down new changes.

--
Martin <http://launchpad.net/~mbp/>

Revision history for this message
chamindra (chamindra) said :
#6

Thank you! Just downloaded the latest bzr version and will try this
out. You might be interested to know I have to do 31hops from Sri
Lanka's main telecom provider to get to the server, which is probably
why it can have varying throughputs.

BTW what does bzr init-repo --0.92 ~/sahana do exacrly? why 0.92?

On Mon, Jan 25, 2010 at 11:15 PM, Martin Pool
<email address hidden> wrote:
> Your question #98657 on Bazaar changed:
> https://answers.launchpad.net/bzr/+question/98657
>
>    Status: Open => Answered
>
> Martin Pool proposed the following answer:
> Hi,
>
> You can get the current bzr stable release through
> https://launchpad.net/~bzr/+archive/ppa?field.series_filter=jaunty
>
> If you're on a slow network I would really recommend you use a real
> branch not a lightweight checkout, which will connect to the server
> for many operations.  Run
>
> bzr init-repo --0.92 ~/sahana
> bzr branch lp:sahaha ~/sahana/trunk
>
> then within that directory say 'bzr pull' to bring down new changes.
>
> --
> Martin <http://launchpad.net/~mbp/>
>
> --
> If this answers your question, please go to the following page to let us
> know that it is solved:
> https://answers.launchpad.net/bzr/+question/98657/+confirm?answer_id=4
>
> If you still need help, you can reply to this email or go to the
> following page to enter your feedback:
> https://answers.launchpad.net/bzr/+question/98657
>
> You received this question notification because you are a direct
> subscriber of the question.
>

--
Chamindra de Silva
Home: http://chamindra-de-silva.blogspot.com
Twitter: http://twitter.com/ChamindraS (note the last S!)

Revision history for this message
Martin Pool (mbp) said :
#7

2010/1/25 chamindra <email address hidden>:
> Question #98657 on Bazaar changed:
> https://answers.launchpad.net/bzr/+question/98657
>
>    Status: Answered => Open
>
> chamindra is still having a problem:
> Thank you! Just downloaded the latest bzr version and will try this
> out. You might be interested to know I have to do 31hops from Sri
> Lanka's main telecom provider to get to the server, which is probably
> why it can have varying throughputs.
>
> BTW what does bzr init-repo --0.92 ~/sahana do exacrly? why 0.92?

0.92 sets the repository format, and it is the same as the format
shown by "bzr info
http://bazaar.launchpad.net/~sahanapy-haiti/sahana/haiti-or/"

If you or any other developers need any other help just ask.

--
Martin <http://launchpad.net/~mbp/>

Revision history for this message
chamindra (chamindra) said :
#8

Sorry I have been trying this for a while. I upgraded to the latest version of bzr as mentioned about and it is still taking me ages to upload and download branches. I am talking about effective speeds of 1-2 KBps on average on a connection that should get a lot more (52Kbps). And it keeps crashing.

I attach a log entry

Thu 2010-04-08 02:54:54 +0530
0.062 bazaar version: 2.1.1
0.062 bzr arguments: [u'branch', u'lp:~sahana-php/s3/php-cvs-trunk']
0.084 looking for plugins in /home/wintermute/.bazaar/plugins
0.084 looking for plugins in /usr/lib/python2.6/dist-packages/bzrlib/plugins
0.177 encoding stdout as sys.stdout encoding 'UTF-8'
2.073 ssh implementation is OpenSSH
9.760 creating repository in file:///mnt/explore/dev/sahana-admin-bot/vcs/php-cvs-trunk/.bzr/.
9.774 Using fetch logic to copy between RemoteRepository(bzr+ssh://bazaar.launchpad.net/~sahana-php/s3/php-cvs-trunk/.bzr/)(RemoteRepositoryFormat(_network_name='Bazaar repository format 2a (needs bzr 1.16 or later)\n')) and CHKInventoryRepository('file:///mnt/explore/dev/sahana-admin-bot/vcs/php-cvs-trunk/.bzr/repository/')(RepositoryFormat2a())
344.074 Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 853, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 1055, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 661, in run_argv_aliases
    return self.run_direct(**all_cmd_args)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 665, in run_direct
    return self._operation.run_simple(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/cleanup.py", line 122, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/cleanup.py", line 156, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/builtins.py", line 1232, in run
    source_branch=br_from)
  File "/usr/lib/python2.6/dist-packages/bzrlib/bzrdir.py", line 1184, in sprout
    result_repo.fetch(source_repository, fetch_spec=fetch_spec)
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 1704, in fetch
    find_ghosts=find_ghosts, fetch_spec=fetch_spec)
  File "/usr/lib/python2.6/dist-packages/bzrlib/decorators.py", line 194, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 3438, in fetch
    pb=pb, find_ghosts=find_ghosts)
  File "/usr/lib/python2.6/dist-packages/bzrlib/fetch.py", line 83, in __init__
    self.__fetch()
  File "/usr/lib/python2.6/dist-packages/bzrlib/fetch.py", line 109, in __fetch
    self._fetch_everything_for_search(search)
  File "/usr/lib/python2.6/dist-packages/bzrlib/fetch.py", line 137, in _fetch_everything_for_search
    stream, from_format, [])
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 4279, in insert_stream
    return self._locked_insert_stream(stream, src_format, is_resume)
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 4316, in _locked_insert_stream
    substream)
  File "/usr/lib/python2.6/dist-packages/bzrlib/groupcompress.py", line 1598, in insert_record_stream
    for _ in self._insert_record_stream(stream, random_id=False):
  File "/usr/lib/python2.6/dist-packages/bzrlib/groupcompress.py", line 1653, in _insert_record_stream
    for record in stream:
  File "/usr/lib/python2.6/dist-packages/bzrlib/versionedfile.py", line 1665, in read
    for bytes in self._bytes_iterator:
  File "/usr/lib/python2.6/dist-packages/bzrlib/smart/repository.py", line 571, in iter_substream_bytes
    for record in self.iter_pack_records:
  File "/usr/lib/python2.6/dist-packages/bzrlib/smart/repository.py", line 561, in iter_stream_decoder
    for bytes in self.byte_stream:
  File "/usr/lib/python2.6/dist-packages/bzrlib/smart/message.py", line 336, in read_streamed_body
    self._read_more()
  File "/usr/lib/python2.6/dist-packages/bzrlib/smart/message.py", line 277, in _read_more
    bytes = self._medium_request.read_bytes(next_read_size)
  File "/usr/lib/python2.6/dist-packages/bzrlib/smart/medium.py", line 464, in read_bytes
    return self._read_bytes(count)
  File "/usr/lib/python2.6/dist-packages/bzrlib/smart/medium.py", line 476, in _read_bytes
    return self._medium.read_bytes(count)
  File "/usr/lib/python2.6/dist-packages/bzrlib/smart/medium.py", line 151, in read_bytes
    return self._read_bytes(bytes_to_read)
  File "/usr/lib/python2.6/dist-packages/bzrlib/smart/medium.py", line 817, in _read_bytes
    bytes = osutils.until_no_eintr(self._read_from.read, bytes_to_read)
  File "/usr/lib/python2.6/dist-packages/bzrlib/osutils.py", line 2024, in until_no_eintr
    return f(*a, **kw)
KeyboardInterrupt

344.075 Transferred: 552KiB (1.6K/s r:551K w:1K)
344.075 return code 3

Thu 2010-04-08 03:00:56 +0530
0.071 bazaar version: 2.1.1
0.071 bzr arguments: [u'branch', u'lp:~sahana-php/s3/php-cvs-trunk']
0.094 looking for plugins in /home/wintermute/.bazaar/plugins
0.094 looking for plugins in /usr/lib/python2.6/dist-packages/bzrlib/plugins
0.188 encoding stdout as sys.stdout encoding 'UTF-8'
12.231 ssh implementation is OpenSSH
19.942 creating repository in file:///mnt/explore/dev/sahana-admin-bot/vcs/php-cvs-trunk/.bzr/.
19.954 Using fetch logic to copy between RemoteRepository(bzr+ssh://bazaar.launchpad.net/~sahana-php/s3/php-cvs-trunk/.bzr/)(RemoteRepositoryFormat(_network_name='Bazaar repository format 2a (needs bzr 1.16 or later)\n')) and CHKInventoryRepository('file:///mnt/explore/dev/sahana-admin-bot/vcs/php-cvs-trunk/.bzr/repository/')(RepositoryFormat2a())
774.804 Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 853, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 1055, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 661, in run_argv_aliases
    return self.run_direct(**all_cmd_args)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 665, in run_direct
    return self._operation.run_simple(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/cleanup.py", line 122, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/cleanup.py", line 156, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/builtins.py", line 1232, in run
    source_branch=br_from)
  File "/usr/lib/python2.6/dist-packages/bzrlib/bzrdir.py", line 1184, in sprout
    result_repo.fetch(source_repository, fetch_spec=fetch_spec)
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 1704, in fetch
    find_ghosts=find_ghosts, fetch_spec=fetch_spec)
  File "/usr/lib/python2.6/dist-packages/bzrlib/decorators.py", line 194, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 3438, in fetch
    pb=pb, find_ghosts=find_ghosts)
  File "/usr/lib/python2.6/dist-packages/bzrlib/fetch.py", line 83, in __init__
    self.__fetch()
  File "/usr/lib/python2.6/dist-packages/bzrlib/fetch.py", line 109, in __fetch
    self._fetch_everything_for_search(search)
  File "/usr/lib/python2.6/dist-packages/bzrlib/fetch.py", line 137, in _fetch_everything_for_search
    stream, from_format, [])
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 4279, in insert_stream
    return self._locked_insert_stream(stream, src_format, is_resume)
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 4316, in _locked_insert_stream
    substream)
  File "/usr/lib/python2.6/dist-packages/bzrlib/groupcompress.py", line 1598, in insert_record_stream
    for _ in self._insert_record_stream(stream, random_id=False):
  File "/usr/lib/python2.6/dist-packages/bzrlib/groupcompress.py", line 1653, in _insert_record_stream
    for record in stream:
  File "/usr/lib/python2.6/dist-packages/bzrlib/versionedfile.py", line 1665, in read
    for bytes in self._bytes_iterator:
  File "/usr/lib/python2.6/dist-packages/bzrlib/smart/repository.py", line 571, in iter_substream_bytes
    for record in self.iter_pack_records:
  File "/usr/lib/python2.6/dist-packages/bzrlib/smart/repository.py", line 561, in iter_stream_decoder
    for bytes in self.byte_stream:
  File "/usr/lib/python2.6/dist-packages/bzrlib/smart/message.py", line 336, in read_streamed_body
    self._read_more()
  File "/usr/lib/python2.6/dist-packages/bzrlib/smart/message.py", line 277, in _read_more
    bytes = self._medium_request.read_bytes(next_read_size)

Revision history for this message
Andrew Bennetts (spiv) said :
#9

Something is a bit fishy here, maybe. Here's the result I get:

Branched 4531 revision(s).
Transferred: 77272kB (25.2kB/s r:77270kB w:1kB)
HPSS calls: 11 (0 vfs) SmartSSHClientMedium(bzr+ssh://<email address hidden>/)

real 51m3.941s
user 0m31.630s
sys 0m4.716s

That does seem unexpectedly slow for streaming of a 2a repository (which this is). Like chamindra, I expect my connection to be capable of much faster than the result I got. The HPSS call log looks normal, just a bit slow at receiving the data. Possibly there was some sort of congestion between my home in Sydney and the Launchpad server in London at the time?

Here's a random snippet of the -Dhpssdetail output from the middle of the stream:

1773.843 2489727 byte part read
1777.087 90127 byte part read
1869.086 2356817 byte part read
1888.879 627596 byte part read
1913.121 578957 byte part read
1931.398 534556 byte part read
1966.998 915203 byte part read
1968.816 32096 byte part read
2094.481 2804618 byte part read
2154.295 1553352 byte part read
2164.415 323977 byte part read
2188.762 447938 byte part read
2235.080 1148231 byte part read
2261.441 621062 byte part read
2293.894 902224 byte part read
2330.604 805523 byte part read
2605.612 6580187 byte part read
2611.862 303912 byte part read

Nice large blocks, as they should be... but look at the times: 30s to transmit 900k? Strange.

Revision history for this message
Andrew Bennetts (spiv) said :
#10

Also, you say "And it keeps crashing." but you haven't pasted any sign of that that I can see?

Can you show us a traceback from a crash, or provide more information about what sort of crashes you are seeing?

Revision history for this message
chamindra (chamindra) said :
#11

Yup we have had multiple reports of the poor performance of bzr from other members from New Zealand and Australia. It is a consistent occurrence for me so for bzr and my other internet traffic is normal. I am not sure what it is, but it almost seems like the connection is being throttled. In terms of the crashing, I am not sure what I can give you. A bzr pull frequently pauses and restarts and when the pause goes over 10-20 mins I presume it has crashed. We are in the process of migrating from SF/CVS to LP/BZR for our project and certainly on the LP site is much better for our needs, but in comparison a pull which took only 5 mins in CVS is taking 30-50mins in bzr and it certainly is a big impact on productivity. Please note this same issue is not faced by our community members in Europe or North America.

BTW on a possibly related note, getting the latest bzr version from the ppa site as mentioned above, I noticed timeouts as well on apt-get install. Again it was almost as if the bandwidth was being throttled. The download would start at normal speed and gradually taper down to zero and completely pause. It took me multiple apt-get update commands to install bzr from the ppa repository above. Again all other apt-sources did not have this same issue and were working fine.

What more can I give you to analyze this issue apart from the bzr log?

Revision history for this message
Martin Pool (mbp) said :
#12

On 8 April 2010 18:52, chamindra <email address hidden> wrote:
> BTW on a possibly related note, getting the latest bzr version from the
> ppa site as mentioned above, I noticed timeouts as well on apt-get
> install. Again it was almost as if the bandwidth was being throttled.
> The download would start at normal speed and gradually taper down to
> zero and completely pause. It took me multiple apt-get update commands
> to install bzr from the ppa repository above. Again all other apt-
> sources did not have this same issue and were working fine.

This sounds a bit like a TCP-level issue, perhaps with path-mtu-discovery.

--
Martin <http://launchpad.net/~mbp/>

Can you help with this problem?

Provide an answer of your own, or ask chamindra for more information if necessary.

To post a message you must log in.