transient queues leak memory
We've written code in which a long-running daemon frequently creates and deletes temporary queues, but there does not seem to be any way in txamqp to properly clean up a queue once it is no longer needed. (Specifically, I'm referring to the TimeoutDeferred
I think that I could just work around this problem by doing "del client.queues[key]" after calling queue_delete and closing the queue, but this seems a bit crude. Is there a mechanism to do this cleanly? (Should there be one?)
Also, an aside: is there really any need for the deferredLock in this code in AMQClient in protocol.py?
@defer.
def queue(self, key):
yield self.queueLock.
try:
try:
q = self.queues[key]
except KeyError:
q = TimeoutDeferred
finally:
There are no yield statements between the acquire() and release() calls, so it doesn't seem like anything else could run concurrent with this function...
Thanks,
- Adam
-----
from twisted.internet import defer, reactor
from twisted.
from txamqp.protocol import AMQClient
from txamqp.client import TwistedDelegate
import txamqp.spec
class LeakTest(object):
def __init__(self, host, port, user, password, vhost, exchange, specfile):
self.host = host
self.port = port
self.user = user
self.vhost = vhost
@defer.
def temp_queue(self, client, channel, key):
yield channel.
yield channel.
sub = yield channel.
queue = yield client.
# XXX normally would do 'yield queue.get()' here or something
yield channel.
@defer.
def run(self):
spec = txamqp.
delegate = TwistedDelegate()
cc = ClientCreator(
# connect and auth
client = yield cc.connectTCP(
yield client.
# create a channel
channel = yield client.channel(1)
yield channel.
# create exchange
yield channel.
# create and delete 100 queues
for i in xrange(100):
yield self.temp_
# XXXXXXXXXXXXXXX
# here lies the problem:
# there are now 100 queues in this dictionary, even though
# we no longer need any of them!
# XXXXXXXXXXXXXXX
print client.queues
if __name__ == '__main__':
l = LeakTest(...)
reactor.
reactor.run()
Question information
- Language:
- English Edit question
- Status:
- Open
- For:
- txAMQP Edit question
- Assignee:
- No assignee Edit question
- Last query:
- Last reply:
Can you help with this problem?
Provide an answer of your own, or ask Adam Goodman for more information if necessary.