OpenStack Compute (nova)

nova-instancemonitor libvirt issue

Asked by Guido Davide Dall'Olio

Hello.
I'm quite new at Openstack and I've used it for few weeks.
I'm trying to log host stats(cpu, memory, network, disk) by nova-instancemonitor service.
Everything works fine...but after 1 minute (without any warning) I get an error.
Same error coming from libvirt.

(nova): TRACE: Traceback (most recent call last):
(nova): TRACE: File "/usr/local/bin/nova-instancemonitor", line 5, in <module>
(nova): TRACE: pkg_resources.run_script('nova==2011.2', 'nova-instancemonitor')
(nova): TRACE: File "/usr/lib/python2.6/dist-packages/pkg_resources.py", line 461, in run_script
(nova): TRACE: self.require(requires)[0].run_script(script_name, ns)
(nova): TRACE: File "/usr/lib/python2.6/dist-packages/pkg_resources.py", line 1194, in run_script
(nova): TRACE: execfile(script_filename, namespace, namespace)
(nova): TRACE: File "/usr/local/lib/python2.6/dist-packages/nova-2011.2-py2.6.egg/EGG-INFO/scripts/nova-instancemonitor", line 49, in <module>
(nova): TRACE: twistd.serve(__file__)
(nova): TRACE: File "/usr/local/lib/python2.6/dist-packages/nova-2011.2-py2.6.egg/nova/twistd.py", line 267, in serve
(nova): TRACE: twistd.runApp(options)
(nova): TRACE: File "/usr/lib/python2.6/dist-packages/twisted/scripts/twistd.py", line 23, in runApp
(nova): TRACE: _SomeApplicationRunner(config).run()
(nova): TRACE: File "/usr/lib/python2.6/dist-packages/twisted/application/app.py", line 423, in run
(nova): TRACE: self.postApplication()
(nova): TRACE: File "/usr/lib/python2.6/dist-packages/twisted/scripts/_twistd_unix.py", line 205, in postApplication
(nova): TRACE: self.startApplication(self.application)
(nova): TRACE: File "/usr/lib/python2.6/dist-packages/twisted/scripts/_twistd_unix.py", line 317, in startApplication
(nova): TRACE: app.startApplication(application, not self.config['no_save'])
(nova): TRACE: File "/usr/lib/python2.6/dist-packages/twisted/application/app.py", line 713, in startApplication
(nova): TRACE: service.IService(application).startService()
(nova): TRACE: File "/usr/lib/python2.6/dist-packages/twisted/application/service.py", line 278, in startService
(nova): TRACE: service.startService()
(nova): TRACE: File "/usr/local/lib/python2.6/dist-packages/nova-2011.2-py2.6.egg/nova/compute/monitor.py", line 416, in startService
(nova): TRACE: self.updateInstances()
(nova): TRACE: File "/usr/local/lib/python2.6/dist-packages/nova-2011.2-py2.6.egg/nova/compute/monitor.py", line 439, in updateInstances
(nova): TRACE: domain_ids = conn.list_instances()
(nova): TRACE: File "/usr/local/lib/python2.6/dist-packages/nova-2011.2-py2.6.egg/nova/virt/libvirt_conn.py", line 286, in list_instances
(nova): TRACE: for x in self._conn.listDomainsID()]
(nova): TRACE: File "/usr/local/lib/python2.6/dist-packages/nova-2011.2-py2.6.egg/nova/virt/libvirt_conn.py", line 248, in _get_connection
(nova): TRACE: self.read_only)
(nova): TRACE: File "/usr/local/lib/python2.6/dist-packages/nova-2011.2-py2.6.egg/nova/virt/libvirt_conn.py", line 280, in _connect
(nova): TRACE: return libvirt.openReadOnly(uri)
(nova): TRACE: File "/usr/lib/python2.6/dist-packages/libvirt.py", line 176, in openReadOnly
(nova): TRACE: if ret is None:raise libvirtError('virConnectOpenReadOnly() failed')
(nova): TRACE: libvirtError: cannot send data: Broken pipe
(nova): TRACE:

It's very strange as the service during this activity connects to libvirt many times without any problem
for example:
2011-08-21 22:39:42,719 DEBUG nova.virt.libvirt_conn [-] Connecting to libvirt: qemu:///system from (pid=2061) _get_connection /usr/local/lib/python2.6/dist-packages/nova-2011.2-py2.6.egg/nova/virt/libvirt_conn.py:246
and it works fine, but suddendly after ca 1 minute it stops(crash) and on the log I read that problem.
I don't understand as libvirt works many time during "connecting" attempts, but after 1 minute it returns an error.

Can someone help me?
Thanks in advance

Question information

Language:
English Edit question
Status:
Answered
For:
OpenStack Compute (nova) Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Brian Lamar (blamar) said : 		#1

nova-instancemonitor was removed from the code and will not be in the next OpenStack Compute release (diablo). This code was not being maintained regularly and it was recently decided to remove it and all other python-twisted based code.

Revision history for this message
Guido Davide Dall'Olio (guidodavide-dallolio) said : 		#2

ok, but if I wanted to do some load balancing, instancemonitor would be a great idea.
I modified instance monitor to do some rcp calls and it works perfectly...if that service won't be maintained, I won't know what to do.
Any suggestion? I'm trying to do some load balancing...but without instance monitor!?

Can you help with this problem?

Provide an answer of your own, or ask Guido Davide Dall'Olio for more information if necessary.

To post a message you must log in.