nova boot : Instance failed to spawn

Asked by Aqua

Ubuntu-12.04-server, Essex:

When follow the document <a>http://docs.openstack.org/essex/openstack-compute/admin/content/booting-a-test-image.html</a> to boot an instance by "nova boot --image cirros-0.3.0-x86_64 --flavor m1.small --key_name test my-first-server " , get the errors:

2012-06-21 17:44:58 TRACE nova.compute.manager [instance: 678200dd-97b9-43c2-be2a-fd9f111d10f2] Traceback (most recent call last):
2012-06-21 17:44:58 TRACE nova.compute.manager [instance: 678200dd-97b9-43c2-be2a-fd9f111d10f2] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 592, in _spawn
2012-06-21 17:44:58 TRACE nova.compute.manager [instance: 678200dd-97b9-43c2-be2a-fd9f111d10f2] self._legacy_nw_info(network_info), block_device_info)
2012-06-21 17:44:58 TRACE nova.compute.manager [instance: 678200dd-97b9-43c2-be2a-fd9f111d10f2] File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 114, in wrapped
2012-06-21 17:44:58 TRACE nova.compute.manager [instance: 678200dd-97b9-43c2-be2a-fd9f111d10f2] return f(*args, **kw)
2012-06-21 17:44:58 TRACE nova.compute.manager [instance: 678200dd-97b9-43c2-be2a-fd9f111d10f2] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py", line 916, in spawn
2012-06-21 17:44:58 TRACE nova.compute.manager [instance: 678200dd-97b9-43c2-be2a-fd9f111d10f2] block_device_info=block_device_info)
2012-06-21 17:44:58 TRACE nova.compute.manager [instance: 678200dd-97b9-43c2-be2a-fd9f111d10f2] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py", line 1525, in to_xml
2012-06-21 17:44:58 TRACE nova.compute.manager [instance: 678200dd-97b9-43c2-be2a-fd9f111d10f2] rescue, block_device_info)
2012-06-21 17:44:58 TRACE nova.compute.manager [instance: 678200dd-97b9-43c2-be2a-fd9f111d10f2] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py", line 1408, in _prepare_xml_info
2012-06-21 17:44:58 TRACE nova.compute.manager [instance: 678200dd-97b9-43c2-be2a-fd9f111d10f2] nics.append(self.vif_driver.plug(instance, network, mapping))
2012-06-21 17:44:58 TRACE nova.compute.manager [instance: 678200dd-97b9-43c2-be2a-fd9f111d10f2] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/vif.py", line 99, in plug
2012-06-21 17:44:58 TRACE nova.compute.manager [instance: 678200dd-97b9-43c2-be2a-fd9f111d10f2] return self._get_configurations(network, mapping)
2012-06-21 17:44:58 TRACE nova.compute.manager [instance: 678200dd-97b9-43c2-be2a-fd9f111d10f2] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/vif.py", line 69, in _get_configurations
2012-06-21 17:44:58 TRACE nova.compute.manager [instance: 678200dd-97b9-43c2-be2a-fd9f111d10f2] 'ip_address': mapping['ips'][0]['ip'],
2012-06-21 17:44:58 TRACE nova.compute.manager [instance: 678200dd-97b9-43c2-be2a-fd9f111d10f2] IndexError: list index out of range
2012-06-21 17:44:58 TRACE nova.compute.manager [instance: 678200dd-97b9-43c2-be2a-fd9f111d10f2]
2012-06-21 17:44:58 DEBUG nova.compute.manager [req-403b6b31-0954-404a-ab93-8ca55c206f43 e694a8fa0eaa4c93b11f6fb8edb53a2a 55a5adda88ba4c47a5472cca7478b626] [instance: 678200dd-97b9-43c2-be2a-fd9f111d10f2] Deallocating network for instance from (pid=16525) _deallocate_network /usr/lib/python2.7/dist-packages/nova/compute/manager.py:616
2012-06-21 17:44:58 DEBUG nova.rpc.amqp [req-403b6b31-0954-404a-ab93-8ca55c206f43 e694a8fa0eaa4c93b11f6fb8edb53a2a 55a5adda88ba4c47a5472cca7478b626] Making asynchronous cast on network... from (pid=16525) cast /usr/lib/python2.7/dist-packages/nova/rpc/amqp.py:346
2012-06-21 17:44:58 ERROR nova.rpc.amqp [req-403b6b31-0954-404a-ab93-8ca55c206f43 e694a8fa0eaa4c93b11f6fb8edb53a2a 55a5adda88ba4c47a5472cca7478b626] Exception during message handling
2012-06-21 17:44:58 TRACE nova.rpc.amqp Traceback (most recent call last):
2012-06-21 17:44:58 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line 252, in _process_data
2012-06-21 17:44:58 TRACE nova.rpc.amqp rval = node_func(context=ctxt, **node_args)
2012-06-21 17:44:58 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 114, in wrapped
2012-06-21 17:44:58 TRACE nova.rpc.amqp return f(*args, **kw)
2012-06-21 17:44:58 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 177, in decorated_function
2012-06-21 17:44:58 TRACE nova.rpc.amqp sys.exc_info())
2012-06-21 17:44:58 TRACE nova.rpc.amqp File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2012-06-21 17:44:58 TRACE nova.rpc.amqp self.gen.next()
2012-06-21 17:44:58 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 171, in decorated_function
2012-06-21 17:44:58 TRACE nova.rpc.amqp return function(self, context, instance_uuid, *args, **kwargs)
2012-06-21 17:44:58 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 651, in run_instance
2012-06-21 17:44:58 TRACE nova.rpc.amqp do_run_instance()
2012-06-21 17:44:58 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 945, in inner
2012-06-21 17:44:58 TRACE nova.rpc.amqp retval = f(*args, **kwargs)
2012-06-21 17:44:58 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 650, in do_run_instance
2012-06-21 17:44:58 TRACE nova.rpc.amqp self._run_instance(context, instance_uuid, **kwargs)
2012-06-21 17:44:58 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 451, in _run_instance
2012-06-21 17:44:58 TRACE nova.rpc.amqp self._set_instance_error_state(context, instance_uuid)
2012-06-21 17:44:58 TRACE nova.rpc.amqp File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2012-06-21 17:44:58 TRACE nova.rpc.amqp self.gen.next()
2012-06-21 17:44:58 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 432, in _run_instance
2012-06-21 17:44:58 TRACE nova.rpc.amqp self._deallocate_network(context, instance)
2012-06-21 17:44:58 TRACE nova.rpc.amqp File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2012-06-21 17:44:58 TRACE nova.rpc.amqp self.gen.next()
2012-06-21 17:44:58 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 429, in _run_instance
2012-06-21 17:44:58 TRACE nova.rpc.amqp injected_files, admin_password)
2012-06-21 17:44:58 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 592, in _spawn
2012-06-21 17:44:58 TRACE nova.rpc.amqp self._legacy_nw_info(network_info), block_device_info)
2012-06-21 17:44:58 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 114, in wrapped
2012-06-21 17:44:58 TRACE nova.rpc.amqp return f(*args, **kw)
2012-06-21 17:44:58 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py", line 916, in spawn
2012-06-21 17:44:58 TRACE nova.rpc.amqp block_device_info=block_device_info)
2012-06-21 17:44:58 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py", line 1525, in to_xml
2012-06-21 17:44:58 TRACE nova.rpc.amqp rescue, block_device_info)
2012-06-21 17:44:58 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py", line 1408, in _prepare_xml_info
2012-06-21 17:44:58 TRACE nova.rpc.amqp nics.append(self.vif_driver.plug(instance, network, mapping))
2012-06-21 17:44:58 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/vif.py", line 99, in plug
2012-06-21 17:44:58 TRACE nova.rpc.amqp return self._get_configurations(network, mapping)
2012-06-21 17:44:58 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/vif.py", line 69, in _get_configurations
2012-06-21 17:44:58 TRACE nova.rpc.amqp 'ip_address': mapping['ips'][0]['ip'],
2012-06-21 17:44:58 TRACE nova.rpc.amqp IndexError: list index out of range
2012-06-21 17:44:58 TRACE nova.rpc.amqp

My network configuration is like this:

/etc/network/interface:
auto eth0
iface eth0 inet static
address 16.158.51.71
netmask 255.255.252.0
gateway 16.158.48.1

auto br100
iface br100 inet static
bridge_ports eth1
bridge_stp off
bridge_maxwait 0
bridge_fd 0
address 192.168.0.1
netmask 255.255.0.0
broadcast 192.168.255.255

/etc/nova/nova.conf

...
--libvirt_use_virtio_for_bridges
--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--public_interface=eth0
--vlan_interface=eth0
--fixed_range=192.168.0.0/16
--network_manager=nova.network.manager.VlanManager
--network_size=31
--routing_source_ip=102.168.0.1
...

nova-manage network list:
id IPv4 IPv6 start address DNS1 DNS2 VlanID project uuid
1 192.168.0.0/24 None 192.168.0.3 None None 100 55a5adda88ba4c47a5472cca7478b626 72fb803c-3906-4186-bab6-64d432381c0e

Could any one help why this happens?
Thanks in advance.

Question information

Language:
English Edit question
Status:
Solved
For:
OpenStack Compute (nova) Edit question
Assignee:
No assignee Edit question
Solved by:
Aqua
Solved:
Last query:
Last reply:
Revision history for this message
Carmelo Lucio (clucio) said :
#1

Looks like your routing source IP is incorrect, also, I believe the interfaces should be br100 and not eth0 in your nova.conf...

Revision history for this message
Aqua (potato-hi) said :
#2

Sirry for the typo, I changed /etc/nova/nova.conf three lines:

--routing_source_ip=192.168.0.1
--public_interface=br100
--vlan_interface=br199

and resart nova services, execute nova boot again, it appears still the same errors.

Revision history for this message
Aqua (potato-hi) said :
#3

sorry again it's
--vlan_interface=br100

Revision history for this message
Aqua (potato-hi) said :
#4

Seems the ips in fixed_ips table are not match my configuration file.
I clear the database and restart the nova-network, it works.

Thanks.

Revision history for this message
Aqua (potato-hi) said :
#5

Seems the ips in fixed_ips table are not match my configuration file.
I clear the database and restart the nova-network, it works.

Thanks.

Revision history for this message
yashiang (yashiang1986) said :
#6

Hi all,
I meet the same problem too.
It happened after I assigned network-ip pair by using "--nic" option.
Before I assigned ip by myself, it works well.
But after that, instance boot always failed.

Revision history for this message
nymous (d1pro) said :
#7

Sometimes network api returns 2 network entries, one having IP, another not having IP. In this case the error rises.

Revision history for this message
Jerry Zhao (zhaoxinyu) said :
#8

Sometimes, the quota of fixed ips is used up. bumping up the quota for fixed ips fixed the problem for me.