OpenStack Compute (nova)

How to set cpuset attribute in vcpu element in instance xml?

Asked by Peeyush Gupta

Hi, in relation to the changes shown here : https://blueprints.launchpad.net/nova/+spec/instance-vcpu-pin , I want to set cpuset attribute. But I can't enable this. I tried to add vcpu_pin_set to nova.conf , but it's not working. I am using devstack on Ubuntu 12.04. What am I doing wrong?

Question information

Language:
English Edit question
Status:
Solved
For:
OpenStack Compute (nova) Edit question
Assignee:
No assignee Edit question
Solved by:
wangpan
Solved:
Last query:
Last reply:
Revision history for this message
wangpan (hzwangpan) said : 		#1

1.what hypervisor are you using?
2.how many pcpus you have?
3.what the vcpu_pin_set config you set?
4.can you give me the libvirt.xml of your instance?

2013-06-27

Wangpan

发件人：Peeyush Gupta <email address hidden>
发送时间：2013-06-26 16:41
主题：[Question #231384]: How to set cpuset attribute in vcpu element in instance xml?
收件人："hzwangpan"<email address hidden>
抄送：

New question #231384 on OpenStack Compute (nova):
https://answers.launchpad.net/nova/+question/231384

Hi, in relation to the changes shown here : https://blueprints.launchpad.net/nova/+spec/instance-vcpu-pin , I want to set cpuset attribute. But I can't enable this. I tried to add vcpu_pin_set to nova.conf , but it's not working. I am using devstack on Ubuntu 12.04. What am I doing wrong?

--
You received this question notification because you are a direct
subscriber of the question.

Revision history for this message
Peeyush Gupta (peeygupt) said : 		#2

Hi wangpan, thanks for the reply. I am running Ubuntu 12.04 as a VM on Fedora 18 host. Here are the details:

1. I am using kvm.
2. I have 3 pcpus.
3. I set vcpu_pin_set = 2 in nova.conf
4. Here is the xml:

<domain type='qemu'>
  <name>instance-00000009</name>
  <uuid>08c556d1-1e23-484e-94b2-e973eeb1b936</uuid>
  <memory>524288</memory>
  <currentMemory>524288</currentMemory>
  <vcpu>1</vcpu>
  <sysinfo type='smbios'>
    <system>
      <entry name='manufacturer'>OpenStack Foundation</entry>
      <entry name='product'>OpenStack Nova</entry>
      <entry name='version'>2013.2</entry>
      <entry name='serial'>bd348807-a480-1b3f-eb66-65b2d88cba58</entry>
      <entry name='uuid'>08c556d1-1e23-484e-94b2-e973eeb1b936</entry>
    </system>
  </sysinfo>
  <os>
    <type arch='x86_64' machine='pc-1.0'>hvm</type>
    <boot dev='hd'/>
    <smbios mode='sysinfo'/>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/opt/stack/data/nova/instances/08c556d1-1e23-484e-94b2-e973eeb1b936/disk'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <interface type='bridge'>
      <mac address='fa:16:3e:79:07:c0'/>
      <source bridge='br100'/>
      <model type='virtio'/>
      <driver name='qemu'/>
      <filterref filter='nova-instance-instance-00000009-fa163e7907c0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='file'>
      <source path='/opt/stack/data/nova/instances/08c556d1-1e23-484e-94b2-e973eeb1b936/console.log'/>
      <target port='0'/>
    </serial>
    <serial type='pty'>
      <target port='1'/>
    </serial>
    <console type='file'>
      <source path='/opt/stack/data/nova/instances/08c556d1-1e23-484e-94b2-e973eeb1b936/console.log'/>
      <target type='serial' port='0'/>
    </console>
    <input type='tablet' bus='usb'/>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' listen='127.0.0.1' keymap='en-us'>
      <listen type='address' address='127.0.0.1'/>
    </graphics>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </memballoon>
  </devices>
</domain>

                                                                                                                                            1,1 Top

Revision history for this message
wangpan (hzwangpan) said : 		#3

where you added the vcpu_pin_set = 1 to nova.conf? the [DEFAULT] section or just at the end of nova.conf?
if you didn't add it to the correct [DEFAULT] section, it will be invalid.
you can try ot add it like this:
ubuntu@wp-devstack:/opt/stack/nova$ head /etc/nova/nova.conf

[DEFAULT]
vcpu_pin_set = 1
log_dir=/var/log/nova
compute_driver = libvirt.LibvirtDriver
firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver

2013-06-27

Wangpan

发件人：Peeyush Gupta <email address hidden>
发送时间：2013-06-27 13:26
主题：Re: [Question #231384]: How to set cpuset attribute in vcpu element in instance xml?
收件人："hzwangpan"<email address hidden>
抄送：

Question #231384 on OpenStack Compute (nova) changed:
https://answers.launchpad.net/nova/+question/231384

    Status: Answered => Open

Peeyush Gupta is still having a problem:
Hi wangpan, thanks for the reply. I am running Ubuntu 12.04 as a VM on
Fedora 18 host. Here are the details:

1. I am using kvm.
2. I have 3 pcpus.
3. I set vcpu_pin_set = 2 in nova.conf
4. Here is the xml:

<domain type='qemu'>
  <name>instance-00000009</name>
  <uuid>08c556d1-1e23-484e-94b2-e973eeb1b936</uuid>
  <memory>524288</memory>
  <currentMemory>524288</currentMemory>
  <vcpu>1</vcpu>
  <sysinfo type='smbios'>
    <system>
      <entry name='manufacturer'>OpenStack Foundation</entry>
      <entry name='product'>OpenStack Nova</entry>
      <entry name='version'>2013.2</entry>
      <entry name='serial'>bd348807-a480-1b3f-eb66-65b2d88cba58</entry>
      <entry name='uuid'>08c556d1-1e23-484e-94b2-e973eeb1b936</entry>
    </system>
  </sysinfo>
  <os>
    <type arch='x86_64' machine='pc-1.0'>hvm</type>
    <boot dev='hd'/>
    <smbios mode='sysinfo'/>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/opt/stack/data/nova/instances/08c556d1-1e23-484e-94b2-e973eeb1b936/disk'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <interface type='bridge'>
      <mac address='fa:16:3e:79:07:c0'/>
      <source bridge='br100'/>
      <model type='virtio'/>
      <driver name='qemu'/>
      <filterref filter='nova-instance-instance-00000009-fa163e7907c0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='file'>
      <source path='/opt/stack/data/nova/instances/08c556d1-1e23-484e-94b2-e973eeb1b936/console.log'/>
      <target port='0'/>
    </serial>
    <serial type='pty'>
      <target port='1'/>
    </serial>
    <console type='file'>
      <source path='/opt/stack/data/nova/instances/08c556d1-1e23-484e-94b2-e973eeb1b936/console.log'/>
      <target type='serial' port='0'/>
    </console>
    <input type='tablet' bus='usb'/>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' listen='127.0.0.1' keymap='en-us'>
      <listen type='address' address='127.0.0.1'/>
    </graphics>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </memballoon>
  </devices>
</domain>

1,1 Top

--
You received this question notification because you are a direct
subscriber of the question.

Revision history for this message
Peeyush Gupta (peeygupt) said : 		#4

I have added it to the [DEFAULT] section only. When I did head /etc/nova/nova.conf, I got this:

root@peeyush-virtual-machine:/opt/stack/nova# head /etc/nova/nova.conf

[DEFAULT]
vcpu_pin_set = 2
log_dir = /var/log/nova
firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver
compute_driver = libvirt.LibvirtDriver
flat_interface = eth0
flat_network_bridge = br100
vlan_interface = eth0
public_interface = br100

I launched a new instance after that and it's still not working. Still there is no cpuset attribute in the vcpu element.

Revision history for this message
wangpan (hzwangpan) said : 		#5

have you restart your nova processes after added vcpu_pin_set to nova.conf? such as nova-compute?
and you may grep 'vcpu_pin_set' /var/log/nova/nova-compute.log and check the result,
mine is:
2013-06-27 05:51:41.335 DEBUG nova.openstack.common.service [-] vcpu_pin_set = 1 from (pid=23026) log_opt_values /usr/local/lib/python2.7/dist-packages/oslo/config/cfg.py:1481

2013-06-27

Wangpan

发件人：Peeyush Gupta <email address hidden>
发送时间：2013-06-27 15:01
主题：Re: [Question #231384]: How to set cpuset attribute in vcpu element in instance xml?
收件人："hzwangpan"<email address hidden>
抄送：

Question #231384 on OpenStack Compute (nova) changed:
https://answers.launchpad.net/nova/+question/231384

    Status: Answered => Open

Peeyush Gupta is still having a problem:
I have added it to the [DEFAULT] section only. When I did head
/etc/nova/nova.conf, I got this:

root@peeyush-virtual-machine:/opt/stack/nova# head /etc/nova/nova.conf

[DEFAULT]
vcpu_pin_set = 2
log_dir = /var/log/nova
firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver
compute_driver = libvirt.LibvirtDriver
flat_interface = eth0
flat_network_bridge = br100
vlan_interface = eth0
public_interface = br100

I launched a new instance after that and it's still not working. Still
there is no cpuset attribute in the vcpu element.

--
You received this question notification because you are a direct
subscriber of the question.

Revision history for this message
Peeyush Gupta (peeygupt) said : 		#6

Okay, looks like devstack doesn't have services, it has screens and I am not sure how to restart them. I am gonna need some time to figure that out. Plus, for God knows what reason my devstack setup doesn't log anything!

Revision history for this message
Best wangpan (hzwangpan) said : 		#7

you should mkdir for log_dir(/var/log/nova) and chown to the user nova process used.
and in my devstack(host is ubuntu 12.04), I just killed the nova-compute process by kill pid-of-nova-compute,
and then start it manually by:
nohup nova-compute 2>/dev/null &
by the way, if the log of nova-compute is nothing after you mkdir and chown, you can start nova-compute by:
nohup nova-compute 2>/tmp/nova-compute.log &
and you can see the log at /tmp/nova-compute.log.

2013-06-27

Wangpan

发件人：Peeyush Gupta <email address hidden>
发送时间：2013-06-27 15:36
主题：Re: [Question #231384]: How to set cpuset attribute in vcpu element in instance xml?
收件人："hzwangpan"<email address hidden>
抄送：

Question #231384 on OpenStack Compute (nova) changed:
https://answers.launchpad.net/nova/+question/231384

    Status: Answered => Open

Peeyush Gupta is still having a problem:
Okay, looks like devstack doesn't have services, it has screens and I am
not sure how to restart them. I am gonna need some time to figure that
out. Plus, for God knows what reason my devstack setup doesn't log
anything!

--
You received this question notification because you are a direct
subscriber of the question.

Revision history for this message
Peeyush Gupta (peeygupt) said : 		#8

Hi wangpan, sorry for replying late, I was busy with some other stuff. I tried it out and guess what, it's working! Thank you very much! One more thing, is there any way to pin cpus dynamically? Meaning that when I give "nova boot" command can I decide which cpus I am going to pin this particular VM and the result reflects in the VM's xml? Does this mechanism works for each instance or just for making groups for libvirt to use?

Revision history for this message
Peeyush Gupta (peeygupt) said : 		#9

Thanks wangpan, that solved my question.

Revision history for this message
wangpan (hzwangpan) said : 		#10

is there any way to pin cpus dynamically?
I believe there is no way to do this, currently the vcpu_pin_set affects all instances.

2013-07-01

Wangpan

发件人：Peeyush Gupta <email address hidden>
发送时间：2013-07-01 16:16
主题：Re: [Question #231384]: How to set cpuset attribute in vcpu element in instance xml?
收件人："hzwangpan"<email address hidden>
抄送：

Question #231384 on OpenStack Compute (nova) changed:
https://answers.launchpad.net/nova/+question/231384

    Status: Answered => Solved

Peeyush Gupta confirmed that the question is solved:
Hi wangpan, sorry for replying late, I was busy with some other stuff. I
tried it out and guess what, it's working! Thank you very much! One more
thing, is there any way to pin cpus dynamically? Meaning that when I
give "nova boot" command can I decide which cpus I am going to pin this
particular VM and the result reflects in the VM's xml? Does this
mechanism works for each instance or just for making groups for libvirt
to use?

--
You received this question notification because you are a direct
subscriber of the question.

Revision history for this message
Peeyush Gupta (peeygupt) said : 		#11

Okay, thanks again for your time.

Revision history for this message
ari (ari-avitzur) said : 		#12

I have an HP server with 12 cores. I want to install a VM on this server with OpenStack and set the cpuset values to 0-11. Currently when installing the vm I see in the xml that the cpuset = '3-5,9-11''

at /etc/nova/nova.conf I don't have the parameter vcpu_pin_set.

two questions: 1. how do I configure cpuset to 0-11?
 2. why cpuset is equal to 3-5,9-11 and what is the defalut value of vcpu_pin_set?

Thank you.