Gateway addresses for subnets are one off.

I am running folsom on ubuntu 12.04. I used this guide. http://docs.openstack.org/folsom/basic-install/content/basic-install_intro.html

The gateway ips generally taken by L3 routers. You can create a router in quantum or use a physical router which takes that IP.

Ya, I understand that, but why is an ip address being created as .2 and not .1?

Hi, Doug. The second IP address (.2 in this case) is used by the DHCP agent. If you list your running processes and look for dnsmasq you'll see a command line option --interface=tap882a8e75-ee for one agent and --interface=tap3680cb06-ab for the other. If you use namespaces, the DHCP agent will also have its own namespace qdhcp-*.

Ok, so the .2 address is handing out IPs. Ok. Thats fine. So why does the dashboard prompt me for a gateway address? Do I have to create a gateway address myself?

the gateway IP address of a subnet will default to being the first IP in the subnet (e.g., the .1 address for a subnet that has a range from .0-.255)

however, you can override this default if you choose, as for example, some people prefer the gateway IP to be another address (e.g., .the last IP in a range, such as .254)

Alright. I understand. The problem is, when my vm wants to leave the subnet, it will send an arp out for the gateway address of .1. The arp will go over the gre tunnel to the l3 agent. But the gateway ip address of .1 does not exist on the layer 3 agent, so there will be no response to the arp. If I issue the command "ifconfig -a" on the box that is running the layer 3 agent I don't see a .1 address anywhere. So my question is this, if I create a gateway address for a tenant subnet via the dashboard. Will it actually get created anywhere...

It surely will. By default Quantum uses namespaces, so that each tenant runs in a separated namespace with their own network interfaces, IP addresses, routing tables, iptables rules, etc. You're not seeing your gateway interface because your looking for it in the initial namespace, where commands run if not explictly specified:

Run 'ip netns' and will see a list of your namespaces, for example
qdhcp-5ceadbb9-33d0-4080-83fd-066f9e672d8e
qrouter-9839d33c-cda7-4509-b93d-52cc55a3a174

The qrouter-* namespace corresponds to each router that you have, where the hexadecimal string corresponds to the id of the router (quantum router-list). There you'll find the router's external network IP address if any configured and the router's interface in that network (acting as the gateway). Example:

# ip netns exec qrouter-9839d33c-cda7-4509-b93d-52cc55a3a174 ifconfig
(...)
qg-01f0ca8d-a7 Link encap:Ethernet HWaddr fa:16:3e:bf:a7:b8
          inet addr:192.168.100.225 Bcast:192.168.100.255 Mask:255.255.255.0
(...)
qr-56c3fa8b-63 Link encap:Ethernet HWaddr fa:16:3e:85:0c:85
          inet addr:10.5.5.1 Bcast:10.5.5.255 Mask:255.255.255.0
(...)

qr-* is the gateway you're looking for and qg-* is my router's external network interface. The string is port id of the router.

The same command 'ip netns exec <namespace>' can be used with all other ordinary network tools like 'iptables, route, netstat, tcpdump, ping, ifup/down, ip *, etc. That's how Linux makes it possible for overlapping IPs, since each network namespace is an isolated environment (virtualization at the OS level).

Hope that helps.

I'm running Ubuntu 12.04.1 LTS kernel 3.2.0-33 generic - Running on box where l3 agent, dhcp agent, ovs agent is running
I'm running Ubuntu 12.04.1 LTS kernel 3.2.0-29-generic - Running on compute node where ovs agent is running

When I run "ip netns" on both the compute node and box where l3 agent, dhcp agent, ovs agent is running, I dont see anything. In googling around I believe I saw at one point the 3.2.0-33 kernel was supported. The 29 one I am going look into now. Am I not seeing any output from ip netns because network namespaces aren't supported?

Hi Doug,
Ricardo (ricardorolim) has explained very well.
Please check your l3_agent.ini and dhcp_agent.ini in l3agent& dhcp agent node.
There are something like that:
"
use_namespaces = true
"
and the default use_namespaces value is true.

good luck for u.

Sorry, I didn't see you're using the docs.openstack.org/folsom/basic-install/content/basic-install_intro.html guide where namespaces are disabled. That's why you don't see anything with 'ip netns'. So if you don't see the router's gateway interface IP in the network (.1), did you add an interface in the router for that network (quantum router-interface-add)?

Alright I have some more information, below is a piece of the /etc/quantum/l3_agent.ini file.

# Allow overlapping IP (Must have kernel build with CONFIG_NET_NS=y and
# iproute2 package that supports namespaces).
use_namespaces = False

# If use_namespaces is set as False then the agent can only configure one router.
# This is done by setting the specific router_id.
router_id = 296ccc78-1766-41c9-9ead-713a0843070f

I checked the internal ports on router_id "296ccc78-1766-41c9-9ead-713a0843070f" with

quantum port-list -- --device_id=296ccc78-1766-41c9-9ead-713a0843070f

And I saw both the internal port with ip of 10.6.6.1 and the external port with ip 192.168.2.130

Any idea why I'm still not seeing any 10.6.6.1 ip address with ifconfig -a? And yes, you are right I am not using namespaces.

Alright I got it working now. I see the gateway for one of the private subnets from my vm just fine. The only problem is when I try to create another network through the dashboard. The dashboard creates a dhcp server to hand out ips for that subnet but a gateway won't be created for that subnet on the layer 3 agent. I am not using namespaces. Is it possible to not use namespaces and have more then one private network off of a layer 3 agent?

I got it. For whatever reason when I go to create a new network within the tenant, a gateway isn't created on the layer3 agent. I have to go in and manually add it to the layer 3 agent. Good stuff though. Thanks for the help.

Hi Doug,

I see you've now marked this question as solved.
However, you should not go and create gateways manually. Quantum should do that for you.
It might be good to understand why that's not happening, and I would be glad if you could share some more information on this issue.

It seems you've followed all the steps, including adding the subnet to the router with router-interface-add.
What did you had to do in order to manually configure the gateway on the l3 agent?
- did you had to create the ovs port and attach it to br-int?
- or did you had to just configure the ip?

It would be great if could post the whole contents of the l3agent.ini file. This might be obvious, but ensure you're running also the l2 agent on the node where the l3 agent is running.

I can also confirm that using namespaces you can have multiple private networks handled by the same l3 agent, even if they're connected by different routers. The only restriction of the Folsom l3 agent is that only a single external network per agent is supported at the moment.

So all I had to do was add the subnet to the router and it worked. So the first thing I did was do a "quantum subnet-list", I noticed that the subnet I made was in the list. I then did a "quantum port-list -- --device_id=76930773-9c65-4742-9e43-2efccbd66d7f" and noticed the subnet I created wasn't "connected" to that router. So then I did a "quantum router-interface-add 76930773-9c65-4742-9e43-2efccbd66d7f 67f4fa58-b293-439f-961a-8b892d4313f6" And that created the gateway address on the router. Like I said before though for whatever reason when I create the subnet in the dashboard, it won't attach the subnet I created to the router for that tenant, and obviously if it won't do that then no gateway for that subnet gets created. That might be something specific to my config or a bug, not really sure.

############
[DEFAULT]
# Show debugging output in log (sets DEBUG log level output)
debug = True

# L3 requires that an interface driver be set. Choose the one that best
# matches your plugin.

# OVS
interface_driver = quantum.agent.linux.interface.OVSInterfaceDriver
# LinuxBridge
#interface_driver = quantum.agent.linux.interface.BridgeInterfaceDriver

# The Quantum user information for accessing the Quantum API.
auth_url = http://192.168.1.2:35357/v2.0
auth_region = RegionOne
admin_tenant_name = service
admin_user = quantum
admin_password = test123

# Use "sudo quantum-rootwrap /etc/quantum/rootwrap.conf" to use the real
# root filter facility.
# Change to "sudo" to skip the filtering and just run the comand directly
root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf
#root_helper = sudo /usr/bin/quantum-rootwrap /etc/quantum/rootwrap.conf

# Allow overlapping IP (Must have kernel build with CONFIG_NET_NS=y and
# iproute2 package that supports namespaces).
use_namespaces = False

# If use_namespaces is set as False then the agent can only configure one router.
# This is done by setting the specific router_id.
router_id = 76930773-9c65-4742-9e43-2efccbd66d7f

# Each L3 agent can be associated with at most one external network. This
# value should be set to the UUID of that external network. If empty,
# the agent will enforce that only a single external networks exists and
# use that external network id
gateway_external_net_id = a25c67f6-cfd7-4f97-b550-450ceac90b7a

# Indicates that this L3 agent should also handle routers that do not have
# an external network gateway configured. This option should be True only
# for a single agent in a Quantum deployment, and may be False for all agents
# if all routers must have an external network gateway
# handle_internal_only_routers = True

# Name of bridge used for external network traffic. This should be set to
# empty value for the linux bridge
# external_network_bridge = br-ex

# IP address used by Nova metadata server
metadata_ip = 192.168.1.2

# TCP Port used by Nova metadata server
# metadata_port = 8775

# The time in seconds between state poll requests
# polling_interval = 3