instance get more than one fixed ip(grizzly-g3) - Ask OpenStack: Q&A Site for OpenStack Users and Developers
Description: i have install grizzly-g3, but quantum does not work well, when i boot 128 instances, i found one of instances got more than one fixed ip, howerver, when i boot 64 intances, it nerver happened, besides that , sometimes i can not ping vm with floatingip, i did not find any error message in my quantum log( all the files in the /var/log/quantum), follows are the error output and configurations | 97a93600-38e2-4700-9851-15ef56c1d628 | slave | ACTIVE | demo-int-net=172.16.100.4 | | 99aeb6b8-4252-4839-a7d1-f87853116100 | slave | ACTIVE | demo-int-net=172.16.100.117 | | 9aa82a35-c9f1-4f44-a108-d14e74eec231 | slave | ACTIVE | demo-int-net=172.16.100.108, 172.16.100.109 | | 9b6b1289-c450-4614-b647-e5ebdffff80a | slave | ACTIVE | demo-int-net=172.16.100.5 | | 9e0d3aa5-0f15-4b24-944a-6d6c3e18ce64 | slave | ACTIVE | demo-int-net=172.16.100.35 | | 9ea62124-9128-43cc-acdd-142f1e7743d6 | slave | ACTIVE | demo-int-net=172.16.100.132 | my setup : one db host(db service), one glance host(glance service) [cell cut: 3773 characters, first 1000 kept]
Keywords: ,Ask, OpenStack,forum,community,cloud,iaas

First time here? Check out the FAQ!

English ▼
Hi there! Please sign in tags users badges help

| ALL UNANSWERED |  | Ask Your Question

1

# instance get more than one fixed ip(grizzly-g3)

- neutron
- migrated

asked 2013-03-26 01:09:38 -0600

xinxin-shu
11 ●1 ●2 ●3

i have install grizzly-g3, but quantum does not work well, when i boot 128 instances, i found one of instances got more than one fixed ip, howerver, when i boot 64 intances, it nerver happened, besides that , sometimes i can not ping vm with floatingip, i did not find any error message in my quantum log( all the files in the /var/log/quantum), follows are the error output and configurations

| 97a93600-38e2-4700-9851-15ef56c1d628 | slave | ACTIVE | demo-int-net=172.16.100.4 | | 99aeb6b8-4252-4839-a7d1-f87853116100 | slave | ACTIVE | demo-int-net=172.16.100.117 | | 9aa82a35-c9f1-4f44-a108-d14e74eec231 | slave | ACTIVE | demo-int-net=172.16.100.108, 172.16.100.109 | | 9b6b1289-c450-4614-b647-e5ebdffff80a | slave | ACTIVE | demo-int-net=172.16.100.5 | | 9e0d3aa5-0f15-4b24-944a-6d6c3e18ce64 | slave | ACTIVE | demo-int-net=172.16.100.35 | | 9ea62124-9128-43cc-acdd-142f1e7743d6 | slave | ACTIVE | demo-int-net=172.16.100.132 |

my setup : one db host(db service), one glance host(glance service), on api host(keystone,nova-api,nova-scheduler, nova-conductor, quantum-server,quantum-dhcp, quantum-l3-agent,quantum-plugin-openvswitch-agent), eight compute host(each host with nova-compute, quantum-plugin-openvswitch-agent), i check that all the service on all hosts works well

i used vlan type network and openvswitch plugin:

my quantum.conf

[DEFAULT]

# Default log level is INFO

# verbose and debug has the same result.

# One of them will set DEBUG log level output

debug = True

# Address to bind the API server

bind_host = 0.0.0.0

# Port the bind the API server to

bind_port = 9696

# Quantum plugin provider module

# core_plugin =

core_plugin = quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2

# Advanced service modules

# service_plugins =

# Paste configuration file

api_paste_config = /etc/quantum/api-paste.ini

# The strategy to be used for auth.

# Supported values are 'keystone'(default), 'noauth'.

auth_strategy = keystone

# Modules of exceptions that are permitted to be recreated

# upon receiving exception data from an rpc call.

# allowed_rpc_exception_modules = quantum.openstack.common.exception, nova.exception

# AMQP exchange to connect to if using RabbitMQ or QPID

control_exchange = quantum

# RPC driver. DHCP agents needs it.

notification_driver = quantum.openstack.common.notifier.rpc_notifier

# default_notification_level is used to form actual topic name(s) or to set logging level

default_notification_level = INFO

# Defined in rpc_notifier, can be comma separated values.

# The actual topic names will be %s.%(default_notification_level)s

notification_topics = notifications

[QUOTAS]

# resource name(s) that are supported in quota features

# quota_items = network,subnet,port

# default number of resource allowed per tenant, minus for unlimited

# default_quota = -1

# number of networks allowed per tenant, and minus means unlimited

# quota_network = 10

# number of subnets allowed per tenant, and minus means unlimited

# quota_subnet = 10

# number of ports allowed per tenant, and minus means unlimited

quota_port = 5000 quota_floatingip = 5000

# default driver to use for quota checks

# quota_driver = quantum.quota.ConfDriver

# =========== items for agent management extension =============

# Seconds to regard the agent as down.

# agent_down_time = 5

# =========== end of items for agent management extension =====

[DEFAULT_SERVICETYPE]

# Description of the default service type (optional)

# description = "default service type"

# Enter a service definition line for each advanced service provided

# by the default service type.

# Each service definition should be in the following format:

# <service>:<plugin>[:driver]

[SECURITYGROUP]

# If set to true this allows quantum to receive proxied security group calls from nova

# proxy_mode = False

[AGENT] root_helper = sudo quantum-rootwrap /etc/quantum ... (more)

edit retag flag offensive close merge delete
add a comment

## 31 answers

Sort by » oldest newest most voted
« previous 1...1 2 3 4...4 next »
0

answered 2013-07-03 17:28:02 -0600

arosen
96 ●3 ●6 http://blog.aaronorose...

Hi Dan,

I filled this blueprint a while ago but haven't gotten a chance to start implementing it https://blueprints.launchpad.net/nova... . I think this will help solve a lot these orchestration issues we have with how nova and quantum work together. What i'm planning for the blueprint is to basically force the clients to create a port in quantum first; then pass the port-id to nova when booting an instance (and to keep the api backwords compatitble allocate the port on the nova-api server if a network is passed in). This should make things simplier and help with this imo. Here is the discuession we had on the list a while ago about this: http://lists.openstack.org/pipermail/...

edit flag offensive delete link more
add a comment
0

answered 2013-06-28 21:13:26 -0600

dmercer
1

I know that we're chiming in on this a bit late, but we recently ran into this issue in our (grizzly+, nvp backed) environment as well.

Thanks for the patch ( https://review.openstack.org/#/c/32691/ ). We will experiment with the new configuration settings and see if this helps us avoid doubly-allocated ports.

I'd like to note that the patch above seems to only be a means of working around the underlying issue -- which AFAICT is coordination of state between nova and neutron. The failure of a discrete api call due to a timeout or some other issue should always be expected and duplicate/retry api calls should be able to be handled in an idempotent manner by the resource manager. Maybe I missed some part of the discussion where this was called out as a larger issue that needs to be addressed and the patch above is just an expedient fix, or perhaps I could be entirely off-base with my understanding of the issue.

Are there any ongoing (or even dusty!) discussions around the way state is managed between openstack services?

edit flag offensive delete link more
add a comment
0

answered 2013-04-12 07:13:46 -0600

snaiksat
1

Okay, but why is nova requesting that two ports be created just in this case when it should be asking for only one port?

edit flag offensive delete link more
add a comment
0

answered 2013-04-12 05:37:46 -0600

danwent
1

Sumit, this is because the order is actually the other way around. Think of it as being that nova contacts quantum to create ports, then when the nova virt layer is actually creating the VM, it looks at how many ports exist for instance X, and creates that many items in the XML. Since both ports are created with the same instance-id as the port's device_id, the underlying virt layer is tricked into thinking there are supposed to be two interfaces.

edit flag offensive delete link more
add a comment
0

answered 2013-04-12 04:33:38 -0600

snaiksat
1

Yes, there is an exception due to time out, and the earlier VM creation failed. But this is the log from the same node, so I am not sure how scheduling on another HV plays into this.

To me, the basic question is why is nova creating two interfaces for this VM (if you see the libvirt conf for this VM). It's not the case that nova creates one interface, but results in two ports being on the Quantum side. Nova is actually creating two interfaces and requesting two ports for those.

edit flag offensive delete link more
add a comment
0

answered 2013-04-11 15:23:43 -0600

danwent
1

Ok, could you possibly post the portions of the relevant nova-compute + quantum logs, along with a port-show for each of the duplicate ports (or some other way of us seeing the UUIDs of the duplicate ports, and the instance id of the vm)? thanks.

On Thu, Apr 11, 2013 at 12:36 AM, Sumit Naiksatam < question225158@answers.launchpad.net > wrote:

> Question #225158 on quantum changed: https://answers.launchpad.net/quantum/+question/225158 (https://answers.launchpad.net/quantum...)

> Sumit Naiksatam posted a new comment: Here is the nova log as we reproduced this: http://paste.openstack.org/show/35783/

> Sorry Dan, I could not add any additional log statements in this run, but I am looking at the setup.

> You received this question notification because you are an answer contact for quantum.

~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dan Wendlandt Nicira, Inc: http://www.nicira.com twitter: danwendlandt ~~~~~~~~~~~~~~~~~~~~~~~~~~~

edit flag offensive delete link more
add a comment
0

answered 2013-04-11 07:35:37 -0600

snaiksat
1

Here is the nova log as we reproduced this: http://paste.openstack.org/show/35783/

Sorry Dan, I could not add any additional log statements in this run, but I am looking at the setup.

edit flag offensive delete link more
add a comment
0

answered 2013-04-03 00:20:27 -0600

danwent
1

Thanks for the update Sumit.

Not sure if you saw my comments above, but it might be good to insert some logging statements into the nova code that talks to quantum, otherwise i'm not sure that we'll have much more information to go on than the info already posted in the question. The quantum logs would be useful as well.

Dan

On Tue, Apr 2, 2013 at 4:06 PM, Sumit Naiksatam < question225158@answers.launchpad.net > wrote:

> Question #225158 on quantum changed: https://answers.launchpad.net/quantum/+question/225158 (https://answers.launchpad.net/quantum...)

> Sumit Naiksatam posted a new comment: This happened again today, however the automation run got wiped out after the test failure (on account of the dual IPs). We have changed the automation script so that this does not happen the next time. Will investigate once we reproduce this again and have live system. Sorry, I am not providing too much additional information, but I just wanted to report back.

> You received this question notification because you are an answer contact for quantum.

~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dan Wendlandt Nicira, Inc: http://www.nicira.com twitter: danwendlandt ~~~~~~~~~~~~~~~~~~~~~~~~~~~

edit flag offensive delete link more
add a comment
0

answered 2013-04-02 23:04:28 -0600

snaiksat
1

This happened again today, however the automation run got wiped out after the test failure (on account of the dual IPs). We have changed the automation script so that this does not happen the next time. Will investigate once we reproduce this again and have live system. Sorry, I am not providing too much additional information, but I just wanted to report back.

edit flag offensive delete link more
add a comment
0

answered 2013-03-28 00:43:45 -0600

xinxin-shu
11 ●1 ●2 ●3

hi Aaron, output of "ps -aef | grep nova"(eight compute hosts) :

nova 13998 1 0 02:04 ? 00:05:16 /usr/bin/python /usr/bin/nova-compute --config-file=/etc/nova/nova.conf --config-file=/etc/nova/nova-compute.conf nova 12501 1 0 02:04 ? 00:04:59 /usr/bin/python /usr/bin/nova-compute --config-file=/etc/nova/nova.conf --config-file=/etc/nova/nova-compute.conf nova 24958 1 0 02:04 ? 00:05:54 /usr/bin/python /usr/bin/nova-compute --config-file=/etc/nova/nova.conf --config-file=/etc/nova/nova-compute.conf nova 22490 1 0 02:04 ? 00:05:20 /usr/bin/python /usr/bin/nova-compute --config-file=/etc/nova/nova.conf --config-file=/etc/nova/nova-compute.conf nova 11333 1 0 02:04 ? 00:04:54 /usr/bin/python /usr/bin/nova-compute --config-file=/etc/nova/nova.conf --config-file=/etc/nova/nova-compute.conf nova 8137 1 0 02:04 ? 00:05:00 /usr/bin/python /usr/bin/nova-compute --config-file=/etc/nova/nova.conf --config-file=/etc/nova/nova-compute.conf nova 17029 1 0 02:04 ? 00:05:19 /usr/bin/python /usr/bin/nova-compute --config-file=/etc/nova/nova.conf --config-file=/etc/nova/nova-compute.conf nova 12178 1 0 02:04 ? 00:05:50 /usr/bin/python /usr/bin/nova-compute --config-file=/etc/nova/nova.conf --config-file=/etc/nova/nova-compute.conf

my internal network(quantum net-show):

+---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | True | | id | 36a78a32-9488-410d-86d2-22578767d6c9 | | name | demo-int-net | | provider:network_type | vlan | | provider:physical_network | DemoNet | | provider:segmentation_id | 100 | | router:external | False | | shared | False | | status | ACTIVE | | subnets | ff4ebc46-5e94-4884-90da-dd66e9b3572b | | tenant_id | ebacbc9c99f84607920b2ac749608623 | +---------------------------+--------------------------------------+

my external network:

+---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | True | | id | 5613fb31-8c0d-430e-8099-0a67b9a65c52 | | name | demo-ext-net | | provider:network_type | vlan | | provider:physical_network | DemoNet | | provider:segmentation_id | 1 | | router:external | True | | shared | False | | status | ACTIVE | | subnets | 57350a07-2329-4cae-9423-607a1cb18ed3 | | tenant_id | ebacbc9c99f84607920b2ac749608623 | +---------------------------+--------------------------------------+

edit flag offensive delete link more
add a comment
« previous 1...1 2 3 4...4 next »

## Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

[hide preview]

# Get to know Ask OpenStack

Resources for moderators

## Question Tools

1 follower

## Stats

Asked: 2013-03-26 01:09:38 -0600

Seen: 229 times

Last updated: Jul 03 '13

## Related questions

A question about ovs-vlan-test

can't ping floating ip

about l3 agent

noVNC support for Google Chrome 14+

Connecting OVS interfaces to a bond with VLAN tagging

Set up devstack with two ethernet interfaces

Keystone did not start

what's the _junk in codes?

Instance failed with "binding_failed" error on RHEL7 Openstack Icehouse [closed]

neutron router iptables not hit

( 2016-11-08 07:30:30 -0600 )edit
none